52 Commits

Author SHA1 Message Date
Abin Simon
a3d573973e
Drop GroupsMap in favor of api.GetAllIDsAndNames (#4967)
Follow up from https://github.com/alcionai/corso/pull/4955#discussion_r1440922835

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [ ]  Unit test
- [x] 💚 E2E
2024-01-08 14:19:33 +00:00
Keepers
963d78b75e
move service api to client methods (#4701)
sdk consumers need a way to configure the graph api client options when using the services api.  This is the first in a two-part change to expose those options.  This step moves the api to a client-based set of methods instead of free functions.  The next step will add client configuration to the service client.

---

#### Does this PR need a docs update or release note?

- [x] 🕐 Yes, but in a later PR

#### Type of change

- [x] 🌻 Feature

#### Test Plan

- [x] 💚 E2E
2023-11-17 01:03:57 +00:00
Keepers
4c72e9eab7
use clues new|wrap|stackWC() (#4684)
replace all cases of new|wrap|stack(...).WithClues(ctx) with the flattened new|wrap|stackWC(ctx, ...) functions introduced in the latest clues bump.

Other changes added:
* remove WithClues builders when the error producer already called it.
* corrected some usages of ictx within loops.

no logic changes, just cleanup.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-11-15 23:01:08 +00:00
Keepers
5e0014307c
remove ctx embedding for counter (#4497)
#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-10-31 16:15:05 +00:00
Keepers
9e40d88265
expose sitesInGroup to sdk consumers (#4532)
exposes the m365 api call to get all sites in a group to the sdk service interface.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🌻 Feature

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-10-24 00:09:48 +00:00
Abhishek Pandey
9bedee9c72
Wire service isolation changes into SDK code (#4186)
<!-- PR description-->

Wires `IsServiceEnabled` code into `UserHasMailbox`, `UserHasDrive`, and `UserGetMailboxInfo` SDK APIs.

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
*  https://github.com/alcionai/corso/issues/3844

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2023-09-08 07:58:38 +00:00
Keepers
af5d98e182
Cleanup services (#4059)
Some quick tech-debt splitting up of the code in services/m365

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-08-23 16:46:28 +00:00
Keepers
20675dbcf7
add the groups resources service addition (#4053)
Adds groups to the m365 services api.  Also adds a bit of touchups/cleanups on the side.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🌻 Feature

#### Issue(s)

* #3989

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-08-18 20:10:56 +00:00
Keepers
3613cb2aa0
look up restore resource if specified (#3853)
If the restore configuration specifies a protected resource as a restore target, use that as the destination for the restore.  First step is to ensure the provided target can be retrieved and identified.

---

#### Does this PR need a docs update or release note?

- [x] 🕐 Yes, but in a later PR

#### Type of change

- [x] 🌻 Feature

#### Issue(s)

* #3562

#### Test Plan

- [x]  Unit test
2023-07-26 17:24:53 +00:00
Keepers
0d6b08204d
allow users to limit page size (#3875)
allows cli users to limit the page size of delta queries by calling a new hidden flag: --delta-page-size.
This also adds the control.Options struct to the api client, so that configurations such as this can be easily handed into, and used by, the client.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🌻 Feature

#### Test Plan

- [x] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2023-07-22 00:00:09 +00:00
Keepers
70bc241831
Add concurrency init to api (#3749)
expose middleware init via api layer
2023-07-03 19:19:21 -06:00
Keepers
cf495a6c8f
add authErr to users mailbox check (#3744)
license issues can produce an authenticationError when retrieving a user's inbox.  This catches that error and treats it as a "mailbox not availble" condition.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🐛 Bugfix

#### Issue(s)

* #3743

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-07-03 19:31:35 +00:00
Keepers
56151a82eb
remove discovery pkg (#3677)
Currently unused, and functionality is largely duplicated by the services/m365 package.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [ ] 🧹 Tech Debt/Cleanup

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-07-01 13:52:57 +00:00
ashmrtn
dcf02f6256
Minor refactoring of graph error checking code (#3686)
Minor code cleanup to error checking and calling code:
- Remove unused function
- Update error check for folder not found
- Remove duplicate check in SDK function

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* fixes #3684

#### Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-06-28 21:21:28 +00:00
Keepers
0451e933d5
handle no sharepoint license err (#3673)
adds handling for graph api responses
where the tenant does not have an active
sharepoint license.  For user's drives, this
will fall under the same "service not enabled"
behavior we use to determine if the drive can
get backed up.  For sites, this causes the
"get all sites" call to return a serviceNotEnabled error.

---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included

#### Type of change

- [x] 🐛 Bugfix

#### Issue(s)

* #3671

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-06-27 02:26:47 +00:00
Keepers
2f6d731993
rename connector -> m365 (#3600)
renames /internal/connector to /internal/m365.  No logic changes in this PR.  Only the dir rename, import renames, and one linter shadowing rename.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #1996

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-06-13 18:35:39 +00:00
Keepers
cbbc8d2f6c
drive-focused api refactoring prep (#3471)
setting up the drive api calls into files/spaces that will cascade naturally to the addition of an api client for users and sites.  contains some partial implementation of these clients, which will get completed in the next pr.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #1996

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-06-02 20:16:47 +00:00
Keepers
cf849e9c5d
fix has drive and has mailbox conditions (#3473)
#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🐛 Bugfix

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-05-23 18:08:34 +00:00
jules
11752d869c
Should not error out on "My site not found" (#3469)
This code should not error out on `Unable to retrieve user's mysite URL`.

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2023-05-22 17:54:44 +00:00
jules
7057d94e35
Adding two new apis to corso for validating a user has exchange or one drive is enabled (#3447)
Adds two new apis that expose calls to check whether a user has one drive or exchange enabled

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2023-05-18 20:25:32 +00:00
Vaibhav Kamra
084b519766
Introduce a UserCompatNoInfo call (#3257)
Instead of returning `Info` in `User`, callers should call the `GetUserInfo` API to get extended user information.

This allows the caller to decide whether info is needed *and* control parallelism for these queries.

This change introduces a `UserCompatNoInfo` call that allows us to make the switch without breaking existing `UserCompat` references. We will remove this API after callers have switched.

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [ ]  Unit test
- [x] 💚 E2E
2023-04-28 15:22:55 +00:00
Keepers
c95a07660a
ditch userInfo middle struct (#3206)
we don't need a m365.UserInfo when we already have an api.UserInfo

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-04-25 19:05:07 +00:00
Keepers
62daf10213
migrate onedrive using prefix collection (#3122)
#### Does this PR need a docs update or release note?

- [x] 🕐 Yes, but in a later PR

#### Type of change

- [x] 🌻 Feature

#### Issue(s)

* #2825

#### Test Plan

- [x] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2023-04-24 17:36:10 +00:00
neha_gupta
de1ae9f30a
Shared mailbox (#3106)
<!-- PR description-->

Handle user's API to provide user purpose of users

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [x] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2023-04-20 18:01:12 +00:00
zackrossman
5166e61115
[Feature] Move connector api module from internal -> pkg (#3166)
<!-- PR description-->

This PR moves the `api` module from `src/internal/connector/discovery`
to `src/pkg/services/m365/api` so that the Client can be reused.

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic
words" - "closes, fixes" to auto-close the Github issue. -->
* #ALC-2214

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E

---------

Co-authored-by: aviator-app[bot] <48659329+aviator-app[bot]@users.noreply.github.com>
2023-04-19 11:17:20 -07:00
Keepers
3cc29649ed
Add GetUserInfo to services/m365 (#2991)
Adds an interface for GetUserInfo, which gets
corso-specific metadata for a single user.  Also
does some refactoring around discovery for
better interface consistency, both as variables
and as access layers.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🌻 Feature

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-04-07 00:36:50 +00:00
Keepers
0d521bb5a7
move sharepoint onto map-based lookup (#2958)
Refactors the sharepoint site lookup to use the
id-to-name maps.  This has a momentary regression
that will get solved in the next PR: we no longer
match on weburl suffixes, and instead require
a complete match.

Also, migrates the sharepoint lookup code out
of GC and into Discovery.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🌻 Feature

#### Issue(s)

* #2825
* #1995
* #1547

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-04-05 19:38:22 +00:00
Keepers
c949c333e4
fix user validation in factory (#3031)
#### Type of change

- [x] 🐛 Bugfix

#### Test Plan

- [x] 💪 Manual
- [x] 💚 E2E
2023-04-05 00:54:04 +00:00
Keepers
2341d61842
introduce id-name lookup maps (#2955)
Adds two maps to resource-owner handling:
id-to-name and name-to-id.  Expectation is that
these maps will either get populated by a caller
as a pre-process before initializing the gc client, or
gc will (later pr) be able to look up the owner and
populate those maps itself.  The maps are
used to set the selector id and name for iface
compliance.  Only supported by exchange in this PR.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🌻 Feature

#### Issue(s)

* #2825

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-04-04 20:21:58 +00:00
Keepers
fbb867a08d
add clues.Hide to various logs (#2994)
Begins hiding PII with clues.Hide injections in our structured logging aggregators.  This is a Non-
comprehensive change, and improvements here will be ongoing.

---

#### Does this PR need a docs update or release note?

- [x] 🕐 Yes, but in a later PR

#### Type of change

- [x] 🤖 Supportability/Tests

#### Issue(s)

* #2024

#### Test Plan

- [x] 💪 Manual
- [x] 💚 E2E
2023-04-04 16:59:31 +00:00
Keepers
9d73d0c8e2
replace errors.* with clues.* (#2924)
Mostly find/replace on errors.N and errors.W. Also turns all wrapf into wrap, and removes as many errorf calls as possible.

Might follow up with a linter to enforce this change.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #1970

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-03-28 04:01:54 +00:00
Aarti Jivrajani
120eb0d256
Add a helper function that returns site ID and site URL in a Site struct (#2778)
This PR adds a helper function that returns a newly introduced `Site` struct that has a site ID and site webURL for a given m365 tenant.

---

#### Does this PR need a docs update or release note?
- [X]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [X] 🌻 Feature

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #2777 

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [X]  Unit test

```
--- PASS: TestM365IntegrationSuite (2.49s)
    --- PASS: TestM365IntegrationSuite/TestSites (1.82s)
        --- PASS: TestM365IntegrationSuite/TestSites/site (0.00s)
        --- PASS: TestM365IntegrationSuite/TestSites/site#01 (0.00s)
        --- PASS: TestM365IntegrationSuite/TestSites/site#02 (0.00s)
        --- PASS: TestM365IntegrationSuite/TestSites/site#03 (0.00s)
        --- PASS: TestM365IntegrationSuite/TestSites/site#04 (0.00s)
        --- PASS: TestM365IntegrationSuite/TestSites/site#05 (0.00s)
        --- PASS: TestM365IntegrationSuite/TestSites/site#06 (0.00s)
        --- PASS: TestM365IntegrationSuite/TestSites/site#07 (0.00s)
        --- PASS: TestM365IntegrationSuite/TestSites/site#08 (0.00s)
        --- PASS: TestM365IntegrationSuite/TestSites/site#09 (0.00s)
        --- PASS: TestM365IntegrationSuite/TestSites/site#10 (0.00s)
        --- PASS: TestM365IntegrationSuite/TestSites/site#11 (0.00s)
        --- PASS: TestM365IntegrationSuite/TestSites/site#12 (0.00s)
        --- PASS: TestM365IntegrationSuite/TestSites/site#13 (0.00s)
        --- PASS: TestM365IntegrationSuite/TestSites/site#14 (0.00s)
        --- PASS: TestM365IntegrationSuite/TestSites/site#15 (0.00s)
        --- PASS: TestM365IntegrationSuite/TestSites/site#16 (0.00s)
        --- PASS: TestM365IntegrationSuite/TestSites/site#17 (0.00s)
```
2023-03-14 01:00:11 +00:00
ashmrtn
aef90f8b86
Remove GraphConnector user population (#2582)
## Description

Don't populate users when GraphConnector instance is created

## Does this PR need a docs update or release note?

- [x]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No 

## Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

## Issue(s)

* #1994

## Test Plan

- [x] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-02-28 21:26:12 +00:00
Keepers
9e783efe3a
fault package funcs rename (#2583)
## Description

Renaming the funcs in the fault
package to be more clear about
their purpose and behavior.  Largely
just find&replace changes, except
for fault.go and the fault examples.

## Does this PR need a docs update or release note?

- [x]  No 

## Type of change

- [x] 🧹 Tech Debt/Cleanup

## Issue(s)

* #1970

## Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-02-25 03:29:02 +00:00
ashmrtn
634076328c
hide fault param in Users call (#2483)
## Description

Temporary change but based on the branch with the calendar ID fix. This PR should not be merged

## Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No 

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-02-13 20:42:18 +00:00
Keepers
680ec2b751
adding fault/clues to non-service connector pkgs (#2380)
## Does this PR need a docs update or release note?

- [x]  No 

## Type of change

- [x] 🧹 Tech Debt/Cleanup

## Issue(s)

* #1970

## Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-02-13 16:54:09 +00:00
Keepers
dad6776861
replace multierror with fault in setSites (#2377)
## Does this PR need a docs update or release note?

- [x]  No 

## Type of change

- [x] 🧹 Tech Debt/Cleanup

## Issue(s)

* #1970

## Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-02-09 18:11:59 +00:00
Keepers
faad5d35a4
refactor structured errors/logs in pkg (#2323)
## Description

Refactors errors and logs to support structured
data throughout the smaller packages within
pkg/...  Larger packages will come later.

## Does this PR need a docs update or release note?

- [x]  No 

## Type of change

- [x] 🧹 Tech Debt/Cleanup

## Issue(s)

* #1970

## Test Plan

- [x]  Unit test
2023-02-06 20:45:09 +00:00
Keepers
24911a358b
quick graph client refactor (#2301)
## Description

A couple minor refactors.  First, combines the
graph/service_helper.go and graph/service.go
files into a single file (just lift'n'shift, no logic
changes).  Second, renames CreateHTTPClient
to just HTTPClient.  Third, replaces the new
LargeItemClient constructor with extensible
options for the http client, starting with a
NoTimeout option.  This will provide longer
term maintenance over multiple constructors.

This change sets up QoL for building multiple services
and clients within api layer handlers, which will appear
in an immediately following PR.

## Does this PR need a docs update or release note?

- [x]  No 

## Type of change

- [x] 🧹 Tech Debt/Cleanup

## Issue(s)

* #2299

## Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-01-27 19:34:00 +00:00
Vaibhav Kamra
bdb7f2b109
Re-use client for OneDrive item downloads (#2276)
## Description

This PR reintroduces the changes from #2266 with a change to *not* reset the transport
when initializing the shared client. 

Doing so was removing the retry and other middlewares
and also resulting in throttled requests being masked as success

Also - we now decorate our download traffic with an ISV tag as recommended [here](https://learn.microsoft.com/en-us/sharepoint/dev/general-development/how-to-avoid-getting-throttled-or-blocked-in-sharepoint-online#how-to-decorate-your-http-traffic)

## Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [x] 🕐 Yes, but in a later PR
- [ ]  No 

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #2266 

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [ ]  Unit test
- [x] 💚 E2E
2023-01-26 15:05:12 +00:00
Vaibhav Kamra
e9354a8429
Revert "add itemClient for re-usable od item downloads" (#2273)
Reverts alcionai/corso#2266. Am seeing issues on data restore that need to be debugged.
2023-01-26 07:52:56 +00:00
Keepers
bda8a5c60c
add itemClient for re-usable od item downloads (#2266)
## Description

onedrive currently constructs a new http client
for every file it downloads.  This causes the OS
to generate extra sockets, and hang onto them
after the download is complete.  Replacing these
one-off clients with a singular, re-used client-
which is the behavior and standard suggested
for golang http clients- minimizes system
resource consumption.

## Does this PR need a docs update or release note?

- [x]  No 

## Type of change

- [x] 🐛 Bugfix

## Issue(s)

* closes #2262

## Test Plan

- [x] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2023-01-26 04:27:52 +00:00
Keepers
b3d4b4687b
discovery api, filter guest and external users (#2188)
## Description

Adds the api client pkg pattern to the connector/
discovery package.  Most code changes are plain
lift-n-shift, with minor clean-ups along the way.

User retrieval is now filtered to only include
member and on-premise accounts.

## Does this PR need a docs update or release note?

- [x]  Yes, it's included

## Type of change

- [x] 🌻 Feature

## Issue(s)

* #2094

## Test Plan

- [x] 💪 Manual
- [x]  Unit test
2023-01-20 20:09:59 +00:00
Keepers
edc4426b9c
Fix --users "*' not finding user in tenant (#2033)
## Description

DataCollections validation step was still using the full resourceOwner list in the selector to validate every backup, rather than checking only the DiscreteOwner.

## Does this PR need a docs update or release note?

- [x]  No 

## Type of change

- [x] 🐛 Bugfix

## Issue(s)

* #1617

## Test Plan

- [x]  Unit test
2023-01-05 00:29:21 +00:00
Keepers
8a29c52cdc
move graphService to graph as Service (#1790)
## Description

Relocates the graphService struct to graph as
the Service struct.  Replaces GC's embedded
graphService with a graph.Servicer reference.

## Type of change

- [x] 🐹 Trivial/Minor

## Issue(s)

* #1725

## Test Plan

- [x]  Unit test
- [x] 💚 E2E
2022-12-14 01:39:00 +00:00
Keepers
a6aa86ce5c
add a weburl to siteid reducer in gc (#1671)
## Description

Adds a func in graphConnector that reduces
siteIDs and webURLs into a set of siteIDs.  This
will be used by callers such as the CLI to
generate id-based selectors for sites even if
they handle webURLs as an alternative id.

## Type of change

- [x] 🌻 Feature

## Issue(s)

* #1616

## Test Plan

- [x]  Unit test
2022-12-08 15:38:24 +00:00
Vaibhav Kamra
5533ada9be
Refactor user query and return user display name (#1679)
## Description

Move user discovery into a `discovery` package. Also use that to return user display name from `m365.Users()` call.

This will also allow us to remove `Users` from the GC struct going forward.

If this pattern works - will follow up with a similar change for `Sites()`.

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-12-06 01:45:51 +00:00
ashmrtn
27d0980526
Split backups by (resource owner, service) at CLI layer (#1609)
## Description

Sketch of how to split backups by (resource owner, service) for exchange. Needs more support from selectors package to be complete.

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

* closes #1505

## Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2022-11-30 21:50:49 +00:00
Keepers
07a8d13d1e
adds tenant site lookup on new graph conn (#1540)
## Description

Adds a lookup for all tenant site ids when a new
graph connector is created.  Also adds an enum
to flag which resource set (users, sites, or all)
that the connector should initialize.

## Type of change

- [x] 🌻 Feature

## Issue(s)

* #1506

## Test Plan

- [x] 💚 E2E
2022-11-18 00:31:41 +00:00
Eno Thereska
fb46f5c7f3
Export call to get user ids (#1069)
## Description

Exposing user IDs in addition to user emails.

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [X] 🐹 Trivial/Minor

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [X]  Unit test
- [ ] 💚 E2E
2022-10-17 18:17:39 +00:00