677 Commits

Author SHA1 Message Date
Keepers
e72fa49018
retry on gateway timeout (#3271)
We don't currently retry on graph API gateway timeout/bad gateway responses.  This change adds those statuses to our retryable codes.

Also adds some qol clues tracking and error formatting.

---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included

#### Type of change

- [x] 🐛 Bugfix

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-05-02 17:47:45 +00:00
ashmrtn
dcb9d81f3f
Fix progress folder display for Exchange (#3219)
---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

* closes #3013

#### Test Plan

- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-04-29 00:19:54 +00:00
ashmrtn
619bcd6eb8
Check for new and older content URLs (#3263)
Some requests appear to be returning an older version of the content URL key. This was causing corso backup failures. This commit expands the logic to check for both the new and old keys for the URL.

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

* closes #3262 

#### Test Plan

- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-04-28 23:06:10 +00:00
Keepers
1cd592d216
minor errs cleanup (#3234)
some code cleanup that happened while investigating a bug repro

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-04-27 16:57:38 +00:00
ashmrtn
b8e868c05b
Return SharePoint exclude information (#3242)
Allows proper deletion of files and also fixes file moves.

---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #3240

#### Test Plan

- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-04-27 16:25:12 +00:00
Keepers
f2f010b9f6
cache created folders when restoring drive items (#3220)
drive items aren't currently getting cached after creation.  Instead, we do a lookup on the folder name every time we walk the folder hierarchy toward the next child to restore.  Normally this would work fine, but apparently if the folder is named `folder`, then graph api returns nothing, causing the process to fail when repeatedly creating the folder.

---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included

#### Type of change

- [x] 🐛 Bugfix

#### Test Plan

- [x] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2023-04-26 20:53:26 +00:00
Vaibhav Kamra
6395dcbe39
Add parent path information to malware detected items (#3221)
This allows identifying the file location when an item is detected with malware

---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #3112 

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-04-26 17:17:52 +00:00
Keepers
edef23bfc4
errant cleanups before incremental support (#3201)
a handful of errant cleanups to help the PR for adding incremental support to sharepoint.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #3136

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-04-25 21:09:41 +00:00
Keepers
0ae1a04097
use nanosecond folders for testing (#3205)
we've been failing on folder name collision, particularly in this package.  Probably because it's using second-granularity folder names.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🐛 Bugfix
- [x] 🤖 Supportability/Tests

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-04-25 20:33:39 +00:00
Keepers
c95a07660a
ditch userInfo middle struct (#3206)
we don't need a m365.UserInfo when we already have an api.UserInfo

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-04-25 19:05:07 +00:00
Abin Simon
08c625b788
Show names in OneDrive restore progressbar (#3208)
Related https://github.com/alcionai/corso/pull/3168

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* https://github.com/alcionai/corso/issues/3113

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-04-25 08:08:14 +00:00
Abhishek Pandey
c38a431079
Add --disable-concurrency-limiter cli flag (#3203)
<!-- PR description-->
The concurrency limiter middleware is enabled by default for exchange backups. Adding a hidden --disable-concurrency-limiter flag to disable this middleware.

Reasons for adding this flag:
We have done limited perf testing with the concurrency limiter. In addition, our understanding of exchange concurrency limits is also a bit fuzzy. This flag acts as a killswitch in case we start to see perf regressions in prod.

I see this as a temporary flag. Ideally we would never have to use it. Also, once we have a better way to configure concurrency limiter, we can eliminate this flag.


---

#### Does this PR need a docs update or release note?

- [ ]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature


#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->

- [ ]  Unit test
2023-04-25 01:22:12 +00:00
Keepers
7b1ce22a64
enable user pn->id migrations for reporef prefix (#3200)
Enables migrations of all user-based repoRef prefixes away from the user's PrincipalName and onto the user's ID for a more stable reference.

---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included

#### Type of change

- [x] 🌻 Feature

#### Issue(s)

* #2825

#### Test Plan

- [x] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2023-04-24 22:37:11 +00:00
Keepers
b331f38654
fixup mailbox info (#3189)
#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🐛 Bugfix
- [x] 🧹 Tech Debt/Cleanup

#### Test Plan

- [x] 💪 Manual
- [x] 💚 E2E
2023-04-24 20:06:41 +00:00
Abhishek Pandey
41f742eba2
Add concurrency limiter middleware package (#3182)
<!-- PR description-->


Graph service only allows [4 concurrent requests per exchange
mailbox.](https://learn.microsoft.com/en-us/graph/throttling-limits#outlook-service-limits)
We are currently not honoring this limit in corso. This causes 429
errors with “Application is over its MailboxConcurrency limit”. This PR
introduces a concurrency limiter middleware. This middleware is
selectively added for exchange backups only.


---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic
words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E

---------

Co-authored-by: aviator-app[bot] <48659329+aviator-app[bot]@users.noreply.github.com>
2023-04-24 11:16:20 -07:00
Keepers
62daf10213
migrate onedrive using prefix collection (#3122)
#### Does this PR need a docs update or release note?

- [x] 🕐 Yes, but in a later PR

#### Type of change

- [x] 🌻 Feature

#### Issue(s)

* #2825

#### Test Plan

- [x] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2023-04-24 17:36:10 +00:00
Keepers
6405c8246e
move parallelism logs/checks upstream (#3003)
The fetch paralellism checks and logs occur on
every item streamed from GC.  This is a bit chatty,
and has been moved upstream in the process for
a more centralized behavior.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Test Plan

- [x] 💪 Manual
- [x]  Unit test
2023-04-21 20:23:41 +00:00
Keepers
cfe8d576b8
cleaning up some gc/selector test construction (#3175)
Duplicate of https://github.com/alcionai/corso/pull/3049, fixing a git rebase screwup.
2023-04-21 19:50:31 +00:00
Keepers
d5fac8a480
custom http wrapper for item downloads (#3132)
the http client customized by the graph client for use in downloading files doesn't include our middleware like expected. This introduces a new http client wrapper that populates the roundtripper with a middleware wrapper that can utilize our kiota middleware in the same way as the graph client does.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🌻 Feature
- [x] 🐛 Bugfix
- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #3129

#### Test Plan

- [x] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2023-04-20 23:20:23 +00:00
neha_gupta
de1ae9f30a
Shared mailbox (#3106)
<!-- PR description-->

Handle user's API to provide user purpose of users

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [x] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2023-04-20 18:01:12 +00:00
Vaibhav Kamra
2050a953fe
Handle SyncStateInvalid error when ImmutableIDs are enabled after initial backup (#3176)
Handles the `SyncStateInvalid` error if a delta URL constructed without the immutable ID header is used.

This will fallback to full backup similar to how we handle other sync errors for delta queries.

Also adds a hidden flag so we can test this using the CLI.

*Note*: Enabling immutable IDs for exchange is not yet supported

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-04-19 23:52:08 +00:00
Abin Simon
2121b40293
Show name instead of id in onedrive backup progress (#3168)
Fixes https://github.com/alcionai/corso/issues/3113

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* https://github.com/alcionai/corso/issues/3113

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-04-19 21:12:53 +00:00
ashmrtn
f01e25ad83
Redo folder generation in backup details (#3140)
Augment the folder backup details entries with
LocationRef and some information about what
data type generated the entry. Also add top-level
container information like drive name/ID if
applicable

Refactor code to do folder generation to make it
more contained and require less parameters to add
entries to backup details

Important changes are in details.go. All other
changes are just to keep up with the slightly
modified function API or update tests

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

* closes #3120
* closes #2138

#### Test Plan

- [x] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-04-19 20:27:55 +00:00
Keepers
09ef350bc0
retry on "connection reset by peer" (#3138)
retry when we receive a "connection reset by peer" error

---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included

#### Type of change

- [x] 🐛 Bugfix

#### Issue(s)

* #3129

#### Test Plan

- [x] 💪 Manual
- [x]  Unit test
2023-04-19 18:51:29 +00:00
zackrossman
5166e61115
[Feature] Move connector api module from internal -> pkg (#3166)
<!-- PR description-->

This PR moves the `api` module from `src/internal/connector/discovery`
to `src/pkg/services/m365/api` so that the Client can be reused.

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic
words" - "closes, fixes" to auto-close the Github issue. -->
* #ALC-2214

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E

---------

Co-authored-by: aviator-app[bot] <48659329+aviator-app[bot]@users.noreply.github.com>
2023-04-19 11:17:20 -07:00
Abin Simon
a4ff93bd47
Update permissions diff computation (#3117)
Previously we were using permission ids to check if two permissions
are equal. While we were aware the permission ids can repeat across
different folder hierarchies we were not aware that this was also the
case within a single folder tree if we delete and break the
inheritance chain.

Assume the following scenario:

1. Create folder `a` and `a/b`
2. Assign user1:read permission to `a`
3. Remove the inherited permission from `a/b`
4. Assign user1:write permission to `a/b`

In this case both the user1:read and user1:write will get the same permissions id.<!-- PR description-->

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* fixes https://github.com/alcionai/corso/issues/3116

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-04-19 17:06:14 +00:00
ashmrtn
ba724ac63d
Add immutable ID header to Get requests (#3157)
This allows using immutable IDs for Exchange objects. It does not add this header to put requests as we don't record the IDs of restored objects anywhere.

May slightly increase performance of events backups as
it now sets the delta query page size to 200 like with
other requests

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #2861

#### Test Plan

- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-04-19 16:16:49 +00:00
ashmrtn
9664afaa12
Restore with OneDrive/SharePoint ID (#3151)
Allow restoring OneDrive and SharePoint items by
M365 ID as well as file name and ShortRef

Does create an edgecase where attempting to
restore an item named the same as another item's
M365 ID causes both items to be restored

Moves the consts for OneDrive file suffixes to a
different package so that the selectors package
can also import them without a cycle

---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

* closes #3142

#### Test Plan

- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-04-19 15:30:03 +00:00
Keepers
d5eee4479d
move largeItem client constructor into new GC (#3130)
It's an unnecessary paramter.  Plus, this sets up the next pr to have a smaller footprint.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #3129

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-04-18 18:52:22 +00:00
Keepers
86f9d60bf7
relocate connector mocks (#3048)
It makes more sense for mock data types to be
owned by the service package, not by the connector package as a whole.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-04-17 17:54:19 +00:00
Keepers
c90f587769
clean unused options, reduce folders call (#3127)
Cleans out some unused exchange options builders.
Also tops the mailFolders userinfo check to a single folder, since we only need one result to verify access.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-04-17 17:22:27 +00:00
ashmrtn
1826b66e8d
Populate and persist OneDrive LocationRef (#3111)
Leverage structs from previous PRs to
populate and persist the LocationRef
for OneDrive

Right now the source of the
LocationRef is still based off the
RepoRef path, but this is in the
OneDrive code only

Manually tested
* updating a subfolder path when parent folder was moved
* populating LocationRef when the base backup didn't have LocationRefs when nothing was changed in OneDrive
* populating LocationRef when the base backup didn't have LocationRefs when something was changed in OneDrive

---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

* closes #2486

#### Test Plan

- [x] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-04-14 21:31:16 +00:00
Keepers
b0869fad4d
add sensitive info flag to cli (#3000)
The sensitive-info flag allows CLI users to specify the Concealer algorithm used to handle PII and
other sensitive info in logging.  Default alg is 'hash', which produces a truncated hmacsha256 string.
'mask' for flat '***' replacement and 'plaintext'
are also available.

SDK consumer are expected to configure clues
themselves, rather than specifying a config option.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🌻 Feature

#### Issue(s)

* #2024

#### Test Plan

- [x] 💪 Manual
- [x]  Unit test
2023-04-14 17:07:14 +00:00
Keepers
482b690e06
fix overlapping bases after fallback union (#3118)
In a case where the current manifests and the
fallback contain both distinct and overlapping
categories, the end result contains multiple
bases for the same category.  Verifydistinctresons
didn't catch this because it includes the resource owner.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🐛 Bugfix

#### Issue(s)

* #2825

#### Test Plan

- [x]  Unit test
2023-04-14 16:29:29 +00:00
Abin Simon
6c64eb0c16
Fetch individual attachments in case of full download failure (#3086)
Failure can happen as the list of attachment and they data can get
pretty big.

---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-04-13 05:51:58 +00:00
Abhishek Pandey
ec064e539b
Add misc typo fixes (#3006)
<!-- PR description-->

---

#### Does this PR need a docs update or release note?


- [ ]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->

- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
2023-04-12 23:30:38 +00:00
Keepers
da8ac5cdbc
moving code around, no logic changes (#3025)
#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #2024

#### Test Plan

- [x]  Unit test
2023-04-12 17:16:05 +00:00
Abin Simon
b1606466c2
Compute parent permissions from available metadata (#3022)
Previously we were querying Graph to get the current permissions and then using that to compute the permission diff. This changes that to use the metadata that we have available locally to do the computation avoiding a graph call to get permissions.

<!-- PR description-->

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* fixes https://github.com/alcionai/corso/issues/2976

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2023-04-12 12:22:35 +00:00
Keepers
30c86797d3
always include delta metadata, additional logging (#3092)
Ensure exchange always includes the delta metadata entry, even if it has zero deltaURLs to record.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🐛 Bugfix

#### Issue(s)

* fixes #3019

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-04-12 00:43:55 +00:00
Keepers
50e92b65c6
add Concealer compliance to paths (#3017)
Adds compliance with clues.Concealer to the paths
package.  Also introduces a new struct in the same
space: Elements, which is a thin wrapper around
a slice of strings so that subsections of a path
or builder can carry the same pii behavior
without additional work on the consumer's end.

---

#### Does this PR need a docs update or release note?

- [x] 🕐 Yes, but in a later PR

#### Type of change

- [x] 🌻 Feature
- [x] 🤖 Supportability/Tests

#### Issue(s)

* #2024

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-04-11 22:04:34 +00:00
ashmrtn
cae3debd28
Lint for using suite.T().Run instead of testify's suite.Run (#2664)
Includes fixups for calls that need updated

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Test
- [x] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #2373

#### Test Plan

- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-04-11 16:39:39 +00:00
Keepers
21cd7210cb
remove weburl suffix support from sharepoint (#3082)
We're dropping support for selection-by-url-suffix in sharepoint.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #2825

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-04-11 16:09:58 +00:00
ashmrtn
2ebab1a78b
Use prefix matcher when merging backup details (#3055)
Store all locations in the prefix matcher
and then look them up when merging details

Intermediate step to get things setup for
having OneDrive locations during merging

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #2486

#### Test Plan

- [x] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-04-11 03:23:13 +00:00
Abhishek Pandey
2300727b09
Sanitize recurrenceTimezone prior to restoring calendar event (#3064)
<!-- PR description-->
* Do not specify recurrenceTimeZone if it's not set in the item being
restored. "" is not a valid value
---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->

- [x] 🐛 Bugfix


#### Issue(s)
* COR-53

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💚 E2E
2023-04-10 13:17:53 -07:00
ashmrtn
27909592b8
Exit early if context is marked as done (#2883)
Keeps the code from blocking and then failing immediately without sending a retry.

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* closes #2882

#### Test Plan

- [ ] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-04-10 18:38:14 +00:00
Keepers
896c05f623
look up users and sites by id or name (#2973)
Adds a lookup step to graph connector to find
an owner's id and name given some identifier.
The identifier, for either sites or users, can be a
well formed id or name.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🌻 Feature

#### Issue(s)

* #2825

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-04-07 22:29:09 +00:00
Keepers
3cc29649ed
Add GetUserInfo to services/m365 (#2991)
Adds an interface for GetUserInfo, which gets
corso-specific metadata for a single user.  Also
does some refactoring around discovery for
better interface consistency, both as variables
and as access layers.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🌻 Feature

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-04-07 00:36:50 +00:00
Abin Simon
baddd9fc83
Get OneDrive item with getM365 (#2791)
Add support for onedrive in getM365

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-04-06 05:32:03 +00:00
ashmrtn
59e17757bf
Switch LocationPath return type (#3038)
LocationRefs don't need prefix directories, so using a path.Path is
overkill. This will also fit better with changes to come where we need
to parse and update the LocationRef of merged backup details entries.

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #2486

#### Test Plan

- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-04-05 21:10:42 +00:00
Keepers
0d521bb5a7
move sharepoint onto map-based lookup (#2958)
Refactors the sharepoint site lookup to use the
id-to-name maps.  This has a momentary regression
that will get solved in the next PR: we no longer
match on weburl suffixes, and instead require
a complete match.

Also, migrates the sharepoint lookup code out
of GC and into Discovery.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🌻 Feature

#### Issue(s)

* #2825
* #1995
* #1547

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-04-05 19:38:22 +00:00