650 Commits

Author SHA1 Message Date
Keepers
b11d80599b
increase page size for all exchange pagers (#3630)
Noticed during testing that we're still paging 10 items at a time for exchange item pager queries.  This increases that limit to the consts we have set for max delta and non-delta pages, respectively.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🐛 Bugfix

#### Issue(s)

* #3562

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-06-22 23:00:57 +00:00
Keepers
f4847404c4
add per-container item collision enumeration (#3629)
populates the item collision cache generators for all exchange data types.  Next we'll use these to compare the items being restored for collision before making post requests, by populating this cache at the top of each collection restore process.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🌻 Feature

#### Issue(s)

* #3562

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-06-22 20:54:06 +00:00
Abin Simon
b0305a5319
Update attachments for events which drift from series master (#3644)
This updates any changes to attachments for individual event instances
for the ones that differ from the series master.<!-- PR description-->

---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* fixes https://github.com/alcionai/corso/issues/2835

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [ ]  Unit test
- [x] 💚 E2E
2023-06-22 13:37:05 +00:00
Keepers
3bcc405327
add generic item enumeratiton iface to api (#3628)
Adds a generic item enumeration pager to the api's item_pager set.  This compliments the more focused getIdsAndAdditional with something that's oriented towards single-folder, non-delta enumeration.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [ ] 🌻 Feature

#### Issue(s)

* #3562

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-06-21 16:43:33 +00:00
Abin Simon
d9b5cda8f1
Fix uploading large attachments for exchange (#3634)
Upload of large attachments were broken previously.

Previously we were relying on the size reported by Graph API, but
that is not reliable and we were not completing uploads because of that.

<!-- PR description-->

---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [ ]  Unit test
- [x] 💚 E2E
2023-06-21 07:40:41 +00:00
Abin Simon
9199db8f45
Handle event exceptions in caledar (#3589)
This ensures that we backup and restore and exceptions in the recurring events. One thing pending here is fixing up the attachments. I hope to create that as a separate PR. This can probably go in separately as well.

---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* https://github.com/alcionai/corso/issues/2835

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2023-06-19 11:57:32 +00:00
Keepers
c70207b1f8
separate pager and plain apis for exchange (#3627)
Separates the pager and enumerattion functionality from the rest of the exchange api funcs for each
category.  Purely code movement, no logic changes.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #3562

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-06-16 00:38:16 +00:00
Keepers
416383a99c
cascade restoreCfg collision policy into onedrive (#3623)
Adds collision policy handling to onedrive item posts.  This allows us to override the default "replace" behavior that currently returns a 409 for the creation endpoint, in case we want to use skip (ie: fail) or copy handling.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🌻 Feature

#### Issue(s)

* #3562

#### Test Plan

- [x] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2023-06-16 00:07:09 +00:00
Abin Simon
cfbed454ea
Add note about skipping attachment fetch errors (#3626)
https://github.com/alcionai/corso/pull/3607#discussion_r1229799665

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-06-15 17:10:29 +00:00
Keepers
49903a11ae
quick logging fixes (#3624)
Co-authored-by: aviator-app[bot] <48659329+aviator-app[bot]@users.noreply.github.com>
2023-06-15 16:38:55 +00:00
Keepers
46eedc0ad1
add default restore config normalizer (#3613)
Adds a func that normalizes the restore config in case inputs don't match accepted or expected values.

Also moves DefaultTestRestoreConfig out of /tester and into /control/testdata to avoid circular dependency issues.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🌻 Feature

#### Issue(s)

* #3562

#### Test Plan

- [x]  Unit test
2023-06-15 16:04:42 +00:00
neha_gupta
0a17a72800
fetch repoID from kopia is not found in config (#3578)
<!-- PR description-->

Some events in have a empty repoID. This could be because of multiple
config files used. And new config files might have repoID missing.
Solution- Fetch repoID from Kopia is not found locally in the config
file.

Note: Since the Corso Start event happens before we connect to Kopia, it
might have repoID missing if the config file is used for the first time.
All the consecutive events will have correct values. Corso start event
will start populating correct values if the config file is used once.

#### Does this PR need a docs update or release note?

- [ ]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🐛 Bugfix

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic
words" - "closes, fixes" to auto-close the Github issue. -->
* https://github.com/alcionai/corso/issues/3388

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
2023-06-15 11:31:57 +05:30
Abin Simon
6e2d72509c
Skip any attachment fetches that fail with OLE conversion error (#3607)
These files from what we can understand are not available and thus can't be fetched. This change ensures that we don't fail the backup just because of this error.

<!-- PR description-->

---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-06-14 05:09:07 +00:00
Keepers
ce72acbcc1
auto-log recoverable errors with stack (#3598)
automatically log when we add a recoverable error or a skipped item to fault.  This log will include a stack trace of the call from the location of the logged recoverable.  Clues does not have a method for pulling a stack trace out of an error yet; that can be added at a future date.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🤖 Supportability/Tests

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-06-13 22:18:18 +00:00
Keepers
2f6d731993
rename connector -> m365 (#3600)
renames /internal/connector to /internal/m365.  No logic changes in this PR.  Only the dir rename, import renames, and one linter shadowing rename.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #1996

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-06-13 18:35:39 +00:00
Keepers
960e8b79a0
cetnralize restoreConfig (#3563)
centralizes all restore configuration management within a restoreConfig struct.  This struct is owned by the control package, which allows it to be utilized by both CLI and SDK consumers.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #3562

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-06-12 23:33:22 +00:00
Keepers
cbbc8d2f6c
drive-focused api refactoring prep (#3471)
setting up the drive api calls into files/spaces that will cascade naturally to the addition of an api client for users and sites.  contains some partial implementation of these clients, which will get completed in the next pr.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #1996

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-06-02 20:16:47 +00:00
Abin Simon
5c4d57b416
Parallelize restores within a collection for OneDrive (#3492)
This should massively speed up when restoring a collection with many
items. Will not impact much if we have a lot of collections with few
items each.

Numbers 🔢 :
- Restoring ~7000 files, mostly small, totaling 1.5GB
  - Sequential: ~70m
  - Parallel: ~50m
- Restoring 1200 50mb files
  - Sequential: 4h 45m
  - Parallel: <40m

---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* https://github.com/alcionai/corso/issues/3011
* closes https://github.com/alcionai/corso/issues/3536

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2023-06-02 04:56:11 +00:00
Keepers
cdf26b7988
refactor exchange restore to use interfaces (#3456)
refactors exchange restore from near-duplicate per-category functions and switch-based process trees with interfaces.

At the top of restoring all collections, each category creates a categoryRestoreHandler to supply the necessary restore behavior.  The appropriate handler gets passed in to the collection restore, and all restore code after that takes a single path using a common restore interface to switch between categorical behavior.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #1996

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-06-02 00:58:09 +00:00
Abin Simon
6b7502bcb8
Ensure calling Details() twice does not duplicate folder names (#3556)
Related to https://github.com/alcionai/corso/pull/3140 . We were editing the deets in place and returning a pointer. This is fine as long as we were only calling it once, but https://github.com/alcionai/corso/pull/3480 created an additional call for this which causes the `konwnFolders` to be appended twice.

Old backups will still have incorrect data, but an incremental backup on top of it should fix up the data.

---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* Fixes https://github.com/alcionai/corso/issues/3555

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-06-01 18:48:37 +00:00
neha_gupta
06b04c6eff
update files count and total size on CLI for backup (#3480)
<!-- PR description-->
Currently the backup output show,
- total backup size - size of files + size of metafiles
- no of files - no of files + no of metafile
With this change the output of backup command shows size of only files and count include only file count.

NOTE: all current three services results will be impacted here.

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🐛 Bugfix

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* https://github.com/alcionai/corso/issues/3304

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ]  Unit test
- [x] 💚 E2E
2023-05-31 18:02:38 +00:00
Abin Simon
8d68bacfb7
Do not print all the restored items if there are a lot of them (#3495)
For large restore, it just ends up printing a lot of text. This change makes it print a summary instead. We still retain printing the full content if the user asks for a json output as we assume they would be piping it somewhere.

```
Connecting to repository:      1s done
Restoring to folder Corso_Restore_24-May-2023_11-49-44
Connecting to M365:         done
Restoring ∙ 7ceb8e03-bdc5-4509-a136-457526165ec0
Discovered 3 items in backup d5c07808-545d-4906-9ade-f224ca8f13de to restore
Enumerating items in repository:      0s done
Restoring data:     54s done
Restored 304 items.
```

---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-05-31 04:03:23 +00:00
neha_gupta
92845ed139
add user info to sharepoint details (#3509)
<!-- PR description-->

add info about owner to Sharepoint details.

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🐛 Bugfix

#### Issue(s)

https://github.com/alcionai/corso/issues/3356

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-05-26 08:35:50 +00:00
Keepers
6e4f30d892
rename ownerName to protectedResourceName (#3513)
#### Does this PR need a docs update or release note?

- [x]  Yes, it's included

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #3478

#### Test Plan

- [x]  Unit test
2023-05-26 01:43:57 +00:00
Keepers
874e3c3a50
cleanup exchange api code, move out of support (#3455)
normalizes exchange api naming and client usage.
Also moves serialization and transformation to and from graph client objects out of graph/support and into the api. Some parts of graph/support have been moved into conn/ exchange instead, since they're more oriented around certain forms of object consumption rather than broad object serialization.

Except for different uses of api client graph servicers, all changes are movement/renaming.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #1996

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-05-25 23:53:11 +00:00
Keepers
e8325711b3
update logging standards doc (#3497)
#### Type of change

- [x] 🗺️ Documentation
2023-05-25 06:28:40 +00:00
Keepers
7181e2ef90
move ad-hoc drive api calls into m365/api (#3451)
ensure that all drive-baased graph client calls exist in the m365/api package, not defined ad-hoc throughout the codebase.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #1996

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-05-24 21:45:07 +00:00
Keepers
1fb8a426dc
add test name to test context (#3484)
populate a "test_name" clues field in the
tester context, so that logs can be associated
with the calling test.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🤖 Supportability/Tests

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-05-24 17:48:36 +00:00
Keepers
7aaabcefdf
check for log file location in artifacts (#3470)
ensure test artifacts have log files

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🤖 Supportability/Tests

#### Test Plan

- [x] 💚 E2E
2023-05-23 20:21:29 +00:00
Keepers
cf849e9c5d
fix has drive and has mailbox conditions (#3473)
#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🐛 Bugfix

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-05-23 18:08:34 +00:00
ashmrtn
15ef7d5e9b
Try to catch concurrent serialization issues (#3479)
Run a bunch of serialization goroutines at the same time to try to catch any buffer reuse errors there may be. Fails on the old version of corso code that didn't make defensive copies.

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Test Plan

- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-05-23 16:55:28 +00:00
Keepers
d3611db059
add ownerName to backup list json (#3481)
#### Does this PR need a docs update or release note?

- [x]  Yes, it's included

#### Type of change

- [x] 🌻 Feature

#### Issue(s)

* #3748

#### Test Plan

- [x] 💪 Manual
- [x]  Unit test
2023-05-23 16:31:52 +00:00
Keepers
3d170c5b66
follow-up refinements to m365/api (#3461)
some renaming, mostly refactoring the configuration (select and header declaration for each query) by combining the consts.go and query_param.go file into config.go, and making a more composable set of funcs and consts.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #1996

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-05-22 18:37:01 +00:00
jules
11752d869c
Should not error out on "My site not found" (#3469)
This code should not error out on `Unable to retrieve user's mysite URL`.

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2023-05-22 17:54:44 +00:00
neha_gupta
8bb9df8e75
get repoID from config for events (#3449)
<!-- PR description-->

- Get repoID from config for events for uniformity across repo
- Set a default value if repoID is nil/blank

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🐛 Bugfix

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* https://github.com/alcionai/corso/issues/3388

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-05-22 15:04:56 +00:00
ashmrtn
f28c6e53d4
Add tests for older versions of backup details data (#3454)
Add tests for older backup details versions
including things like:
* SharePoint using OneDriveItem type
* not having LocationRef
* not having ItemRef
* folder names in RepoRef
* file names in RepoRef

Recommend viewing with ignore whitespace changes
---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #3269

#### Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-05-20 01:41:37 +00:00
ashmrtn
9520de1a37
More robust folder fetching for container cache (#3464)
Update folder cache population
* try to fetch missing parent folders
* refresh current folder if parent wasn't cached
* only populate paths during populatePaths not during IDToFolder

Manually tested email folder resolver population with
stable reproducer and succeeded

---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

#### Test Plan

- [x] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-05-20 00:05:52 +00:00
jules
7057d94e35
Adding two new apis to corso for validating a user has exchange or one drive is enabled (#3447)
Adds two new apis that expose calls to check whether a user has one drive or exchange enabled

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2023-05-18 20:25:32 +00:00
Keepers
6b7745745a
refactor api options (#3428)
now that exchange api has been folded in with the rest of the m365 api, it doesn't make sense to maintain an options file with only exchange functionality.  Since all calls in the file were used 1:1 with some api func, those options have been moved into their respective api funcs.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #1996

#### Test Plan

- [x] 💚 E2E
2023-05-18 18:49:44 +00:00
Keepers
e64f93a7d3
migrate exchange api to pkg/services/m365 (#3416)
No logic changes, just code movement.

Begins the process ofcentralizing the graph package in pkg/services/m365, instead of dividing up the funcs between different connector subpackages.  While the ownership of apis in each connector subpackage does make sense, this movement will reduce duplication, centralize import arrangements, and make it easy
to export usage of the api to sdk consumers.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #1996

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-05-17 00:21:20 +00:00
ashmrtn
089a96d437
Create CLI command for maintenance (#3226)
Creates hidden CLI command to run
maintenance.

Flag names and descriptions can be
updated if something else would fit
better

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #3077

#### Test Plan

- [x] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-05-16 21:03:06 +00:00
Keepers
666813865e
case sensitive selection for item IDs (#3397)
Use a strictEquals matcher for item selection when
not matching on item names (no name match assumes
canonical system id match, which is case sensitive).

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🐛 Bugfix

#### Issue(s)

* #3313

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-05-16 18:15:18 +00:00
Abin Simon
67e055360d
Bump msgraph deps to 1.0.0 (#3345)
Bumps ms graph and all dependent libs to 1.0.0

*Still can't drop betasdk as `models.SitePageable` is still not available.*

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* Closes https://github.com/alcionai/corso/pull/3327
* Closes https://github.com/alcionai/corso/pull/3326
* Closes https://github.com/alcionai/corso/pull/3324
* Closes https://github.com/alcionai/corso/pull/3323
* Closes https://github.com/alcionai/corso/pull/3322
* Closes https://github.com/alcionai/corso/pull/3321
* Closes https://github.com/alcionai/corso/pull/3320
* Closes https://github.com/alcionai/corso/issues/3354

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2023-05-16 17:44:16 +00:00
Keepers
6febc3ce5b
add namespace to fault items (#3415)
prevent fault items and skips from clobbering eachother by adding a namespace to the fault item that defines a deduplication boundary.  This allows multiple items in a single service operation to have identical ids so long as their namespace differs.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🐛 Bugfix
- [x] 🤖 Supportability/Tests

#### Issue(s)

* #3283

#### Test Plan

- [x] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2023-05-16 17:18:51 +00:00
Keepers
5ca595b7b0
filters cleanup prep for case handling (#3395)
Some tidying up in filters before adding a case specific comparator.  Goals include:
* better test handling/coverage
* swap iota for string in comparator type
* more organized param normalization
* reduction in code duplication

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #3313

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-05-16 16:41:39 +00:00
ashmrtn
04a30ac662
Don't display folders in restore output (#3414)
#### Does this PR need a docs update or release note?

- [x]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

* closes #3281

#### Test Plan

- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-05-15 23:28:04 +00:00
Keepers
3c26b09d71
separate log level and format config (#3398)
Separates the log level and log format.  This will allows sdk consumers using `SeedLogger` to utilize JSON
format logs without sacrificing log level controls.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🤖 Supportability/Tests

#### Test Plan

- [x] 💪 Manual
2023-05-15 22:03:37 +00:00
Keepers
97ed97c1c3
logic change for sharepoint permission support (#3333)
adds logical changes required for sharepoint permission handling, both backup and restore.

---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included

#### Type of change

- [x] 🌻 Feature

#### Issue(s)

* #3135

#### Test Plan

- [x] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2023-05-15 21:03:00 +00:00
Keepers
a9918d2f78
add repoRef pii transformer (#3383)
adds a transformer func in path/elements.go to safely log a repoRef (or any other dir ref) without needing to conceal the entire string.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Test Plan

- [x]  Unit test
2023-05-12 21:43:16 +00:00
Abin Simon
5995e7aff6
Fix json tag for modified time in sharepoint info (#3353)
Small typo fix for metadata store.

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-05-12 17:25:40 +00:00