104 Commits

Author SHA1 Message Date
Abhishek Pandey
4dcb820201 Linter 2024-01-12 11:56:00 -08:00
Abhishek Pandey
7b193a06a0 Augment post details with recipient information 2024-01-12 11:42:09 -08:00
Keepers
1694581e26
make a not found core error (#4975)
make a canonical 'not found' error for our core sentinels. This replaces the graph.DeletedInFlight error, which acted as an interpretation of "not found".

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #4685

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2024-01-12 01:29:39 +00:00
Keepers
279a6dab3c
normalize observe usage, ensure deferred closers (#5006)
some housecleaning before reviewing deadlock issues

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Test Plan

- [x]  Unit test
2024-01-11 22:42:10 +00:00
Keepers
b1af13b7b6
centralize core err interpretation (#4972)
now that we have a core error library, we can start swapping out the percolation of graph api errors with core error sentinels.  This PR begins that process by adding a centralized transformer to the graph service handlers that ensures all calls produce the same set of core errors according to some standard categorizations. This is just an initial step, and does not transfer all graph errors into core sentinels.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🤖 Supportability/Tests

#### Issue(s)

* #4685 

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2024-01-10 17:25:25 +00:00
Keepers
af64b487ac
swap pkg/metrics to threadsafe map (#4899)
<!-- PR description-->

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🐛 Bugfix
- [x] 🧹 Tech Debt/Cleanup

#### Test Plan

- [x] 💚 E2E
2024-01-08 20:22:54 +00:00
Abin Simon
ca50b1e908
Filter out non existent entities in permissions and link shares (#4955)
<!-- PR description-->

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2024-01-08 13:44:25 +00:00
Hitesh Pattanayak
87104ce404
exports sharepoint lists (#4959)
provision to export sharepoint lists

#### Does this PR need a docs update or release note?
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature

#### Issue(s)
#4752 

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2024-01-08 05:57:32 +00:00
Hitesh Pattanayak
d113fa3926
store lists previouspath (#4938)
saves previous path for lists in storage. 

#### Does this PR need a docs update or release note?
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature

#### Issue(s)
#4754 
#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2024-01-04 13:49:33 +00:00
Keepers
971d874462
introduce errs/core (#4897)
first step towards having a centralized set of error sentinels that can be passed around corso instead of re-using low level error sentinels like those found in the graph/errors.go file.

This PR works as a standalone, and only handles the lowest hanging fruit. SDK consumers will need to change their error enum references, but all api behavior should remain the same.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🤖 Supportability/Tests

#### Issue(s)

* #4685

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2024-01-03 18:13:41 +00:00
Abin Simon
dcff3056ff
Contacts export for exchange (#4883)
<!-- PR description-->

---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* closes https://github.com/alcionai/corso/issues/3891

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2024-01-03 05:22:54 +00:00
Hitesh Pattanayak
13f549c6cb
lists prefetech (#4943)
refactors sharepoint lists collection by:
- renaming `Collection` interface as `preFetchCollection` interface
- making `finishPopulation` method as independent function 
- seralizing list using `serializeContent` function

#### Does this PR need a docs update or release note?
- [x]  No

#### Type of change
- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)
#4754 

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2023-12-30 15:08:29 +00:00
Abhishek Pandey
c6b9615c7d
Integrate lazy item reader into conversations backup (#4921)
<!-- PR description-->




---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature


#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* Closes https://github.com/alcionai/corso/issues/4862

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [x] 💚 E2E
2023-12-27 07:09:12 +00:00
ashmrtn
aa876cabb9
Sanitize JSON input if it's invalid on the first attempt (#4928)
Add code and tests for sanitizing emails

Also adds code for sanitizing events and contacts but we can remove
that if we'd like to address it if/when needed. We haven't yet seen
or tried to create instances of those

---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-12-26 20:26:07 +00:00
Hitesh Pattanayak
a1590e0d2f
enables sharepoint to use lists restore handler for lists ops (#4923)
enables sharepoint to use lists restore handler for lists ops
Changes previously approved in: 
- https://github.com/alcionai/corso/pull/4854
- https://github.com/alcionai/corso/pull/4910

#### Does this PR need a docs update or release note?
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature

#### Issue(s)
#4754 

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2023-12-22 17:18:24 +00:00
Hitesh Pattanayak
dc47001cba
enables sharepoint to use lists backup handler for lists ops (#4920)
enables sharepoint to use lists backup handler for lists ops
Changes previously approved in:
-  https://github.com/alcionai/corso/pull/4786
- https://github.com/alcionai/corso/pull/4787
- https://github.com/alcionai/corso/pull/4909

#### Does this PR need a docs update or release note?
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature

#### Issue(s)
#4754 

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2023-12-22 14:26:23 +00:00
Hitesh Pattanayak
1270e64637
adds list client apis and pagers (#4917)
adds list client apis and pagers to fetch, create and delete lists and list relations.
Changes previously approved in PRs:
- https://github.com/alcionai/corso/pull/4785
- https://github.com/alcionai/corso/pull/4815
- https://github.com/alcionai/corso/pull/4852

#### Does this PR need a docs update or release note?
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature

#### Issue(s)
#4754 

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2023-12-22 08:02:32 +00:00
Hitesh Pattanayak
50fd28aab2
adds ListInfo details to SharepointInfo details (#4916)
adds ListInfo details to SharepointInfo details
Changes previously approved in PR: https://github.com/alcionai/corso/pull/4851

#### Does this PR need a docs update or release note?
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature

#### Issue(s)
#4754 

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2023-12-22 06:50:34 +00:00
ashmrtn
e46119a818
Remove options form service handler initializer (#4895)
Now that optiosn are being broken up a bit more passing a struct of a single type to the service handler initializer doesn't make much sense. Just remove the parameter since it was unused anyway.

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

#### Issues

- #4896

#### Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-12-20 18:49:28 +00:00
ashmrtn
5669619a8d
Remove other event fields ignored by server (#4888)
Update the set of ignored fields for event restores. Most important inclusion is the `iCalUId_v2` field which will cause failures if it's not removed.

---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Test Plan

- [x] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-12-20 00:26:10 +00:00
Abin Simon
b896405e92
Add tests to check for invalid email addresses in eml export (#4881)
<!-- PR description-->

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-12-19 19:41:42 +00:00
Keepers
c58cd9302f
move /cli/config to /pkg (#4858)
Part 1 of two changes I've been carting around since the hackathon.
This PR moves the /cli/config package into /pkg/config.  This allows
sdk consumers to access corso configuration handling, intead of
requiring the CLI package to access it.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🤖 Supportability/Tests


#### Test Plan

- [x] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2023-12-18 18:24:33 +00:00
Abin Simon
891814f15c
Add shortref and rename item_id (#4868)
<!-- PR description-->

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-12-18 08:42:40 +00:00
Keepers
dc3cfd1ec3
clean up drive unit test helpers (#4805)
Adds some cleanups to the drive unit test helpers.  Goals were as follows:
1. remove idx() and namex() funcs, replace with id() and name()
2. minimize factory duplication in helper_test.go
3. improve readability in id() and name() calls by adding file and folder variants to handle the 99% of cases we use them in.

No logic changes in this PR.  Only test func updates.
---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🤖 Supportability/Tests

#### Issue(s)

* #4689

#### Test Plan

- [x]  Unit test
2023-12-14 17:02:44 +00:00
Keepers
17c1bbda19
fix messages backup deadlock (#4809)
Moving scope backup code to their own functions allows us to defer closing the progress bar for the scope, rather than closing it on a specific line of code, which might get missed and cause a lock.

---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included

#### Type of change

- [x] 🐛 Bugfix

#### Test Plan

- [x] 💪 Manual
2023-12-08 17:33:17 +00:00
Keepers
54ba241fbe
require rootID on tree construction (#4746)
Turns out the root ID name isn't an appropriate match for establishing the root node.  Instead, the backup hander is now extended with a getRootFolder method and will pass the expected root folder ID into the tree's constructor func to ensure we establish the correct root node.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🐛 Bugfix

#### Issue(s)

* #4689

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-12-07 23:54:45 +00:00
Abhishek Pandey
3f98aa33de
Switch to custom drive items for backup operations (#4784)
<!-- PR description-->

Switch to using `custom.DriveItem` instead of `models.DriveItemable` during backups. There is a slight impact to restore as well, since backup and restore both use a few common interfaces e.g. `AugmentItemInfo`. 

---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2023-12-05 22:48:02 +00:00
Keepers
6307d8cbf6
prep groups handler for addition of conversations (#4583)
Makes modifications to the groups handlers to facilitate the incoming introduction of conversations backup handling. Changes include:
* new handler methods (canMakeDeltas and LocationRef)
* binding handlers to generics
* some naming normalization

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🌻 Feature

#### Issue(s)

* #4536

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-12-01 00:40:37 +00:00
Keepers
6aff258c8b
add boilerplate for drive tree processing (#4716)
begins implementation of the drive delta tree support by adding boilerplate funcs for getting the backup data, and adding a framework of unit tests that will be used bring testing parity up to par with current tests.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Issue(s)

* #4689

#### Test Plan

- [x]  Unit test
2023-11-28 21:07:49 +00:00
ashmrtn
d36636285a
Use service-handler for restore operations (#4714)
Switch restore code path to use service-level handlers. Does a few
things:
* switches existing service-level functions to be methods on
  service-level handlers
* update interfaces as necessary
* moves some logic from old controller-level restore function to either
  the new handlers or the operation-level function
* removes old code

May be easiest to review by commit

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #4254

#### Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2023-11-28 16:47:45 +00:00
ashmrtn
c48329f844
Return stats directly from restore function (#4713)
Rework restore return status so that later PRs will have a smaller diff

Instead of returning a status and then waiting on the message just return the restore stats directly. The status getter was only setup to wait for one item anyway and was setup to wait after the entire restore operation already completed (at end of
m365/restore.go:ConsumeRestoreCollections())

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #4254

#### Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2023-11-27 17:09:37 +00:00
ashmrtn
dac4c70f61
Return details from restore function (#4712)
Minor cleanup that will also help reduce diff for future changes.

Instead of taking in a details builder and adding to it during
restore, just create a local details builder and return the built
details to the caller

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #4254

#### Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2023-11-18 02:23:46 +00:00
ashmrtn
bcf290d628
Preview backup for drive-backed services (#4699)
Add logic and tests for preview backups in drive-backed services.
Does slightly change a few of the options for preview backup limits

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-11-18 00:46:36 +00:00
Keepers
2a85b61213
clean up unused graph error sentinels (#4683)
A handful of the error sentinel values in graph/errors.go were only around to comply with an old expectation that IsErr funcs have a matching sentinel.  That assumption is out of date, and there's no point in keeping around unused sentinels.

This PR makes three changes to clean up that issue:
1. remove unused sentinels (unused is qualifed as "not produced").
2. fix external pacakges which referenced those errors in tests alone by replacing those sentinels with odata errors.
3. centralize odata error test code to help support upstream package odata error creation.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-11-16 23:59:38 +00:00
Keepers
9ecefd2569
use counter in exchange backup enum (#4661)
Increase proliferation of the count bus to record runtime stats.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🤖 Supportability/Tests

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-11-16 23:22:00 +00:00
ashmrtn
1cbabdb73a
Exchange restore service level handler functions (#4693)
Add service-level restore handlers for exchange

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #4254

merge after:
* #4687
* #4651

#### Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-11-16 18:22:33 +00:00
Abin Simon
b46f242bc4
Make the cli colorful and informational (#4525)
This commit include a few things that improves the progresbars and other info messages are printed out during the CLI run.

![2023-10-20-17-17-09](https://github.com/alcionai/corso/assets/14259816/30a25832-24cb-48ef-944d-6aaced1cd62f)

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [x] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-11-16 12:11:37 +00:00
Abin Simon
140402361a
Add support for email exports (#4642)
<!-- PR description-->

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [x] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* https://github.com/alcionai/corso/issues/3893

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-11-16 05:43:05 +00:00
ashmrtn
51f44c2988
Add functions used for restore to existing service-level handlers (#4687)
This continues the push towards having service-level handlers that know
how to perform different operations. It adds the helper functions that
will be used during restore operations to the existing handler code

This logic is not currently used nor does this PR change the restore
call path

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #4254

#### Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-11-16 02:48:54 +00:00
Keepers
4c72e9eab7
use clues new|wrap|stackWC() (#4684)
replace all cases of new|wrap|stack(...).WithClues(ctx) with the flattened new|wrap|stackWC(ctx, ...) functions introduced in the latest clues bump.

Other changes added:
* remove WithClues builders when the error producer already called it.
* corrected some usages of ictx within loops.

no logic changes, just cleanup.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-11-15 23:01:08 +00:00
Abhishek Pandey
02108fab95
Remove unused control option for disabling concurrency limiter (#4677)
<!-- PR description-->

Concurrency limiter is stable and has been in use since several months. We had added a killswitch to disable it for safety reasons. It's not being used by SDK consumers, and hidden in corso cli. 

This PR removes the `--disable-concurrency-limiter` flag & associated control option.

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-11-14 19:45:34 +00:00
ashmrtn
bb27b8e44d
Rename service handlers (#4571)
Rename these handlers to be base<Service>Handler so that we get some segmentation of functionality. This will allow us a bit of compile-time safety when it comes to accessing graph because the base handler doesn't contain a client instance. Essentially it allows us local access to things for operations like restore.

Future additions to the handler that require a client should wrap this base handler to provide that functionality. Export functions shouldn't be updated to be part of the new handler wrapper but should stay as part of the base handler so they continue to not have access to graph.

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #4254

#### Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-11-13 23:08:14 +00:00
Keepers
5362137116
expose graph options to sdk clients (#4653)
1. moves the m365/graph package from internal to pkg/services/api so that options are accessible to sdk users.
2. exposes graph.Options in the api client.Service call.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🌻 Feature
2023-11-10 19:26:59 +00:00
Abin Simon
e2e2012220
Use .json extension for message exports (#4600)
We were previously exporting json but without the proper file extension.

---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* closes https://github.com/alcionai/corso/issues/4581

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-11-02 14:40:57 +00:00
ashmrtn
74cf0ab737
Create service specific handlers that know how to run an export (#4491)
First step in reducing the number of places we have to check the service type manually. Create a way to get a handle to a service specific handler and implement exports for those handlers

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #4254

#### Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2023-11-01 15:55:30 +00:00
Keepers
5e0014307c
remove ctx embedding for counter (#4497)
#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-10-31 16:15:05 +00:00
ashmrtn
f61448d650
Groups version bump (#4561)
Bump the backup version and force a full backup if
there's a backup for teams/groups that has base(s)
from an older version of corso

This will avoid propagating older details formats
forward. Those formats don't have all the data
newer formats do

This is mostly a stop-gap, a more robust solution
can be added later

Manually tested that it forces a full backup

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #4569

#### Test Plan

- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-10-27 19:37:39 +00:00
Keepers
2bc3c89885
fix api/mock circular ref (#4544)
No logic changes.  Just code movement/renaming.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup
2023-10-26 01:27:49 +00:00
Keepers
89f59eea91
mark complete subtrees as packages (#4501)
marks the full subtree of a drive package item
as being within the package.  This ensures
that all collections whose directory is a subdir of a package-type folder are themselves marked as package-typed.

---

#### Does this PR need a docs update or release note?

- [x] 🕐 Yes, but in a later PR

#### Type of change

- [x] 🐛 Bugfix

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-10-24 23:01:06 +00:00
Keepers
03c7667679
iterate pager results with channels (#4414)
corrects the prior PRs full-scale batching of driveItems by adding a channel to the api layer's enumerateItems and driveEnumerateItems funcs.  To avoid changes
in most parts of code, also adds batch wrappers
that hide these changes from existing pager usage.

DriveItem collection processing now ranges over
a channel of pager results, allowing it to process pages in a stream rather than batching all items.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup


#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-10-14 03:22:12 +00:00