1164 Commits

Author SHA1 Message Date
ashmrtn
f672325193
Update clues dependency (#3933)
Pull in fix for double hashing some values

----

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #3895

#### Test Plan

- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-07-28 20:46:49 +00:00
ashmrtn
e0368ba45a
Track and print stats about merging (#3930)
Track and print information about the number
of directories that were moved, deleted,
recursively moved or deleted, and that
didn't move at all

This can help shed light on what's happening
and generally help with debugging issues
that may crop up

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #3929

#### Test Plan

- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-07-28 19:00:53 +00:00
ashmrtn
b12bb50a4a
Retention update op (#3865)
Create new operation type to update retention parameters
for immutable backups

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup


#### Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-07-28 18:27:54 +00:00
ashmrtn
1e126bd2af
Allow setting retention parameters on repo init (#3903)
Take in information about retention so that
when the repo is initialized we can configure
retention

Not currently exposed by CLI layer

**Requires changing the interface for repo
init so SDK consumers will require changes**

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #3519

#### Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-07-28 15:39:17 +00:00
Abin Simon
c654dfba1b
Export data from SharePoint (#3824)
This borrows a lot of the core logic from OneDrive as the internal structure is mostly the same.

<!-- PR description-->
Prev: https://github.com/alcionai/corso/pull/3822

---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* fixes https://github.com/alcionai/corso/issues/3823

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-07-28 08:40:35 +00:00
Abin Simon
d7443c2211
CLI for exporting data from OneDrive (#3822)
This adds the final piece to enable OneDrive exports. The CLI interface which consumes NewExport interface from Repository and lets the user to export and write the contents in a backup to the local filesystem.

<!-- PR description-->
Prev: https://github.com/alcionai/corso/pull/3821
Next: https://github.com/alcionai/corso/pull/3824

---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* https://github.com/alcionai/corso/pull/3797
* https://github.com/alcionai/corso/issues/3670

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-07-28 08:01:12 +00:00
Keepers
34d6c18d74
clean up exchange adv restore test (#3870)
#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🤖 Supportability/Tests
- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #3562

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-07-27 19:16:11 +00:00
Keepers
a973f69d18
rename export interface (#3916)
to better match current standards

#### Type of change

- [x] 🧹 Tech Debt/Cleanup
2023-07-27 06:29:56 +00:00
Keepers
3613cb2aa0
look up restore resource if specified (#3853)
If the restore configuration specifies a protected resource as a restore target, use that as the destination for the restore.  First step is to ensure the provided target can be retrieved and identified.

---

#### Does this PR need a docs update or release note?

- [x] 🕐 Yes, but in a later PR

#### Type of change

- [x] 🌻 Feature

#### Issue(s)

* #3562

#### Test Plan

- [x]  Unit test
2023-07-26 17:24:53 +00:00
Abin Simon
f6d44b3c60
Export data from OneDrive (#3819)
Core logic for exporting data from OneDrive

Next: https://github.com/alcionai/corso/pull/3820

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [x] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* https://github.com/alcionai/corso/issues/3670
* https://github.com/alcionai/corso/pull/3797

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-07-26 12:30:39 +00:00
ashmrtn
3a98bcdcf5
Minor code cleanup (#3901)
Remove now-unused code

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #2360

#### Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-07-25 20:04:09 +00:00
ashmrtn
dd55744f15
Hide storage path for merge collections (#3902)
#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #3895

#### Test Plan

- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-07-25 19:38:12 +00:00
Keepers
92412645ec
pii handling for restore config (#3896)
add pii concealer compliance for restore config structs.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🌻 Feature
2023-07-25 17:25:34 +00:00
ashmrtn
42adc033d9
Pass BackupBases directly to ConsumeBackupCollections (#3876)
Shift things so BackupBases is passed directly to
the kopia package which then extracts information
from it. This allows for fine-grained control
over kopia-assisted incremental bases and merge
bases. Generating subtree paths from Reasons is
also shifted to the kopia package

Also expands tests for better coverage of various
incremental backup situations

Viewing by commit may help and individual commit
comments usually contain reasons for changes,
especially for test removal

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #2068

#### Test Plan

- [x] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2023-07-24 23:31:54 +00:00
Keepers
d2dda00195
update all graph packages (#3879)
#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup
2023-07-24 21:45:57 +00:00
ashmrtn
f7496e5242
Require passing AWS bucket for integration tests (#3852)
#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup


#### Test Plan

- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-07-24 19:46:21 +00:00
ashmrtn
ab344422d6
Cleanup figuring out kopia retention parameters (#3857)
Create a struct that handles:
* initialization from existing kopia config info
* in-memory updates to config info
* detecting which config info structs from kopia need updated
* returning kopia config info structs

Overall, this allows us to isolate the logic for
calculating the new retention configuration info
in kopia

Viewing by commit may help.
First commit just splits up existing code, moving it into
either conn.go (will be used later) or retention/opts.go.
Subsequent commits switch to using a struct, add tests,
and fixup existing logic

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

#### Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-07-24 18:43:27 +00:00
ashmrtn
c073ccc5ea
Add a couple more linters (#3886)
Lint for kv-pairs in log calls and things
that can be replaced with constants from
the standard library

Also add the linters that are enabled by
default just so things are more consistent
and clear going forward

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Supportability/Tests
- [x] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #3654

#### Test Plan

- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-07-22 01:34:38 +00:00
Keepers
0d6b08204d
allow users to limit page size (#3875)
allows cli users to limit the page size of delta queries by calling a new hidden flag: --delta-page-size.
This also adds the control.Options struct to the api client, so that configurations such as this can be easily handed into, and used by, the client.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🌻 Feature

#### Test Plan

- [x] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2023-07-22 00:00:09 +00:00
ashmrtn
62d4c68c04
Have kopia package generate tags during backup based on Reasons (#3869)
Move tag generation from the backup op to the
kopia package. This makes it match the pattern
that finding base backups uses where a set of
Reasons and a separate set of additional tags
are provided

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #2360

#### Test Plan

- [x] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2023-07-21 22:49:42 +00:00
Keepers
7677299ace
use replace collisions for nightly tests (#3882)
#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🤖 Supportability/Tests

#### Issue(s)

* #3562

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-07-21 22:19:13 +00:00
ashmrtn
916bb0b27c
Create an interface and implementation for Reason struct (#3868)
Intermediate step to a few different goals including
* moving interface definitions to better locations while avoid cycles
* adding a flag to disable kopia-assisted incrementals

Create an interface and implementation for the existing Reason
struct. The goal is to set stuff up so that eventually the kopia
package can ask the struct for the subtree path to work with
when merging the hierarchy instead of having the backup operation
pass that information in

Code changes are mostly just turning stuff into a struct and
fixing up compile errors. Some functions have been excluded from
the struct (i.e. `Key`) and made into functions in the kopia
package itself

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #2360

#### Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2023-07-21 21:05:38 +00:00
Keepers
d555501093
add integration tests for missing drives (#3847)
#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🤖 Supportability/Tests

#### Issue(s)

* #3652

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-07-21 01:13:09 +00:00
Keepers
9359679f99
add api funcs for creating documentLibs (#3793)
Adds api handlers for creating document libraries in sharepoint. This is the first step in allowing us to restore drives that were deleted between backup and restore.

---

#### Does this PR need a docs update or release note?

- [x] 🕐 Yes, but in a later PR

#### Type of change

- [x] 🌻 Feature

#### Issue(s)

* #3562

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-07-20 21:05:30 +00:00
ashmrtn
2c6417d2eb
Kopia wrapper code to set retention parameters (#3792)
Code at the kopia wrapper layer that allows configuring kopia
for retention if desired. Includes some tests that require
running against a bucket with object locking enabled

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #3519

#### Test Plan

- [x] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-07-19 20:50:17 +00:00
Keepers
5a78f478a1
add operations test for adv rest (#3783)
ads operations level tests for advanced restore
configuration on all three services.
Code is largely boilerplate between each service, but with just enough quirks that full consolidation would require excess jumping through hoops.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🤖 Supportability/Tests

#### Issue(s)

* #3562

#### Test Plan

- [x] 💚 E2E
2023-07-17 03:56:08 +00:00
Keepers
09e5e9464a
add test for 503 resp body (#3812)
#### Type of change

- [x] 🤖 Supportability/Tests

#### Issue(s)

* closes #3811

#### Test Plan

- [x]  Unit test
2023-07-14 23:40:30 +00:00
Keepers
fd6dff3270
retry on http/2 stream errors (#3802)
http/2 stream error failures don't allow
us to leverage client middleware.  As a
result we're lacking a retry handler.  This
change adds retries for http_wrapper
in the explicit case of a http/2 stream
error.

---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included

#### Type of change

- [x] 🐛 Bugfix

#### Test Plan

- [x] 💪 Manual
- [x] 💚 E2E
2023-07-14 22:57:51 +00:00
Keepers
e60610193c
specify sharepoint restore libraries (#3801)
#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🤖 Supportability/Tests

#### Test Plan

- [x] 💚 E2E
2023-07-14 01:21:53 +00:00
Abhishek Pandey
19d56b1e29
Return original read closer if no extension factories are supplied (#3784)
<!-- PR description-->

As title

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2023-07-11 21:46:33 +00:00
ashmrtn
83c18e5276
Add tests that boomerang items (#3779)
Add a test that will fail if something is wrong with tracking items that are moved from a folder to a different folder and back to the original folder between incremental backups. This is important as it will help us ensure we don't accidentally turn on Exchange immutable IDs before they're fixed on the Graph side

Only tests Exchange mail and OneDrive/SharePoint as other Exchange data types don't support move operations via Graph API (it is possible to move events between calendars in the web UI though)

Manually tested to ensure it does actually fail if immutable IDs are enabled

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

* closes #3755

#### Test Plan

- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-07-11 00:18:19 +00:00
ashmrtn
87b8631037
Fixups for comparing details to expected set (#3778)
Collection of fixes to get tests passing again including:
* ensure expected sets are tracked for the lifetime of the struct
* add check that will fail if there's no expected deets to compare against
* minor fixups to path building when reading in an entire details file
* trim down data sent for patch operations to only the data that's changed
* update how expected items are gathered -- just pull it from graph directly
* update expected written items for OneDrive -- most likely due to matching both test folders (e.x. `corso_incr_test_1` and `corso_incr_test_1/corso_incr_test_1`)

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

* closes #3777

#### Test Plan

- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-07-10 23:46:13 +00:00
ashmrtn
97c10cbc68
Skip failing SharePoint pages tests (#3781)
The SharePoint pages code is neither available from the CLI nor maintained right now. When the code is maintained again the tests can be unskipped.

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #3780

#### Test Plan

- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-07-10 21:29:27 +00:00
Abhishek Pandey
83e56ed396
Integrate extensions into drive item reader (#3750)
<!-- PR description-->

* Integrates corso extensions into onedrive/sharepoint library item backup flows.
* Also includes integration tests for OD/SP, unit tests for collections.
* Includes a small fix for `rw *backupStreamReader Close()`

Remaining things which will be covered in later PRs:
* extension tests with incremental backups
* Observability related changes for extensions

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [x] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* internal
#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2023-07-06 23:01:18 +00:00
Keepers
5ea194dc87
tweaks and fixes found while testing (#3735)
a variety of small updates that came
from manual testing of restore with various
collision and destination combinations.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🐛 Bugfix

#### Issue(s)

* #3562

#### Test Plan

- [x] 💪 Manual
2023-07-06 22:24:26 +00:00
Keepers
39a93e9054
move default folder consts to api (#3734)
only code movement, no logic changes

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #3562

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-07-06 20:25:06 +00:00
Keepers
2a150cc610
add counter for skip-collision counting (#3722)
Introduces a counting bus, and threads it into restore operations so that we can count the number of
collision skips that occur.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🌻 Feature

#### Issue(s)

* #3562

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-07-06 19:22:02 +00:00
Keepers
8ba79709a6
split tester into separate files (#3762)
This is primarily an exercise in reducing the number of circular imports we get from adding the tester package to other packages.

No logic changes.  Purely movement/renaming.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🤖 Supportability/Tests

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-07-06 15:43:57 +00:00
Keepers
fe6ae28b68
distribute operations integration tests (#3706)
Moves operations integration backup tests into a
subpackage, and slices up the test folder into
multiple, service specific folders.

No logic changes, only code copy-pasting and
renaming.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🤖 Supportability/Tests

#### Test Plan

- [x] 💚 E2E
2023-07-05 22:19:04 +00:00
Keepers
8cc3069a6a
ensure od and exchange can restore in-place (#3703)
Adds changes (mostly to exchange) that allow in- place item restoration.  Onedrive basically already supported in-place restores, however we weren't attempting them.  Exchange needed full support of GetContainerByName across its categories.

Next steps: add CLI integration to simplify manual testing.  Then add automation testing to exercise all in-place restore conditions and configurations.

---

#### Does this PR need a docs update or release note?

- [x] 🕐 Yes, but in a later PR

#### Type of change

- [x] 🌻 Feature

#### Issue(s)

* #3562

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-07-05 21:44:55 +00:00
Abhishek Pandey
6fff3f7d1d
Refactor populateItems (#3732)
<!-- PR description-->

Benefits:
* Narrowed scope & better testability. 
* Preps it for follow up PRs which will inject per-item corso extension interfaces
* Not adding unit tests yet, as the args & outputs of this new function will change in a follow up PR.
---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-07-05 18:52:21 +00:00
neha_gupta
09f3196ee0
Onedrive restore- handle unknown path error (#3643)
<!-- PR description-->
 In case of Onedrive Restore give a proper message if the path provided do not exist or restore is done on empty backup.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🐛 Bugfix

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* https://github.com/alcionai/corso/issues/3579

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
2023-07-04 11:37:16 +00:00
Keepers
662d626809
handle file-folder collision cases (#3688)
handle onedrive item collision cases where a file to be restored is in conflict with an existing folder. Skip and Copy will function normally.  For Replace behavior, we'll defer to Copy in this situation, so that the file gets restored without changing or deleting the folder.  If a folder creation attempts to replace an item, we do a similar action and make a renamed copy of the folder instead.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #3562

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-07-04 02:34:53 +00:00
Keepers
cf495a6c8f
add authErr to users mailbox check (#3744)
license issues can produce an authenticationError when retrieving a user's inbox.  This catches that error and treats it as a "mailbox not availble" condition.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🐛 Bugfix

#### Issue(s)

* #3743

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-07-03 19:31:35 +00:00
Keepers
56151a82eb
remove discovery pkg (#3677)
Currently unused, and functionality is largely duplicated by the services/m365 package.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [ ] 🧹 Tech Debt/Cleanup

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-07-01 13:52:57 +00:00
Keepers
8b81728488
fix concurrency limiter init race (#3715)
initializes the concurrency limiter as part of CLI pre-run initialization to ensure we don't accidentally skip out on including the limiter in middleware.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🐛 Bugfix

#### Issue(s)

* #3695

#### Test Plan

- [x] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2023-06-30 23:54:54 +00:00
Keepers
5949a3bcd7
add logging indicating full backups (#3721)
#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🤖 Supportability/Tests

#### Issue(s)

* #3720
2023-06-30 23:02:01 +00:00
Keepers
b74f65e301
properly declare resourceNotFound for sites (#3728)
Missing sites return an ItemNotFound error code,
which we aren't currently handling when doing
queries for individual sites.  This change catches that error code and stacks the error with a
resourceOwnerNotFound sentinel.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🐛 Bugfix

#### Test Plan

- [x] 💪 Manual
- [x] 💚 E2E
2023-06-30 18:14:06 +00:00
Keepers
f2a4fd0ba3
od: add per-container collision caching and checks (#3640)
pushes collision checks in onedrive upward from the api layer (due to the bug with ConflictBehavior) and into the restore controller.  Collision resolution in drive follows the same pattern that we use in exchange.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🌻 Feature

#### Issue(s)

* #3562

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-06-30 16:13:35 +00:00
Keepers
c218dd865b
move event instance funcs to new file (#3659)
No logic changes, just code movement.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-06-30 15:50:04 +00:00