Shuffle around some logic for details merging so that
we always attempt to extract a LocationRef from the
backup base entry that's currently being examined.
A LocationRef should always be available from either
the LocationRef field in the details entry (newer
backups) or by extracting it from the RepoRef (older
backups)
Manually tested incremental backups for exchange in
the following scenarios:
1. v0.3.0 backup (calendars use IDs in RepoRef) ->
incremental with this patch -> incremental with
this patch
1. v0.2.0 backup (exchange uses folder names in
RepoRef) -> incremental with this patch ->
incremental with this patch
The above tests should cover the cases where:
* base backup details don't have LocationRef for
exchange items
* base backup details have LocationRef for exchange
items
---
#### Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
#### Type of change
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup
#### Issue(s)
* closes#3716
#### Test Plan
- [x] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
#### Does this PR need a docs update or release note?
- [x] ⛔ No
#### Type of change
- [x] 🌻 Feature
#### Issue(s)
* #3989
#### Test Plan
- [x] ⚡ Unit test
- [x] 💚 E2E
Some quick tech-debt splitting up of the code in services/m365
---
#### Does this PR need a docs update or release note?
- [x] ⛔ No
#### Type of change
- [x] 🧹 Tech Debt/Cleanup
#### Test Plan
- [x] ⚡ Unit test
- [x] 💚 E2E
CLI changes for groups.
---
#### Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [x] 🕐 Yes, but in a later PR
- [ ] ⛔ No
#### Type of change
<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
#### Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* https://github.com/alcionai/corso/issues/3990
*
#### Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
Adds the boilerplate for groups backup collection processing. Not necessarily functional at this time, due to missing dependencies and consts that aren't yet in the branch. Thus the lack of tests. It's just good enough to keep progress rolling forward.
---
#### Does this PR need a docs update or release note?
- [x] ⛔ No
#### Type of change
- [x] 🌻 Feature
#### Issue(s)
* #3989
#### Test Plan
<!-- PR description-->
add Handlers interface for Channels.
#### Does this PR need a docs update or release note?
- [ ] ⛔ No
#### Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
#### Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>
#### Test Plan
<!-- How will this be tested prior to merging.-->
This commit has the initial rough set of changes needed to create collections from the group's default SharePoint site.
This still does not have all the functionality that we need, but the idea was that we could get this in and iterate over time.
<!-- PR description-->
---
#### Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [x] 🕐 Yes, but in a later PR
- [ ] ⛔ No
#### Type of change
<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
#### Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* https://github.com/alcionai/corso/issues/3990
#### Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [x] 💚 E2E
<!-- PR description-->
This PR optimizes memory & cache refresh time for URL cache. The cache only makes use of a small subset of drive item properties, namely ID, deleted, file, folder, content download URL. We have found that reducing the number of query properties has a sizable impact on corso mem usage. This is especially relevant for large scale backups.
See below graph for a comparison between original delta queries & mod. Note that this is with corso instrumentations to show comparisons side by side in the same run.
- Reading this graph
- We are doing 3 orig delta queries followed right after by 3 mod. Vertical lines are delta query spans.
Originally, this investigation was done to improve mem usage for scale backups. But we also found that url cache delta query time drops by 22% with this PR. This is because we are now transferring & processing fewer bytes.
---
#### Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
#### Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
#### Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>
#### Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [x] 💚 E2E
#### Does this PR need a docs update or release note?
- [x] ⛔ No
#### Type of change
- [x] 🌻 Feature
#### Issue(s)
* #3989
#### Test Plan
- [x] ⚡ Unit test
I've needed to catch gotchas that arise from contributors adding a value in the middle of an iota list, not to mention have dealt with prior bugs that happened the same way, now too many times to feel safe about its usage.
This PR removes the use of iota from all const declarations. The intent is to not allow the use of iota within the codebase.
---
#### Does this PR need a docs update or release note?
- [x] ⛔ No
#### Type of change
- [x] 🧹 Tech Debt/Cleanup
#### Issue(s)
* #3993
#### Test Plan
- [x] ⚡ Unit test
- [x] 💚 E2E
#### Does this PR need a docs update or release note?
- [x] ⛔ No
#### Type of change
<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
#### Issue(s)
* #3989
#### Test Plan
- [x] 💪 Manual
- [x] ⚡ Unit test
Adds groups to the m365 services api. Also adds a bit of touchups/cleanups on the side.
---
#### Does this PR need a docs update or release note?
- [x] ⛔ No
#### Type of change
- [x] 🌻 Feature
#### Issue(s)
* #3989
#### Test Plan
- [x] ⚡ Unit test
- [x] 💚 E2E
Munging code originally existed so that SDK
users would get unique ShortRefs if an item
moved or was renamed. However, we no longer
need to support that so this removes that
munging code
Manually tested making a new backup with this
PR where the merge base was a backup made prior
to this PR. A folder with a subfolder and items
was moved between the two backups. Backup
details for the new backup contained all the
expected entries
Restore selector logic should be unaffected
as that assumes the user has passed in a
ShortRef that was previously printed to the
CLI by `corso backup details`. Input is
compared against the ShortRef stored in the
entry, `ShortRef()` is not called on any `Path`
---
#### Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
#### Type of change
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup
#### Issue(s)
* closes#4012
#### Test Plan
- [x] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
<!-- PR description-->
Teams discovery API
- fetch all groups from endpoint and filter teams out of it.
- fetch team with ID.
Permission added-
- Application- Team.ReadBasic.All
- Group.Read.All
#### Does this PR need a docs update or release note?
- [ ] 🕐 Yes, but in a later PR
#### Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
#### Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* https://github.com/alcionai/corso/issues/3836
#### Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [ ] ⚡ Unit test
- [ ] 💚 E2E
Allows ignoring missing backups during deletion by
passing in an addition bool param.
Since this changes the API, we can remove the final
commit to make ignoring missing backups the
default behavior. This will change CLI behavior
though
---
#### Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
#### Type of change
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup
#### Issue(s)
* #4019
#### Test Plan
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
Allow SDK users to delete multiple backups
at the same time. This will result in fewer
kopia manifest blobs being made in S3 which
makes things simpler overall
This does change the name of the delete
function, which will break SDK consumers.
That change can be removed from the PR if
that's a better path forward
This does not add the ability to delete
multiple backups at once to the CLI. That
change should be fairly simple to do in
another PR if desired
---
#### Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
#### Type of change
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup
#### Issue(s)
* closes#4019
#### Test Plan
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
Reduce the number of kopia manifest blobs
created during backup deletion by batching
the deletion of snapshots and the backup
model into a single kopia operation
Also reorangizes/updates test code for
backup deletion
---
#### Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
#### Type of change
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup
#### Issue(s)
* #4019
#### Test Plan
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [x] 💚 E2E
Take a vararg of manifests to delete and delete them all in a single write session. This should help reduce the number of manifest blobs that kopia creates since it will only flush the pending manifest entries once during the operation.
---
#### Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
#### Type of change
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup
#### Issue(s)
* #4019
#### Test Plan
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
small cleanup in paths, primarily splitting files
so that file contents are more clearly owned, which should be a little better for readability and code placement.
Also renames `ServicePrefix` to `BuildPrefix` in
anticipation of multi-service prefixes.
no logic changes, just movement/renaming.
---
#### Does this PR need a docs update or release note?
- [x] ⛔ No
#### Type of change
- [x] 🧹 Tech Debt/Cleanup
#### Issue(s)
* #3993
#### Test Plan
- [x] ⚡ Unit test
- [x] 💚 E2E
#### Does this PR need a docs update or release note?
- [x] ⛔ No
#### Type of change
- [x] 🧹 Tech Debt/Cleanup
#### Issue(s)
* #3993
#### Test Plan
- [x] ⚡ Unit test
- [x] 💚 E2E
No longer required by SDK users and not exposed directly to CLI users unless they're looking at the JSON output from details. This field is not documented anywhere though so there's no guarantees that we technically need to uphold with it.
Manually tested:
* restoring from a backup that had this field
* making a fresh backup without this field when the merge base did have this field
---
#### Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
#### Type of change
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup
#### Issue(s)
* closes#3986
#### Test Plan
- [x] 💪 Manual
- [x] ⚡ Unit test
- [x] 💚 E2E
Remove references to the kopia package from
`pkg/store` package so that kopia can import
that package itself. Do this by using
interfaces where needed in `pkg/store`
instead of concrete struct types
These changes will make cleaning up
incomplete backups a little neater since
that code will need to lookup both
manifests and backup models
This PR is just minor renaming and fixups,
no logic changes
---
#### Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
#### Type of change
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup
#### Issue(s)
* #3217
#### Test Plan
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [x] 💚 E2E
Only code movement, no code changes.
Moved services to `/internal/m365/services/{onedrive,sharepoint,exchange}`
Moved collections to `/internal/m365/collection/{drive,site}`
---
#### Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
#### Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup
#### Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>
#### Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [ ] ⚡ Unit test
- [ ] 💚 E2E
<!-- PR description-->
GET permissions fails if the item doesn't exist. We are hitting sanity test failures due to a race condition between test run & purge scripts.
`getting item metadata: getting item metadata: error status code received from the API: Item not found:`
Fix here is to ignore GET permissions failure if it returns not found. This is safe to do here, since the graph call for permissions [looks up the item by id](https://github.com/alcionai/corso/blob/main/src/pkg/services/m365/api/drive.go#L254C17-L254C17), and not the permissions by id. Hence, we can safely assume that the item that this permission is tied to has been deleted.
---
#### Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
#### Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
#### Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* https://github.com/alcionai/corso/issues/3998
#### Test Plan
<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ] ⚡ Unit test
- [ ] 💚 E2E
<!-- PR description-->
This PR adds changes to persist backup models for assist backups. Primary reason for doing this is to ensure we don't lose corso extension data(deets) for items which were downloaded and processed by kopia during this backup op.
Note: This is 1/n PRs, and will not be merged into main until all n are reviewed, given we are touching state here.
**Changes in this PR:**
1. Error handling changes to ensure assist backups get persisted
2. Introduce tags for assist/merge backups. These tags will be used for filtering during base selection process.
3. Don't persist backups if they are not assist backup, and they have non zero errors.
4. Remove `LabelForceNoBackupCreation` criteria for not persisting a backup. More details are in [PR comment](https://github.com/alcionai/corso/pull/3905/files#r1278030140).
5. Switch kopia failure from non-recoverable to recoverable. More details are in [PR comment](https://github.com/alcionai/corso/pull/3905/files#r1278019992).
**What are assist backups?**
Below conditions must be true for a backup operation to qualify as an assist backup.
1. new deets were produced during this backup operation
2. we have a valid snapshot ID
3. we don't have any non-recoverable errors
4. we have recoverable errors
**A merge backup satisfies the following conditions:**
1. Is not an assist backup
2. No primary or recoverable errors occurred during the backup operation.
Tests will be added in later PRs.
---
#### Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
#### Type of change
<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
#### Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* internal
#### Test Plan
<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ] ⚡ Unit test
- [ ] 💚 E2E
some code movement and removal in selectors and scopes before adding scope-to-reason. Removed code
was only being used in testing.
---
#### Does this PR need a docs update or release note?
- [x] ⛔ No
#### Type of change
- [x] 🧹 Tech Debt/Cleanup
#### Issue(s)
* #3993
#### Test Plan
- [x] ⚡ Unit test
- [x] 💚 E2E
Moves the kopia.Reasoner interface out of the
kopia package and into pkg/backup/identity.
No logical changes here, just renaming and movement.
---
#### Does this PR need a docs update or release note?
- [x] ⛔ No
#### Type of change
- [x] 🧹 Tech Debt/Cleanup
#### Issue(s)
* #3993
#### Test Plan
- [x] ⚡ Unit test
- [x] 💚 E2E
Add a way for SDK users to drop
kopia-assisted incremental bases
thus forcing item data redownload
if the item wasn't sourced from
a merge base
---
#### Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
#### Type of change
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup
#### Issue(s)
* #2360
#### Test Plan
- [x] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
Now that we store backup models for
backups that had errors filter them
out during backup list. This
ensures behavior doesn't change
when we merge PRs for making and
labeling backup models with the
assist backup tag
---
#### Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
#### Type of change
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
#### Issue(s)
* #3973
#### Test Plan
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
Adding in some basic boilerplate for groups service.
---
#### Does this PR need a docs update or release note?
- [x] ⛔ No
#### Type of change
- [x] 🌻 Feature
Add, parse, and configure info about retention
when initializing a repo. These flags are
currently marked as hidden
Medium-term, they can be used in longevity
tests when making the repo for the first time
(we can configure this out of band if needed
though)
Long-term, they can be exposed to the user
for immutable backup support
---
#### Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [x] 🕐 Yes, but in a later PR
- [ ] ⛔ No
#### Type of change
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
#### Issue(s)
* #3799
#### Test Plan
- [x] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
run `go get -u` to update all packages.
---
#### Does this PR need a docs update or release note?
- [x] ⛔ No
#### Type of change
- [x] 🧹 Tech Debt/Cleanup
Bumps [github.com/microsoftgraph/msgraph-sdk-go](https://github.com/microsoftgraph/msgraph-sdk-go) from 1.12.0 to 1.13.0.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a href="https://github.com/microsoftgraph/msgraph-sdk-go/blob/main/CHANGELOG.md">github.com/microsoftgraph/msgraph-sdk-go's changelog</a>.</em></p>
<blockquote>
<h2>[1.13.0]- 2023-07-28</h2>
<h3>Changed</h3>
<ul>
<li>Weekly generation.</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="e276af0eaf"><code>e276af0</code></a> Generated models and request builders (<a href="https://redirect.github.com/microsoftgraph/msgraph-sdk-go/issues/542">#542</a>)</li>
<li>See full diff in <a href="https://github.com/microsoftgraph/msgraph-sdk-go/compare/v1.12.0...v1.13.0">compare view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
You can trigger a rebase of this PR by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
</details>
Pull in fix for double hashing some values
----
#### Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
#### Type of change
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup
#### Issue(s)
* #3895
#### Test Plan
- [x] 💪 Manual
- [ ] ⚡ Unit test
- [ ] 💚 E2E
Create new operation type to update retention parameters
for immutable backups
---
#### Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
#### Type of change
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
#### Test Plan
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
Take in information about retention so that
when the repo is initialized we can configure
retention
Not currently exposed by CLI layer
**Requires changing the interface for repo
init so SDK consumers will require changes**
---
#### Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
#### Type of change
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
#### Issue(s)
* #3519
#### Test Plan
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
This borrows a lot of the core logic from OneDrive as the internal structure is mostly the same.
<!-- PR description-->
Prev: https://github.com/alcionai/corso/pull/3822
---
#### Does this PR need a docs update or release note?
- [x] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ] ⛔ No
#### Type of change
<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
#### Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* fixes https://github.com/alcionai/corso/issues/3823
#### Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
This adds the final piece to enable OneDrive exports. The CLI interface which consumes NewExport interface from Repository and lets the user to export and write the contents in a backup to the local filesystem.
<!-- PR description-->
Prev: https://github.com/alcionai/corso/pull/3821
Next: https://github.com/alcionai/corso/pull/3824
---
#### Does this PR need a docs update or release note?
- [x] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ] ⛔ No
#### Type of change
<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
#### Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* https://github.com/alcionai/corso/pull/3797
* https://github.com/alcionai/corso/issues/3670
#### Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
Check for a populated StreamStoreID and fallback to the DetailsID if necessary.
---
#### Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
#### Type of change
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
#### Issue(s)
* closes#3913
#### Test Plan
- [x] 💪 Manual
- [ ] ⚡ Unit test
- [ ] 💚 E2E
If the restore configuration specifies a protected resource as a restore target, use that as the destination for the restore. First step is to ensure the provided target can be retrieved and identified.
---
#### Does this PR need a docs update or release note?
- [x] 🕐 Yes, but in a later PR
#### Type of change
- [x] 🌻 Feature
#### Issue(s)
* #3562
#### Test Plan
- [x] ⚡ Unit test
Core logic for exporting data from OneDrive
Next: https://github.com/alcionai/corso/pull/3820
---
#### Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [x] 🕐 Yes, but in a later PR
- [ ] ⛔ No
#### Type of change
<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
#### Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* https://github.com/alcionai/corso/issues/3670
* https://github.com/alcionai/corso/pull/3797
#### Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
add pii concealer compliance for restore config structs.
---
#### Does this PR need a docs update or release note?
- [x] ⛔ No
#### Type of change
- [x] 🌻 Feature
allows cli users to limit the page size of delta queries by calling a new hidden flag: --delta-page-size.
This also adds the control.Options struct to the api client, so that configurations such as this can be easily handed into, and used by, the client.
---
#### Does this PR need a docs update or release note?
- [x] ⛔ No
#### Type of change
- [x] 🌻 Feature
#### Test Plan
- [x] 💪 Manual
- [x] ⚡ Unit test
- [x] 💚 E2E
Adds api handlers for creating document libraries in sharepoint. This is the first step in allowing us to restore drives that were deleted between backup and restore.
---
#### Does this PR need a docs update or release note?
- [x] 🕐 Yes, but in a later PR
#### Type of change
- [x] 🌻 Feature
#### Issue(s)
* #3562
#### Test Plan
- [x] ⚡ Unit test
- [x] 💚 E2E
Code at the kopia wrapper layer that allows configuring kopia
for retention if desired. Includes some tests that require
running against a bucket with object locking enabled
---
#### Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
#### Type of change
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
#### Issue(s)
* #3519
#### Test Plan
- [x] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
ads operations level tests for advanced restore
configuration on all three services.
Code is largely boilerplate between each service, but with just enough quirks that full consolidation would require excess jumping through hoops.
---
#### Does this PR need a docs update or release note?
- [x] ⛔ No
#### Type of change
- [x] 🤖 Supportability/Tests
#### Issue(s)
* #3562
#### Test Plan
- [x] 💚 E2E
http/2 stream error failures don't allow
us to leverage client middleware. As a
result we're lacking a retry handler. This
change adds retries for http_wrapper
in the explicit case of a http/2 stream
error.
---
#### Does this PR need a docs update or release note?
- [x] ✅ Yes, it's included
#### Type of change
- [x] 🐛 Bugfix
#### Test Plan
- [x] 💪 Manual
- [x] 💚 E2E
