## Description
This fixes tying to parse link shares as permissions and some error/retry handling.
## Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>
## Test Plan
<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
## Description
Add backup format version information to the backups so that we can distinguish between backups which use a single file vs the ones that use both .data and .meta files.
Overrides https://github.com/alcionai/corso/pull/2297. I've also set it against `main` so that the diff shows up properly.
Ref: https://github.com/alcionai/corso/pull/2324#issuecomment-1409709118
## Does this PR need a docs update or release note?
- [x] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ] ⛔ No
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* fixes https://github.com/alcionai/corso/issues/2230
* https://github.com/alcionai/corso/issues/2253
## Test Plan
<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual (Tested manually, will add an e2e test in a followup)
- [ ] ⚡ Unit test
- [ ] 💚 E2E
## Description
This PR adds option to backup and restore additional metadata(currently permissions) for OneDrive.
**This PR also adds the ability to restore empty folders for OneDrive along with their permissions.**
Breaking change: Any old backups will not work as we expect both `.data` and `.meta`/`.dirmeta` files to be available for OneDrive backups.
## Does this PR need a docs update or release note?
*Added changelog, docs pending.*
- [x] ✅ Yes, it's included
- [x] 🕐 Yes, but in a later PR
- [ ] ⛔ No
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* https://github.com/alcionai/corso/issues/1774
## Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [x] 💚 E2E
## Description
new blog post on storage options for corso
## Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [x] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>
## Test Plan
<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ] ⚡ Unit test
- [ ] 💚 E2E
## Description
Prevents showing the log file location when calling commands that defer to help output or env details.
Certain cases aren't caught, such as when calling a command with no flags (ex: `corso backup create exchange`).
In those cases we catch the lack of flags and manually display the usage within the command handler.
## Does this PR need a docs update or release note?
- [x] ⛔ No
## Type of change
- [x] 🧹 Tech Debt/Cleanup
## Issue(s)
* #2329
## Test Plan
- [x] 💪 Manual
## Description
Updates within /internal/connector/exchange package. Adds helper function for displaying the internal object item type that is included as an attachment. This will help overall with supporting additional `ItemAttachment` objects.
<!-- Insert PR description-->
## Does this PR need a docs update or release note?
- [x] ⛔ No
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [x] 🧹 Tech Debt/Cleanup
## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* related to #2353<issue>
## Test Plan
<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.44.191 to 1.44.192.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/aws/aws-sdk-go/releases">github.com/aws/aws-sdk-go's releases</a>.</em></p>
<blockquote>
<h1>Release v1.44.192 (2023-02-01)</h1>
<h3>Service Client Updates</h3>
<ul>
<li><code>service/devops-guru</code>: Updates service API and documentation</li>
<li><code>service/forecast</code>: Updates service API and documentation</li>
<li><code>service/iam</code>: Updates service documentation
<ul>
<li>Documentation updates for AWS Identity and Access Management (IAM).</li>
</ul>
</li>
<li><code>service/mediatailor</code>: Updates service API and documentation</li>
<li><code>service/sns</code>: Updates service documentation
<ul>
<li>Additional attributes added for set-topic-attributes.</li>
</ul>
</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="9fac3673d9"><code>9fac367</code></a> Release v1.44.192 (2023-02-01) (<a href="https://github-redirect.dependabot.com/aws/aws-sdk-go/issues/4712">#4712</a>)</li>
<li>See full diff in <a href="https://github.com/aws/aws-sdk-go/compare/v1.44.191...v1.44.192">compare view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
You can trigger a rebase of this PR by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
</details>
## Description
Standardize on `go-humanize`
## Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup
## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #1038
## Test Plan
<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ] ⚡ Unit test
- [ ] 💚 E2E
## Description
`Item.Attachments` have unique properties that require transformation to be uploaded into the M365 system. All objects of this type are temporarily prevented from uploading via this patch. As Issue #2353 is addressed, there will be more coverage for the various types of item attachments.
<!-- Insert PR description-->
## Does this PR need a docs update or release note?
- [x] ✅ Yes, it's included
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
*related to #2353<issue>
## Test Plan
<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
## Description
Create a test case that allows checking most of the Get call (getting
item data is not mocked right now). This allows better checking on
things like:
* having multiple drives to backup
* having items split across multiple delta query pages
* errors returned by the API
* aggregation of delta URLs and folder paths in metadata
Eventually these tests could help with checking:
* consumption of metadata for incremental backups
* consumption of delta tokens for incremental backups
Long-term, we may want to merge these with the UpdateCollections tests
as there is overlap between the two.
We may also want to consider a more stable API for checking the
returned items. Right now it partly relies on being able to cast to a
onedrive.Collection struct instead of using `Items()`
## Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
## Type of change
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
## Issue(s)
* #2264
## Test Plan
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
## Description
Bug fix: Ensures that `item.Attachment` content is completely backed up.
- Exchange
Changes affect the behavior of Mail and Events. Downloads can be potentially bigger if the user have these types of attachments.
- `item.Attachments` are specific to Outlook objects such as `Events` or `Messages` are included as an attachment rather than inline.
<!-- Insert PR description-->
## Does this PR need a docs update or release note?
- [x] ✅ Yes, it's included
## Type of change
- [x] 🐛 Bugfix
## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* related to #2353<issue>
## Test Plan
<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
## Description
Currently no folder path entry is stored for the root folder. This leads to not having a folders map but having a valid delta token if all items are in the root of the drive.
Update the code to add at least an empty folders map entry so we don't think we have missing metadata. Add tests to check for these edge cases.
Long-term we may want to add a folder entry for the root folder. Whether we do or not may depend on how we decide to set the state field for that collection.
## Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
## Type of change
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
## Issue(s)
* #2122
## Test Plan
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
## Description
Create a pager for drive items that allows for better testing via mocking. Increased testing will come in later PRs
## Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
## Type of change
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
## Issue(s)
* #2264
## Test Plan
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [x] 💚 E2E
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.44.190 to 1.44.191.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/aws/aws-sdk-go/releases">github.com/aws/aws-sdk-go's releases</a>.</em></p>
<blockquote>
<h1>Release v1.44.191 (2023-01-31)</h1>
<h3>Service Client Updates</h3>
<ul>
<li><code>service/accessanalyzer</code>: Adds new service</li>
<li><code>service/appsync</code>: Updates service API and documentation</li>
<li><code>service/cloudtrail</code>: Updates service API and documentation
<ul>
<li>Add new "Channel" APIs to enable users to manage channels used for CloudTrail Lake integrations, and "Resource Policy" APIs to enable users to manage the resource-based permissions policy attached to a channel.</li>
</ul>
</li>
<li><code>service/cloudtrail-data</code>: Adds new service</li>
<li><code>service/codeartifact</code>: Updates service API and documentation</li>
<li><code>service/connectparticipant</code>: Adds new service</li>
<li><code>service/ec2</code>: Updates service API and documentation
<ul>
<li>This launch allows customers to associate up to 8 IP addresses to their NAT Gateways to increase the limit on concurrent connections to a single destination by eight times from 55K to 440K.</li>
</ul>
</li>
<li><code>service/groundstation</code>: Updates service API and documentation</li>
<li><code>service/iot</code>: Updates service API and documentation
<ul>
<li>Added support for IoT Rules Engine Cloudwatch Logs action batch mode.</li>
</ul>
</li>
<li><code>service/kinesis</code>: Adds new service
<ul>
<li>Enabled FIPS endpoints for GovCloud (US) regions in SDK.</li>
</ul>
</li>
<li><code>service/opensearch</code>: Updates service API and documentation</li>
<li><code>service/outposts</code>: Adds new service</li>
<li><code>service/polly</code>: Updates service API
<ul>
<li>Amazon Polly adds two new neural American English voices - Ruth, Stephen</li>
</ul>
</li>
<li><code>service/sagemaker</code>: Updates service API and documentation
<ul>
<li>Amazon SageMaker Automatic Model Tuning now supports more completion criteria for Hyperparameter Optimization.</li>
</ul>
</li>
<li><code>service/securityhub</code>: Updates service API and documentation</li>
<li><code>service/support</code>: Adds new service
<ul>
<li>This fixes incorrect endpoint construction when a customer is explicitly setting a region.</li>
</ul>
</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="4d8fb842b8"><code>4d8fb84</code></a> Release v1.44.191 (2023-01-31) (<a href="https://github-redirect.dependabot.com/aws/aws-sdk-go/issues/4710">#4710</a>)</li>
<li>See full diff in <a href="https://github.com/aws/aws-sdk-go/compare/v1.44.190...v1.44.191">compare view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
You can trigger a rebase of this PR by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
</details>
## Description
Add some nuance to log file docs.
## Does this PR need a docs update or release note?
- [x] ⛔ No
## Type of change
- [x] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
## Issue(s)
* #2329
## Description
Adds a log message to collection updates to indicate how many items we have streamed into the repository
## Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #2329
## Test Plan
<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ] ⚡ Unit test
- [ ] 💚 E2E
## Description
Adds logging to delta enumeration to log every 1000 items or so (each page is set to 200 items)
## Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #2329
## Test Plan
<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ] ⚡ Unit test
- [ ] 💚 E2E
## Does this PR need a docs update or release note?
- [x] ⛔ No
## Type of change
- [x] 🧹 Tech Debt/Cleanup
## Issue(s)
* #2329
## Test Plan
- [x] 💪 Manual
## Does this PR need a docs update or release note?
- [x] ⛔ No
## Type of change
- [x] 🧹 Tech Debt/Cleanup
## Issue(s)
* #2329
## Test Plan
- [x] 💪 Manual
## Does this PR need a docs update or release note?
- [x] ⛔ No
## Type of change
- [x] 🧹 Tech Debt/Cleanup
## Issue(s)
* #2329
## Test Plan
- [x] 💪 Manual
## Description
This package is mostly generated code, but since it's not formatted and check in it causes subsequent runs of gofmt to format it and add chaff to git diff. Checking in the formatted version removes the chaff.
## Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
## Type of change
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup
## Description
Reduces the number of roundtrips when requesting delta records for exchange
TODO: Need to investigate OneDrive behavior - this header doesn't appear to work with OneDrive.
## Does this PR need a docs update or release note?
- [x] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ] ⛔ No
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #2332
## Test Plan
<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ] ⚡ Unit test
- [x] 💚 E2E
## Description
Create helper functions to deserialize OneDrive metadata during subsequent backups. Currently deserialized data is not passed to the function that generates Collections nor is metadata passed in even though it's wired through GraphConnector
Additional changes to BackupOp and operations/manifests.go are required to begin passing in metadata
## Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
## Type of change
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
## Issue(s)
* closes#2122
## Test Plan
- [x] 💪 Manual
- [ ] ⚡ Unit test
- [ ] 💚 E2E
## Description
Begins updating operations/backup with the new
error handling procedures. For backwards
compatibility, errors are currently duplicated in
the old stats.Errs and the new Errors struct.
## Does this PR need a docs update or release note?
- [x] ⛔ No
## Type of change
- [x] 🧹 Tech Debt/Cleanup
## Issue(s)
* #1970
## Test Plan
- [x] ⚡ Unit test
- [x] 💚 E2E
## Description
Added retry handling for delta queries in OneDrive. Also, bumping time timeout for graph api calls from 90s to 3m as we were seeing client timeouts for graph api calls. ~Haven't added retry for every request in exchange as I'm hoping https://github.com/alcionai/corso/issues/2287 will be a better way to handle this.~
## Does this PR need a docs update or release note?
- [x] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ] ⛔ No
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>
## Test Plan
<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ] ⚡ Unit test
- [ ] 💚 E2E
## Description
Adds the fault.Errors struct (now exported) to the
operations base. stats.Errs is retained in the
backup and restore wrappers to avoid breaking
changes and allow for deserialization. We will
continue to use the current error return until
dependencies are fully updated to use Errors.
## Does this PR need a docs update or release note?
- [x] ⛔ No
## Type of change
- [x] 🧹 Tech Debt/Cleanup
## Issue(s)
* #1970
## Test Plan
- [x] ⚡ Unit test
- [x] 💚 E2E
## Description
Creates Beta Service for betasdk package. Abstraction loosely complies with the methods found in `graph.Servicer()`.
Will aid when Sharepoint functionality is supported in v1.0 and `betasdk` package is removed.
<!-- Insert PR description-->
## Does this PR need a docs update or release note?
- [x] ⛔ No
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
- related to * #2174<issue>
## Test Plan
<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
## Does this PR need a docs update or release note?
- [x] ⛔ No
## Type of change
- [x] 🧹 Tech Debt/Cleanup
## Issue(s)
* #2329
## Test Plan
- [x] 💪 Manual
## Description
To add needed features, Kiota-Generated Beta Connector created
<!-- Insert PR description-->
## Does this PR need a docs update or release note?
- [x] ✅ Yes, it's included
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* closes #2086<issue>
* closes#2174
* closes#2223
## Test Plan
Compilation and current CI tests will fail if the package does not align with current operational standards.
- [x] ⚡ Unit test
- [x] 💚 E2E
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.44.189 to 1.44.190.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/aws/aws-sdk-go/releases">github.com/aws/aws-sdk-go's releases</a>.</em></p>
<blockquote>
<h1>Release v1.44.190 (2023-01-30)</h1>
<h3>Service Client Updates</h3>
<ul>
<li><code>service/clouddirectory</code>: Adds new service
<ul>
<li>Enabled FIPS endpoints for GovCloud (US) regions in SDK.</li>
</ul>
</li>
<li><code>service/cloudformation</code>: Updates service API and documentation
<ul>
<li>This feature provides a method of obtaining which regions a stackset has stack instances deployed in.</li>
</ul>
</li>
<li><code>service/discovery</code>: Updates service API
<ul>
<li>Update ImportName validation to 255 from the current length of 100</li>
</ul>
</li>
<li><code>service/dlm</code>: Adds new service</li>
<li><code>service/ec2</code>: Updates service API and documentation
<ul>
<li>We add Prefix Lists as a new route destination option for LocalGatewayRoutes. This will allow customers to create routes to Prefix Lists. Prefix List routes will allow customers to group individual CIDR routes with the same target into a single route.</li>
</ul>
</li>
<li><code>service/imagebuilder</code>: Adds new service</li>
<li><code>service/kafka</code>: Adds new service</li>
<li><code>service/mediaconvert</code>: Adds new service
<ul>
<li>Enabled FIPS endpoints for GovCloud (US) regions in SDK.</li>
</ul>
</li>
<li><code>service/swf</code>: Adds new service
<ul>
<li>Enabled FIPS endpoints for GovCloud (US) regions in SDK.</li>
</ul>
</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="76684dd605"><code>76684dd</code></a> Release v1.44.190 (2023-01-30) (<a href="https://github-redirect.dependabot.com/aws/aws-sdk-go/issues/4707">#4707</a>)</li>
<li>See full diff in <a href="https://github.com/aws/aws-sdk-go/compare/v1.44.189...v1.44.190">compare view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
You can trigger a rebase of this PR by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
</details>
## Description
Add docs on Coso's fault tolerance and reliability
## Does this PR need a docs update or release note?
- [x] ✅ Yes, it's included
## Type of change
- [x] 🗺️ Documentation
## Description
Bumps ok-to-test to v3. Plus fix the issue where someone might have been able to run unverified code if they push between commenting `/ok-to-test` and the job starting.
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [x] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor
## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* https://github.com/alcionai/corso/pull/1651
## Test Plan
<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ] ⚡ Unit test
- [ ] 💚 E2E
## Description
unskips the backup load tests for a single user.
Also adds a wrapper to make toggling restore
load tests controllable for each test suite,
instead of a file-wide configuration.
## Does this PR need a docs update or release note?
- [x] ⛔ No
## Type of change
- [x] 🤖 Test
## Issue(s)
* #2279
## Test Plan
- [x] 💚 E2E
## Description
Return an item exclude list from GraphConnector to BackupOp. BackupOp does not yet pass this to kopia wrapper.
Returned list is set to nil (eventually) by all components so even if this were wired to kopia wrapper it wouldn't change the current behavior of the system
## Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
## Type of change
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
## Issue(s)
* #2243
merge after:
* #2143
## Test Plan
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
## Description
Minor refactor that came up while hunting bugs.
For the life of me, I cannot find the reason for
read/write counts being off in event incrementals.
## Does this PR need a docs update or release note?
- [x] ⛔ No
## Type of change
- [x] 🧹 Tech Debt/Cleanup
## Issue(s)
* #2022
## Test Plan
- [x] 💪 Manual
- [x] 💚 E2E
## Description
This flag is confusing, and is better represented by
tracking errors.
## Type of change
- [x] 🧹 Tech Debt/Cleanup
## Test Plan
- [x] ⚡ Unit test
## Description
Clarify how Corso repositories and object storage buckets are related.
## Does this PR need a docs update or release note?
- [x] ⛔ No
## Type of change
- [x] 🗺️ Documentation
BUG FIX: Adds clause to return an error if experienced during path building of a collection.
## Description
<!-- Insert PR description-->
## Does this PR need a docs update or release note?
- [x] ⛔ No
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [x] 🐛 Bugfix
- related to #2257
## Test Plan
<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.44.188 to 1.44.189.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/aws/aws-sdk-go/releases">github.com/aws/aws-sdk-go's releases</a>.</em></p>
<blockquote>
<h1>Release v1.44.189 (2023-01-27)</h1>
<h3>Service Client Updates</h3>
<ul>
<li><code>service/application-autoscaling</code>: Adds new service</li>
<li><code>service/appstream</code>: Updates service waiters
<ul>
<li>Fixing the issue where Appstream waiters hang for fleet_started and fleet_stopped.</li>
</ul>
</li>
<li><code>service/elasticbeanstalk</code>: Adds new service
<ul>
<li>Enabled FIPS endpoints for GovCloud (US) regions in SDK.</li>
</ul>
</li>
<li><code>service/fis</code>: Adds new service</li>
<li><code>service/glacier</code>: Adds new service
<ul>
<li>Enabled FIPS endpoints for GovCloud (US) regions in SDK.</li>
</ul>
</li>
<li><code>service/greengrass</code>: Adds new service
<ul>
<li>Enabled FIPS endpoints for GovCloud (US) regions in SDK.</li>
</ul>
</li>
<li><code>service/greengrassv2</code>: Adds new service</li>
<li><code>service/mediatailor</code>: Updates service API and documentation</li>
<li><code>service/outposts</code>: Updates service API and documentation</li>
<li><code>service/runtime.sagemaker</code>: Updates service API and documentation</li>
<li><code>service/sagemaker</code>: Updates service API and documentation
<ul>
<li>This release supports running SageMaker Training jobs with container images that are in a private Docker registry.</li>
</ul>
</li>
<li><code>service/serverlessrepo</code>: Adds new service</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="d4386d55dd"><code>d4386d5</code></a> Release v1.44.189 (2023-01-27) (<a href="https://github-redirect.dependabot.com/aws/aws-sdk-go/issues/4706">#4706</a>)</li>
<li>See full diff in <a href="https://github.com/aws/aws-sdk-go/compare/v1.44.188...v1.44.189">compare view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
You can trigger a rebase of this PR by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
</details>
## Description
separates item get/retry when iterating over
items in exchange collection streams, and adds
a mocked unit test on that func.
## Does this PR need a docs update or release note?
- [x] ⛔ No
## Type of change
- [x] 🤖 Test
## Issue(s)
* closes#2299
## Test Plan
- [x] ⚡ Unit test
## Description
Create a shared set of helper functions and interfaces for getting next and delta links from Graph API responses. This helps standardize how they are handled with respect to nil/empty values and makes comparisons easier in other code
## Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
## Type of change
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup
## Issue(s)
* #2264
## Test Plan
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [x] 💚 E2E
## Description
Make a separate interface for fetching drive information from Graph API. Interface allows for better testing via mocks
Merges SharePoint and OneDrive code for getting drives.
Also fixes potential bug where not all drives would be fetched. This could have occurred because the previous implementation for both SharePoint and OneDrive were not checking for paginated results
Viewing by commit is recommended
## Does this PR need a docs update or release note?
- [x] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ] ⛔ No
## Type of change
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Test
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup
## Issue(s)
* #2264
## Test Plan
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
