Changes:
* Attach URL cache to a collection only if it's parent drive has < 300k items. Higher item count reduces the ROI of using cache, and can even degrade performance in severe cases. We will stick to per item GETs for those scenarios, until we figure out a longer term solution.
*
---
#### Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
#### Type of change
<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
#### Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* https://github.com/alcionai/corso/issues/3069
#### Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [x] 💚 E2E
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.44.283 to 1.44.284.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/aws/aws-sdk-go/releases">github.com/aws/aws-sdk-go's releases</a>.</em></p>
<blockquote>
<h1>Release v1.44.284 (2023-06-16)</h1>
<h3>Service Client Updates</h3>
<ul>
<li><code>service/account</code>: Updates service paginators</li>
<li><code>service/connect</code>: Updates service API and documentation</li>
<li><code>service/discovery</code>: Updates service API and documentation
<ul>
<li>Add Amazon EC2 instance recommendations export</li>
</ul>
</li>
<li><code>service/iam</code>: Updates service documentation
<ul>
<li>Documentation updates for AWS Identity and Access Management (IAM).</li>
</ul>
</li>
<li><code>service/s3</code>: Updates service API, documentation, and examples
<ul>
<li>This release adds SDK support for request-payer request header and request-charged response header in the "GetBucketAccelerateConfiguration", "ListMultipartUploads", "ListObjects", "ListObjectsV2" and "ListObjectVersions" S3 APIs.</li>
</ul>
</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="87a671cef0"><code>87a671c</code></a> Release v1.44.284 (2023-06-16) (<a href="https://redirect.github.com/aws/aws-sdk-go/issues/4887">#4887</a>)</li>
<li>See full diff in <a href="https://github.com/aws/aws-sdk-go/compare/v1.44.283...v1.44.284">compare view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
You can trigger a rebase of this PR by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
</details>
This ensures that we backup and restore and exceptions in the recurring events. One thing pending here is fixing up the attachments. I hope to create that as a separate PR. This can probably go in separately as well.
---
#### Does this PR need a docs update or release note?
- [x] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ] ⛔ No
#### Type of change
<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
#### Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* https://github.com/alcionai/corso/issues/2835
#### Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [x] 💚 E2E
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.44.282 to 1.44.283.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/aws/aws-sdk-go/releases">github.com/aws/aws-sdk-go's releases</a>.</em></p>
<blockquote>
<h1>Release v1.44.283 (2023-06-15)</h1>
<h3>Service Client Updates</h3>
<ul>
<li><code>service/auditmanager</code>: Updates service API and documentation</li>
<li><code>service/elasticfilesystem</code>: Updates service documentation
<ul>
<li>Documentation updates for EFS.</li>
</ul>
</li>
<li><code>service/guardduty</code>: Updates service documentation
<ul>
<li>Updated descriptions for some APIs.</li>
</ul>
</li>
<li><code>service/location</code>: Updates service API and documentation</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="efb91b58d6"><code>efb91b5</code></a> Release v1.44.283 (2023-06-15) (<a href="https://redirect.github.com/aws/aws-sdk-go/issues/4884">#4884</a>)</li>
<li>See full diff in <a href="https://github.com/aws/aws-sdk-go/compare/v1.44.282...v1.44.283">compare view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
You can trigger a rebase of this PR by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
</details>
Bumps [golang.org/x/tools](https://github.com/golang/tools) from 0.9.3 to 0.10.0.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/golang/tools/releases">golang.org/x/tools's releases</a>.</em></p>
<blockquote>
<h2>gopls/v0.10.0</h2>
<p>This release contains initial support for standalone packages and package renaming. Please see below for more details.</p>
<p>We are also changing our release policy to better align with semver.</p>
<h2>Support changes</h2>
<p>This version of gopls contains changes to our release policy, deprecates support for some older Go versions, and deprecates support for several experimental features.</p>
<h3>New release policy</h3>
<p>As described in <a href="https://redirect.github.com/golang/go/issues/55267">golang/go#55267</a>, we are tightening our release policy to better follow semver, increase stability, and reduce release-related toil. Significant new features will only be introduced in <code>*.*.0</code> patch versions, and subsequent patch releases will consist only of bugfixes. For example, this version (v0.10.0) introduces several new features, described below. Subsequent v0.10.* releases will contain only bugfixes.</p>
<h3>Final support for Go 1.13-1.15</h3>
<p>Consistent with the above release policy and our stated <a href="https://github.com/golang/tools/tree/master/gopls#supported-go-versions-and-build-systems">support window</a>, the <code>v0.10.*</code> minor version will be the final set of releases to support being used with Go 1.13-1.15. See <a href="https://redirect.github.com/golang/go/issues/52982">golang/go#52982</a> for details.</p>
<p>Gopls will pop up a warning if it resolves a version of the <code>go</code> command that is older than 1.16. Starting with gopls@v0.11.0, gopls will cease to function when used with a <code>go</code> command with a version older than 1.16.</p>
<h3>Deprecated experimental features</h3>
<p>The following experimental features are deprecated, and will be removed in gopls@v0.11.0:</p>
<ul>
<li><strong>experimentalWorkspaceModule</strong><code>golang/go#52897</code><code>go.work</code> files. See our <a href="https://github.com/golang/tools/blob/master/gopls/doc/workspace.md#go-workspaces-go-118">documentation</a> for information on how to use <code>go.work</code> files to work on multiple modules.</li>
<li><strong>experimentalWatchedFileDelay</strong><code>golang/go#55268</code><code>workspace/didChangeWatchedFiles</code> notifications.</li>
<li><strong>experimentalUseInvalidMetadata</strong><code>golang/go#54180</code></li>
</ul>
<h2>New Features</h2>
<h3>Support for "standalone packages"</h3>
<p>Gopls now recognizes certain files as "standalone main packages", meaning they should be interpreted as main packages consisting of a single file. To do this, gopls looks for packages named <code>main</code> containing a single build constraint that matches one of the tags configured by the new <a href="https://github.com/golang/tools/blob/master/gopls/doc/settings.md#standalonetags-string"><code>standaloneTags</code></a> setting.</p>
<p>This enables cross references and other features when working in a file such as the example below that contains a <code>//go:build ignore</code> build constraint.</p>
<p><img src="https://user-images.githubusercontent.com/57144380/197039428-72d78300-cbea-4abe-a408-e171991094b8.png" alt="image" /></p>
<h3>(preview) Support for package renaming</h3>
<p><code>golang/go#41567</code><a href="https://redirect.github.com/golang/go/issues/56184">golang/go#56184</a>.</p>
<p>To rename a package, initiate a rename request on the package clause of a file in the package:
<img src="https://user-images.githubusercontent.com/57144380/197041786-bf9e617c-9961-46ae-b21a-c240a18a3014.png" alt="image" /></p>
<p>When this renaming is applied, gopls will adjust other package files accordingly, rename the package directory, and update import paths referencing the current package or any nested package in the renamed directory.
<img src="https://user-images.githubusercontent.com/57144380/197042160-e034e38d-3cd3-4c62-b912-747d8dc2f4ff.png" alt="image" /></p>
<h3>Method information in hover</h3>
<p>Hovering over a type now shows a summary of its methods.</p>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="7261b32692"><code>7261b32</code></a> gopls/internal/regtest: fix goimports on windows when using vendoring</li>
<li><a href="41e4e56549"><code>41e4e56</code></a> gopls/internal/lsp/source/completion: ensuring completion completeness</li>
<li><a href="ac2946029a"><code>ac29460</code></a> go/ssa: fix bug in writeSignature on external functions</li>
<li><a href="3b62e7e256"><code>3b62e7e</code></a> go/ssa: use core type within (*builder).receiver</li>
<li><a href="f394d451f8"><code>f394d45</code></a> gopls/internal/lsp/cache: compute xrefs and methodsets asynchronously</li>
<li><a href="27dbf85279"><code>27dbf85</code></a> go.mod: update golang.org/x dependencies</li>
<li><a href="c6c9830549"><code>c6c9830</code></a> go/types/objectpath: memoize scope lookup in objectpath.Encoder</li>
<li><a href="0245e1dfc6"><code>0245e1d</code></a> gopls/internal/regtest/codelens: set GOWORK=off for go mod vendor</li>
<li><a href="85be8882c9"><code>85be888</code></a> go/analysis/passes/defers: add analyser for defer mistake</li>
<li><a href="c43232f868"><code>c43232f</code></a> cmd/digraph: improve examples using go list, mod</li>
<li>Additional commits viewable in <a href="https://github.com/golang/tools/compare/v0.9.3...v0.10.0">compare view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
You can trigger a rebase of this PR by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
</details>
Bumps [github.com/microsoft/kiota-serialization-json-go](https://github.com/microsoft/kiota-serialization-json-go) from 1.0.1 to 1.0.2.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/microsoft/kiota-serialization-json-go/releases">github.com/microsoft/kiota-serialization-json-go's releases</a>.</em></p>
<blockquote>
<h2>v1.0.2</h2>
<ul>
<li>Safely serialize null values in collections of Objects, Enums or primitives.</li>
</ul>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a href="https://github.com/microsoft/kiota-serialization-json-go/blob/main/CHANGELOG.md">github.com/microsoft/kiota-serialization-json-go's changelog</a>.</em></p>
<blockquote>
<h2>[1.0.2] - 2023-06-14</h2>
<ul>
<li>Safely serialize null values in collections of Objects, Enums or primitives.</li>
</ul>
<h3>Changed</h3>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="7f47527b69"><code>7f47527</code></a> Fix/null serialization collection (<a href="https://redirect.github.com/microsoft/kiota-serialization-json-go/issues/92">#92</a>)</li>
<li><a href="e5fc62c297"><code>e5fc62c</code></a> Merge pull request <a href="https://redirect.github.com/microsoft/kiota-serialization-json-go/issues/90">#90</a> from microsoft/dependabot/go_modules/github.com/stretc...</li>
<li><a href="0fc7f862f0"><code>0fc7f86</code></a> Bump github.com/stretchr/testify from 1.8.3 to 1.8.4</li>
<li>See full diff in <a href="https://github.com/microsoft/kiota-serialization-json-go/compare/v1.0.1...v1.0.2">compare view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
You can trigger a rebase of this PR by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
</details>
This should help in CI and other environments where the TLS handshake appears to take longer under load
---
#### Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
#### Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
#### Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [ ] ⚡ Unit test
- [x] 💚 E2E
Separates the pager and enumerattion functionality from the rest of the exchange api funcs for each
category. Purely code movement, no logic changes.
---
#### Does this PR need a docs update or release note?
- [x] ⛔ No
#### Type of change
- [x] 🧹 Tech Debt/Cleanup
#### Issue(s)
* #3562
#### Test Plan
- [x] ⚡ Unit test
- [x] 💚 E2E
Adds collision policy handling to onedrive item posts. This allows us to override the default "replace" behavior that currently returns a 409 for the creation endpoint, in case we want to use skip (ie: fail) or copy handling.
---
#### Does this PR need a docs update or release note?
- [x] ⛔ No
#### Type of change
- [x] 🌻 Feature
#### Issue(s)
* #3562
#### Test Plan
- [x] 💪 Manual
- [x] ⚡ Unit test
- [x] 💚 E2E
Missed adding the foldermatcher to the handlers
for onedrive and sharepoint during the api refactor.
---
#### Does this PR need a docs update or release note?
- [x] ⛔ No
#### Type of change
- [x] 🧹 Tech Debt/Cleanup
#### Issue(s)
* #1996
#### Test Plan
- [x] ⚡ Unit test
- [x] 💚 E2E
Now that BackupBases defines functions, leverage them
in other code to reduce the number of times we fetch
Backup models and leverage the stronger invariants
the new FindBases function has
---
#### Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
#### Type of change
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup
#### Issue(s)
* #3525
#### Test Plan
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
https://github.com/alcionai/corso/pull/3607#discussion_r1229799665
---
#### Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ] ⛔ No
#### Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
#### Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>
#### Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [ ] ⚡ Unit test
- [ ] 💚 E2E
Adds a func that normalizes the restore config in case inputs don't match accepted or expected values.
Also moves DefaultTestRestoreConfig out of /tester and into /control/testdata to avoid circular dependency issues.
---
#### Does this PR need a docs update or release note?
- [x] ⛔ No
#### Type of change
- [x] 🌻 Feature
#### Issue(s)
* #3562
#### Test Plan
- [x] ⚡ Unit test
PR contents
1. Address a corner case where cache may be half filled ( e.g. scenario: delta query failing after a few pages). Empty the cache on any delta failures
2. Add unit tests
---
#### Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
#### Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
#### Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>
#### Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
<!-- PR description-->
Some events in have a empty repoID. This could be because of multiple
config files used. And new config files might have repoID missing.
Solution- Fetch repoID from Kopia is not found locally in the config
file.
Note: Since the Corso Start event happens before we connect to Kopia, it
might have repoID missing if the config file is used for the first time.
All the consecutive events will have correct values. Corso start event
will start populating correct values if the config file is used once.
#### Does this PR need a docs update or release note?
- [ ] ⛔ No
#### Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🐛 Bugfix
#### Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic
words" - "closes, fixes" to auto-close the Github issue. -->
* https://github.com/alcionai/corso/issues/3388
#### Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
Loosen restrictions slightly by returning snapshots that are missing their bakcup or details models as assist bases that can be used by kopia assisted incrementals.
---
#### Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
#### Type of change
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
#### Issue(s)
* #3525
#### Test Plan
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
Had other names because they were made when other similar test code existed.
---
#### Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
#### Type of change
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup
#### Issue(s)
* #3525
#### Test Plan
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
This last step in the cleanup has two goals: first to minimize the number of arbitrarily named and located files so that code is better condensed and co-located. This is all just file renaming and a minor amount of code copy-pasting.
The second is to create a centralized package to own the ColInfo type structs that we use to both stub out test data, and also generate factory data for cmds. The current ownership is haphazard, and while this movement is a little more condensed, it's still jumping through some weird hoops to get things to work. Treat it as one good step forward, and we'll have to return to polish it another time. At least it'll be separated from the m365 folder at large this way, and more easily identified as supporting design rather than production usage.
---
#### Does this PR need a docs update or release note?
- [x] ⛔ No
#### Type of change
- [x] 🧹 Tech Debt/Cleanup
#### Issue(s)
* closes#1996
#### Test Plan
- [x] ⚡ Unit test
- [x] 💚 E2E
Bumps [sass](https://github.com/sass/dart-sass) from 1.63.3 to 1.63.4.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/sass/dart-sass/releases">sass's releases</a>.</em></p>
<blockquote>
<h2>Dart Sass 1.63.4</h2>
<p>To install Sass 1.63.4, download one of the packages below and <a href="https://katiek2.github.io/path-doc/">add it to your PATH</a>, or see <a href="https://sass-lang.com/install">the Sass website</a> for full installation instructions.</p>
<h1>Changes</h1>
<h3>JavaScript API</h3>
<ul>
<li>
<p>Re-enable support for <code>import sass from 'sass'</code> when loading the package from an ESM module in Node.js. However, this syntax is now deprecated; ESM users should use <code>import * as sass from 'sass'</code> instead.</p>
<p>On the browser and other ESM-only platforms, only <code>import * as sass from 'sass'</code> is supported.</p>
</li>
<li>
<p>Properly export the legacy API values <code>TRUE</code>, <code>FALSE</code>, <code>NULL</code>, and <code>types</code> from the ECMAScript module API.</p>
</li>
</ul>
<h3>Embedded Sass</h3>
<ul>
<li>
<p>Fix a race condition where closing standard input while requests are in-flight could sometimes cause the process to hang rather than shutting down gracefully.</p>
</li>
<li>
<p>Properly include the root stylesheet's URL in the set of loaded URLs when it fails to parse.</p>
</li>
</ul>
<p>See the <a href="https://github.com/sass/dart-sass/blob/master/CHANGELOG.md#1634">full changelog</a> for changes in earlier releases.</p>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a href="https://github.com/sass/dart-sass/blob/main/CHANGELOG.md">sass's changelog</a>.</em></p>
<blockquote>
<h2>1.63.4</h2>
<h3>JavaScript API</h3>
<ul>
<li>
<p>Re-enable support for <code>import sass from 'sass'</code> when loading the package from
an ESM module in Node.js. However, this syntax is now deprecated; ESM users
should use <code>import * as sass from 'sass'</code> instead.</p>
<p>On the browser and other ESM-only platforms, only <code>import * as sass from 'sass'</code> is supported.</p>
</li>
<li>
<p>Properly export the legacy API values <code>TRUE</code>, <code>FALSE</code>, <code>NULL</code>, and <code>types</code> from
the ECMAScript module API.</p>
</li>
</ul>
<h3>Embedded Sass</h3>
<ul>
<li>
<p>Fix a race condition where closing standard input while requests are in-flight
could sometimes cause the process to hang rather than shutting down
gracefully.</p>
</li>
<li>
<p>Properly include the root stylesheet's URL in the set of loaded URLs when it
fails to parse.</p>
</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="e34a0c7fa9"><code>e34a0c7</code></a> Cut a release (<a href="https://redirect.github.com/sass/dart-sass/issues/2016">#2016</a>)</li>
<li><a href="3de612e67c"><code>3de612e</code></a> Re-enable support for default exports (<a href="https://redirect.github.com/sass/dart-sass/issues/2009">#2009</a>)</li>
<li><a href="ce58d87dcf"><code>ce58d87</code></a> Add missing ESM exports (<a href="https://redirect.github.com/sass/dart-sass/issues/2010">#2010</a>)</li>
<li><a href="aa59a5f4a9"><code>aa59a5f</code></a> Fix race condition between spawning and killing isolates during shutdown (<a href="https://redirect.github.com/sass/dart-sass/issues/2007">#2007</a>)</li>
<li><a href="760fa2ead1"><code>760fa2e</code></a> Add current stylesheet to loadedUrls when throwing parsing error (<a href="https://redirect.github.com/sass/dart-sass/issues/1991">#1991</a>)</li>
<li>See full diff in <a href="https://github.com/sass/dart-sass/compare/1.63.3...1.63.4">compare view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
You can trigger a rebase of this PR by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
</details>
These files from what we can understand are not available and thus can't be fetched. This change ensures that we don't fail the backup just because of this error.
<!-- PR description-->
---
#### Does this PR need a docs update or release note?
- [x] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ] ⛔ No
#### Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
#### Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>
#### Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
automatically log when we add a recoverable error or a skipped item to fault. This log will include a stack trace of the call from the location of the logged recoverable. Clues does not have a method for pulling a stack trace out of an error yet; that can be added at a future date.
---
#### Does this PR need a docs update or release note?
- [x] ⛔ No
#### Type of change
- [x] 🤖 Supportability/Tests
#### Test Plan
- [x] ⚡ Unit test
- [x] 💚 E2E
Release notes: https://tip.golang.org/doc/go1.20
Changes required:
- Updated `go.mod` - this also controls what CI uses
- `rand` is now automatically seeded
---
#### Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
#### Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [x] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
#### Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #3568
#### Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [x] 💚 E2E
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.44.281 to 1.44.282.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/aws/aws-sdk-go/releases">github.com/aws/aws-sdk-go's releases</a>.</em></p>
<blockquote>
<h1>Release v1.44.282 (2023-06-13)</h1>
<h3>Service Client Updates</h3>
<ul>
<li><code>service/cloudtrail</code>: Updates service API and documentation
<ul>
<li>This feature allows users to view dashboards for CloudTrail Lake event data stores.</li>
</ul>
</li>
<li><code>service/codeguru-security</code>: Adds new service</li>
<li><code>service/drs</code>: Updates service API, documentation, and paginators</li>
<li><code>service/ec2</code>: Updates service API, documentation, and paginators
<ul>
<li>This release introduces a new feature, EC2 Instance Connect Endpoint, that enables you to connect to a resource over TCP, without requiring the resource to have a public IPv4 address.</li>
</ul>
</li>
<li><code>service/imagebuilder</code>: Updates service documentation</li>
<li><code>service/lightsail</code>: Updates service API and documentation
<ul>
<li>This release adds pagination for the Get Certificates API operation.</li>
</ul>
</li>
<li><code>service/s3</code>: Updates service API and examples
<ul>
<li>Integrate double encryption feature to SDKs.</li>
</ul>
</li>
<li><code>service/securityhub</code>: Updates service API, documentation, and examples</li>
<li><code>service/simspaceweaver</code>: Updates service API and documentation</li>
<li><code>service/verifiedpermissions</code>: Adds new service</li>
<li><code>service/wafv2</code>: Updates service API and documentation</li>
<li><code>service/wellarchitected</code>: Updates service API, documentation, and paginators</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="8962904930"><code>8962904</code></a> Release v1.44.282 (2023-06-13) (<a href="https://redirect.github.com/aws/aws-sdk-go/issues/4882">#4882</a>)</li>
<li>See full diff in <a href="https://github.com/aws/aws-sdk-go/compare/v1.44.281...v1.44.282">compare view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
You can trigger a rebase of this PR by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
</details>
Bumps [golang.org/x/net](https://github.com/golang/net) from 0.10.0 to 0.11.0.
<details>
<summary>Commits</summary>
<ul>
<li><a href="6c96ca5daf"><code>6c96ca5</code></a> go.mod: update golang.org/x dependencies</li>
<li><a href="5541298b83"><code>5541298</code></a> quic: add packet pacer</li>
<li><a href="88a50b6484"><code>88a50b6</code></a> all: update x/sys to HEAD</li>
<li><a href="7e6923f9c4"><code>7e6923f</code></a> quic: add RTT estimator</li>
<li><a href="2796e09d6a"><code>2796e09</code></a> bpf: check for little endian CPU for OS VM comparison</li>
<li><a href="10cf388024"><code>10cf388</code></a> quic: add a data structure for tracking lists of sent packets</li>
<li><a href="ccc217c97e"><code>ccc217c</code></a> quic: parameterize rangeset</li>
<li><a href="f16447cf6c"><code>f16447c</code></a> quic: add go1.21 build constraint</li>
<li><a href="f7250ea19d"><code>f7250ea</code></a> quic: add a type tracking sent values</li>
<li><a href="1b5a2d8538"><code>1b5a2d8</code></a> quic: packet encoding/decoding</li>
<li>Additional commits viewable in <a href="https://github.com/golang/net/compare/v0.10.0...v0.11.0">compare view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
You can trigger a rebase of this PR by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
</details>
renames /internal/connector to /internal/m365. No logic changes in this PR. Only the dir rename, import renames, and one linter shadowing rename.
---
#### Does this PR need a docs update or release note?
- [x] ⛔ No
#### Type of change
- [x] 🧹 Tech Debt/Cleanup
#### Issue(s)
* #1996
#### Test Plan
- [x] ⚡ Unit test
- [x] 💚 E2E
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.44.280 to 1.44.281.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/aws/aws-sdk-go/releases">github.com/aws/aws-sdk-go's releases</a>.</em></p>
<blockquote>
<h1>Release v1.44.281 (2023-06-12)</h1>
<h3>Service Client Updates</h3>
<ul>
<li><code>service/amplifyuibuilder</code>: Updates service API, documentation, and paginators</li>
<li><code>service/dynamodb</code>: Updates service API, documentation, waiters, paginators, and examples
<ul>
<li>Documentation updates for DynamoDB</li>
</ul>
</li>
<li><code>service/fsx</code>: Updates service API and documentation</li>
<li><code>service/opensearch</code>: Updates service API and documentation</li>
<li><code>service/rekognition</code>: Updates service API, documentation, paginators, and examples
<ul>
<li>This release adds support for improved accuracy with user vector in Amazon Rekognition Face Search. Adds new APIs: AssociateFaces, CreateUser, DeleteUser, DisassociateFaces, ListUsers, SearchUsers, SearchUsersByImage. Also adds new face metadata that can be stored: user vector.</li>
</ul>
</li>
<li><code>service/sagemaker</code>: Updates service API and documentation
<ul>
<li>Sagemaker Neo now supports compilation for inferentia2 (ML_INF2) and Trainium1 (ML_TRN1) as available targets. With these devices, you can run your workloads at highest performance with lowest cost. inferentia2 (ML_INF2) is available in CMH and Trainium1 (ML_TRN1) is available in IAD currently</li>
</ul>
</li>
<li><code>service/streams.dynamodb</code>: Updates service documentation</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="dd4970433c"><code>dd49704</code></a> Release v1.44.281 (2023-06-12) (<a href="https://redirect.github.com/aws/aws-sdk-go/issues/4881">#4881</a>)</li>
<li>See full diff in <a href="https://github.com/aws/aws-sdk-go/compare/v1.44.280...v1.44.281">compare view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
You can trigger a rebase of this PR by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
</details>
Surface the note about link shares mentioned in [changelog](4545feeb80/CHANGELOG.md (L163)) to known issues.
<!-- PR description-->
---
#### Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ] ⛔ No
#### Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
#### Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>
#### Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [ ] ⚡ Unit test
- [ ] 💚 E2E
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.44.276 to 1.44.280.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/aws/aws-sdk-go/releases">github.com/aws/aws-sdk-go's releases</a>.</em></p>
<blockquote>
<h1>Release v1.44.280 (2023-06-09)</h1>
<h3>Service Client Updates</h3>
<ul>
<li><code>service/acm-pca</code>: Updates service documentation</li>
<li><code>service/connect</code>: Updates service API, documentation, and paginators</li>
</ul>
<h1>Release v1.44.279 (2023-06-08)</h1>
<h3>Service Client Updates</h3>
<ul>
<li><code>service/athena</code>: Updates service API and documentation
<ul>
<li>You can now define custom spark properties at start of the session for use cases like cluster encryption, table formats, and general Spark tuning.</li>
</ul>
</li>
<li><code>service/comprehendmedical</code>: Updates service API</li>
<li><code>service/payment-cryptography</code>: Adds new service</li>
<li><code>service/payment-cryptography-data</code>: Adds new service</li>
<li><code>service/servicecatalog</code>: Updates service API and documentation
<ul>
<li>New parameter added in ServiceCatalog DescribeProvisioningArtifact api - IncludeProvisioningArtifactParameters. This parameter can be used to return information about the parameters used to provision the product</li>
</ul>
</li>
<li><code>service/timestream-write</code>: Updates service API and documentation</li>
</ul>
<h1>Release v1.44.278 (2023-06-07)</h1>
<h3>Service Client Updates</h3>
<ul>
<li><code>service/cloudformation</code>: Updates service API and documentation
<ul>
<li>AWS CloudFormation StackSets is updating the deployment experience for all stackset operations to skip suspended AWS accounts during deployments. StackSets will skip target AWS accounts that are suspended and set the Detailed Status of the corresponding stack instances as SKIPPED_SUSPENDED_ACCOUNT</li>
</ul>
</li>
<li><code>service/customer-profiles</code>: Updates service API, documentation, and paginators</li>
<li><code>service/directconnect</code>: Updates service documentation
<ul>
<li>This update corrects the jumbo frames mtu values from 9100 to 8500 for transit virtual interfaces.</li>
</ul>
</li>
<li><code>service/emr-containers</code>: Updates service API and documentation</li>
<li><code>service/iotdeviceadvisor</code>: Updates service API and documentation</li>
<li><code>service/logs</code>: Updates service API and documentation
<ul>
<li>This change adds support for account level data protection policies using 3 new APIs, PutAccountPolicy, DeleteAccountPolicy and DescribeAccountPolicy. DescribeLogGroup API has been modified to indicate if account level policy is applied to the LogGroup via "inheritedProperties" list in the response.</li>
</ul>
</li>
</ul>
<h1>Release v1.44.277 (2023-06-06)</h1>
<h3>Service Client Updates</h3>
<ul>
<li><code>service/connect</code>: Updates service documentation</li>
<li><code>service/elasticmapreduce</code>: Updates service API and documentation
<ul>
<li>This release provides customers the ability to specify an allocation strategies amongst PRICE_CAPACITY_OPTIMIZED, CAPACITY_OPTIMIZED, LOWEST_PRICE, DIVERSIFIED for Spot instances in Instance Feet cluster. This enables customers to choose an allocation strategy best suited for their workload.</li>
</ul>
</li>
<li><code>service/iam</code>: Updates service API and documentation
<ul>
<li>This release updates the AccountAlias regex pattern with the same length restrictions enforced by the length constraint.</li>
</ul>
</li>
<li><code>service/inspector2</code>: Updates service API and documentation</li>
<li><code>service/iot</code>: Updates service API, documentation, and paginators
<ul>
<li>Adding IoT Device Management Software Package Catalog APIs to register, store, and report system software packages, along with their versions and metadata in a centralized location.</li>
</ul>
</li>
<li><code>service/iot-data</code>: Updates service API</li>
<li><code>service/models.lex.v2</code>: Updates service API, documentation, and paginators</li>
<li><code>service/quicksight</code>: Updates service API and documentation
<ul>
<li>QuickSight support for pivot table field collapse state, radar chart range scale and multiple scope options in conditional formatting.</li>
</ul>
</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="12e9c10fa1"><code>12e9c10</code></a> Release v1.44.280 (2023-06-09) (<a href="https://redirect.github.com/aws/aws-sdk-go/issues/4880">#4880</a>)</li>
<li><a href="a5bef15429"><code>a5bef15</code></a> Release v1.44.279 (2023-06-08) (<a href="https://redirect.github.com/aws/aws-sdk-go/issues/4878">#4878</a>)</li>
<li><a href="f9eb254e13"><code>f9eb254</code></a> Release v1.44.278 (2023-06-07) (<a href="https://redirect.github.com/aws/aws-sdk-go/issues/4876">#4876</a>)</li>
<li><a href="12004a7b5d"><code>12004a7</code></a> Release v1.44.277 (2023-06-06) (<a href="https://redirect.github.com/aws/aws-sdk-go/issues/4874">#4874</a>)</li>
<li>See full diff in <a href="https://github.com/aws/aws-sdk-go/compare/v1.44.276...v1.44.280">compare view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
You can trigger a rebase of this PR by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
</details>
Move most of the stuff that was acting on backup bases to be functions that are defined for backup bases. Other code can be removed at a later point
New functions aren't called yet in other code
---
#### Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
#### Type of change
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup
#### Issue(s)
* #3525
#### Test Plan
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
centralizes all restore configuration management within a restoreConfig struct. This struct is owned by the control package, which allows it to be utilized by both CLI and SDK consumers.
---
#### Does this PR need a docs update or release note?
- [x] ⛔ No
#### Type of change
- [x] 🧹 Tech Debt/Cleanup
#### Issue(s)
* #3562
#### Test Plan
- [x] ⚡ Unit test
- [x] 💚 E2E
Keeps the same auto-merge behavior (for `semver-minor` updates) but changes it to use our mergequeue.
This improves DX because it prevents dependabot merges from bypassing the queue and also means that sanity
tests for these PRs don't fail (they run with the correct env)
---
#### Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ] ⛔ No
#### Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
#### Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #3591
#### Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [ ] ⚡ Unit test
- [ ] 💚 E2E
Retry on `ECONNRESET` even if HTTP response is set to 2xx. This is unlikely but it can happen. Currently we fail the backup for this scenario with `connection reset by peer` error.
---
#### Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
#### Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
#### Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>
#### Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
This was only used in tests and so should not affect any code, but let's fix it.
---
#### Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
#### Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
#### Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>
#### Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
Corso detects malware in a few places for OD/SP.
1. During item enumeration, using [malware property](https://learn.microsoft.com/en-us/graph/api/resources/driveitem?view=graph-rest-1.0#properties) which is set by graph - [code](ed47c134b0/src/internal/connector/onedrive/collections.go (L658))
2. During item fetch using download URL - [code](ed47c134b0/src/internal/connector/onedrive/item.go (LL53C18-L53C18))
- Graph seems to return 403 with `{"error":{"code":"malwareDetected","message":"Malware detected"}}`. Tested with eicar.
3. By analyzing http response headers in `IsMalwareResp` Don't see these headers documented by graph. This is for [sharepoint protocol compatibility](https://learn.microsoft.com/en-us/openspecs/sharepoint_protocols/ms-wdvmoduu/6fa6d4a9-ac18-4cd7-b696-8a3b14a98291) it seems.
4. By analyzing response body in `IsMalwareResp` . We check for malwareDetected string inside http response body.
We are accidentally dumping entire file ( http response dump) while checking for malware in 4. This is leading to high mem utilization/OOMs, especially while processing very large files.
I don't think we need 4 at all. If graph is the entity which detects malware, I think we will never get past 2. This PR removes item 4.
---
#### Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
#### Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
#### Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>
#### Test Plan
<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ] ⚡ Unit test
- [ ] 💚 E2E
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.44.275 to 1.44.276.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/aws/aws-sdk-go/releases">github.com/aws/aws-sdk-go's releases</a>.</em></p>
<blockquote>
<h1>Release v1.44.276 (2023-06-05)</h1>
<h3>Service Client Updates</h3>
<ul>
<li><code>service/cloudformation</code>: Updates service API and documentation
<ul>
<li>AWS CloudFormation StackSets provides customers with three new APIs to activate, deactivate, and describe AWS Organizations trusted access which is needed to get started with service-managed StackSets.</li>
</ul>
</li>
<li><code>service/ec2</code>: Updates service API
<ul>
<li>Making InstanceTagAttribute as the required parameter for the DeregisterInstanceEventNotificationAttributes and RegisterInstanceEventNotificationAttributes APIs.</li>
</ul>
</li>
<li><code>service/finspace</code>: Updates service API, documentation, and paginators</li>
<li><code>service/frauddetector</code>: Updates service API and documentation</li>
<li><code>service/keyspaces</code>: Updates service API and documentation</li>
<li><code>service/kms</code>: Updates service API, documentation, and examples
<ul>
<li>This release includes feature to import customer's asymmetric (RSA and ECC) and HMAC keys into KMS. It also includes feature to allow customers to specify number of days to schedule a KMS key deletion as a policy condition key.</li>
</ul>
</li>
<li><code>service/lambda</code>: Updates service API and documentation
<ul>
<li>Add Ruby 3.2 (ruby3.2) Runtime support to AWS Lambda.</li>
</ul>
</li>
<li><code>service/mwaa</code>: Updates service API and documentation</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="d4aaf18d8d"><code>d4aaf18</code></a> Release v1.44.276 (2023-06-05) (<a href="https://redirect.github.com/aws/aws-sdk-go/issues/4873">#4873</a>)</li>
<li>See full diff in <a href="https://github.com/aws/aws-sdk-go/compare/v1.44.275...v1.44.276">compare view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
You can trigger a rebase of this PR by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
</details>
Remove a reference to localhost
#### Does this PR need a docs update or release note?
- [x] ⛔ No
#### Type of change
- [x] 🐛 Bugfix
- [x] 🗺️ Documentation
Adds a website name for Google to pick up.
---
#### Does this PR need a docs update or release note?
- [x] ⛔ No
#### Type of change
- [x] 🗺️ Documentation
Bumps [github.com/microsoftgraph/msgraph-sdk-go](https://github.com/microsoftgraph/msgraph-sdk-go) from 1.3.0 to 1.4.0.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a href="https://github.com/microsoftgraph/msgraph-sdk-go/blob/main/CHANGELOG.md">github.com/microsoftgraph/msgraph-sdk-go's changelog</a>.</em></p>
<blockquote>
<h2>[1.4.0]- 2023-05-30</h2>
<h3>Changed</h3>
<ul>
<li>Weekly generation.</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="0dfacb5e1b"><code>0dfacb5</code></a> Generated models and request builders (<a href="https://redirect.github.com/microsoftgraph/msgraph-sdk-go/issues/502">#502</a>)</li>
<li><a href="674ff171e4"><code>674ff17</code></a> Merge pull request <a href="https://redirect.github.com/microsoftgraph/msgraph-sdk-go/issues/499">#499</a> from microsoftgraph/dependabot/go_modules/github.com/...</li>
<li><a href="2263771027"><code>2263771</code></a> Bump github.com/microsoft/kiota-serialization-json-go</li>
<li><a href="213f447dbe"><code>213f447</code></a> Merge pull request <a href="https://redirect.github.com/microsoftgraph/msgraph-sdk-go/issues/498">#498</a> from microsoftgraph/dependabot/github_actions/dependa...</li>
<li><a href="cb4d466c30"><code>cb4d466</code></a> Bump dependabot/fetch-metadata from 1.5.0 to 1.5.1</li>
<li>See full diff in <a href="https://github.com/microsoftgraph/msgraph-sdk-go/compare/v1.3.0...v1.4.0">compare view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
You can trigger a rebase of this PR by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
</details>
Don't read all the files that we need to restore, just the collections in the beginning. The rest of them are read using Fetch on demand when they will be restored.
I haven't tested for any changes in memory consumption, but this brings down time taken for "Enumerating items in repository" to <5s even for huge number of files.
<!-- PR description-->
---
#### Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [x] 🕐 Yes, but in a later PR
- [ ] ⛔ No
#### Type of change
<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
#### Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* https://github.com/alcionai/corso/issues/3011
* closes https://github.com/alcionai/corso/issues/3440
* closes https://github.com/alcionai/corso/issues/3537
#### Test Plan
<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ] ⚡ Unit test
- [x] 💚 E2E
<!-- PR description-->
---
#### Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ] ⛔ No
#### Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
#### Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>
#### Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [ ] ⚡ Unit test
- [ ] 💚 E2E
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.44.274 to 1.44.275.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/aws/aws-sdk-go/releases">github.com/aws/aws-sdk-go's releases</a>.</em></p>
<blockquote>
<h1>Release v1.44.275 (2023-06-02)</h1>
<h3>Service Client Updates</h3>
<ul>
<li><code>service/athena</code>: Updates service API and documentation
<ul>
<li>This release introduces the DeleteCapacityReservation API and the ability to manage capacity reservations using CloudFormation</li>
</ul>
</li>
<li><code>service/cloudtrail</code>: Updates service API and documentation
<ul>
<li>This feature allows users to start and stop event ingestion on a CloudTrail Lake event data store.</li>
</ul>
</li>
<li><code>service/sagemaker</code>: Updates service API and documentation
<ul>
<li>This release adds Selective Execution feature that allows SageMaker Pipelines users to run selected steps in a pipeline.</li>
</ul>
</li>
<li><code>service/wafv2</code>: Updates service API and documentation</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="0af146fcb0"><code>0af146f</code></a> Release v1.44.275 (2023-06-02) (<a href="https://redirect.github.com/aws/aws-sdk-go/issues/4870">#4870</a>)</li>
<li>See full diff in <a href="https://github.com/aws/aws-sdk-go/compare/v1.44.274...v1.44.275">compare view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
You can trigger a rebase of this PR by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
</details>
This refactor is being done to allow calling `downloadFile` with an URL obtained from URL cache.
---
#### Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
#### Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup
#### Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>
#### Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
