1363 Commits

Author SHA1 Message Date
ashmrtn
24c918e99c
Patch and test some edge cases for deserializing OneDrive metadata (#2341)
## Description

Currently no folder path entry is stored for the root folder. This leads to not having a folders map but having a valid delta token if all items are in the root of the drive.

Update the code to add at least an empty folders map entry so we don't think we have missing metadata. Add tests to check for these edge cases.

Long-term we may want to add a folder entry for the root folder. Whether we do or not may depend on how we decide to set the state field for that collection.

## Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No 

## Type of change

- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

## Issue(s)

* #2122 

## Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-02-01 18:57:38 +00:00
ashmrtn
1594a86c22
OneDrive Items API for mocking (#2322)
## Description

Create a pager for drive items that allows for better testing via mocking. Increased testing will come in later PRs

## Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No 

## Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

## Issue(s)

* #2264

## Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2023-02-01 17:03:23 +00:00
dependabot[bot]
3fd3da7caf
⬆️ Bump github.com/aws/aws-sdk-go from 1.44.190 to 1.44.191 in /src (#2351)
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.44.190 to 1.44.191.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/aws/aws-sdk-go/releases">github.com/aws/aws-sdk-go's releases</a>.</em></p>
<blockquote>
<h1>Release v1.44.191 (2023-01-31)</h1>
<h3>Service Client Updates</h3>
<ul>
<li><code>service/accessanalyzer</code>: Adds new service</li>
<li><code>service/appsync</code>: Updates service API and documentation</li>
<li><code>service/cloudtrail</code>: Updates service API and documentation
<ul>
<li>Add new &quot;Channel&quot; APIs to enable users to manage channels used for CloudTrail Lake integrations, and &quot;Resource Policy&quot; APIs to enable users to manage the resource-based permissions policy attached to a channel.</li>
</ul>
</li>
<li><code>service/cloudtrail-data</code>: Adds new service</li>
<li><code>service/codeartifact</code>: Updates service API and documentation</li>
<li><code>service/connectparticipant</code>: Adds new service</li>
<li><code>service/ec2</code>: Updates service API and documentation
<ul>
<li>This launch allows customers to associate up to 8 IP addresses to their NAT Gateways to increase the limit on concurrent connections to a single destination by eight times from 55K to 440K.</li>
</ul>
</li>
<li><code>service/groundstation</code>: Updates service API and documentation</li>
<li><code>service/iot</code>: Updates service API and documentation
<ul>
<li>Added support for IoT Rules Engine Cloudwatch Logs action batch mode.</li>
</ul>
</li>
<li><code>service/kinesis</code>: Adds new service
<ul>
<li>Enabled FIPS endpoints for GovCloud (US) regions in SDK.</li>
</ul>
</li>
<li><code>service/opensearch</code>: Updates service API and documentation</li>
<li><code>service/outposts</code>: Adds new service</li>
<li><code>service/polly</code>: Updates service API
<ul>
<li>Amazon Polly adds two new neural American English voices - Ruth, Stephen</li>
</ul>
</li>
<li><code>service/sagemaker</code>: Updates service API and documentation
<ul>
<li>Amazon SageMaker Automatic Model Tuning now supports more completion criteria for Hyperparameter Optimization.</li>
</ul>
</li>
<li><code>service/securityhub</code>: Updates service API and documentation</li>
<li><code>service/support</code>: Adds new service
<ul>
<li>This fixes incorrect endpoint construction when a customer is explicitly setting a region.</li>
</ul>
</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="4d8fb842b8"><code>4d8fb84</code></a> Release v1.44.191 (2023-01-31) (<a href="https://github-redirect.dependabot.com/aws/aws-sdk-go/issues/4710">#4710</a>)</li>
<li>See full diff in <a href="https://github.com/aws/aws-sdk-go/compare/v1.44.190...v1.44.191">compare view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=github.com/aws/aws-sdk-go&package-manager=go_modules&previous-version=1.44.190&new-version=1.44.191)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

You can trigger a rebase of this PR by commenting `@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


</details>
2023-02-01 16:33:53 +00:00
Niraj Tolia
d918001ee4
Improve log file docs (#2347)
## Description

Add some nuance to log file docs.

## Does this PR need a docs update or release note?

- [x]  No 

## Type of change

- [x] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

## Issue(s)

* #2329
2023-02-01 05:32:17 +00:00
dependabot[bot]
45eb71f1c2
⬆️ Bump sass from 1.57.1 to 1.58.0 in /website (#2350) 2023-02-01 05:18:49 +00:00
Vaibhav Kamra
2873befe53
Log collection updates (#2346)
## Description

Adds a log message to collection updates to indicate how many items we have streamed into the repository

## Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No 

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #2329 

## Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-02-01 03:57:19 +00:00
Vaibhav Kamra
4ab5040569
Add progress logging to delta queries (#2345)
## Description

Adds logging to delta enumeration to log every 1000 items or so (each page is set to 200 items)

## Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No 

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #2329 

## Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-02-01 03:39:00 +00:00
Keepers
7ee1575dc2
log end-of-observe even when hidden (#2339)
## Does this PR need a docs update or release note?

- [x]  No 

## Type of change

- [x] 🧹 Tech Debt/Cleanup

## Issue(s)

* #2329

## Test Plan

- [x] 💪 Manual
2023-02-01 02:41:06 +00:00
Keepers
ec7d3c6fc5
add logging around operation completion (#2337)
## Does this PR need a docs update or release note?

- [x]  No 

## Type of change

- [x] 🧹 Tech Debt/Cleanup

## Issue(s)

* #2329

## Test Plan

- [x] 💪 Manual
2023-02-01 02:19:44 +00:00
Keepers
4ac81e6253
print location of log file (#2343)
## Does this PR need a docs update or release note?

- [x]  No 

## Type of change

- [x] 🧹 Tech Debt/Cleanup

## Issue(s)

* #2329

## Test Plan

- [x] 💪 Manual
2023-02-01 01:47:49 +00:00
ashmrtn
b5b4576393
Run gofmt on beta package (#2340)
## Description

This package is mostly generated code, but since it's not formatted and check in it causes subsequent runs of gofmt to format it and add chaff to git diff. Checking in the formatted version removes the chaff.

## Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No 

## Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup
2023-02-01 01:25:27 +00:00
Vaibhav Kamra
c510af3fda
Bump up max page size preference for exchange delta requests (#2338)
## Description

Reduces the number of roundtrips when requesting delta records for exchange

TODO: Need to investigate OneDrive behavior - this header doesn't appear to work with OneDrive.

## Does this PR need a docs update or release note?

- [x]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No 

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #2332 

## Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ]  Unit test
- [x] 💚 E2E
2023-02-01 01:01:31 +00:00
ashmrtn
387f8e8cd7
Deserialize OneDrive metadata during backup (#2263)
## Description

Create helper functions to deserialize OneDrive metadata during subsequent backups. Currently deserialized data is not passed to the function that generates Collections nor is metadata passed in even though it's wired through GraphConnector

Additional changes to BackupOp and operations/manifests.go are required to begin passing in metadata

## Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No 

## Type of change

- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

## Issue(s)

* closes #2122

## Test Plan

- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-01-31 22:48:30 +00:00
Keepers
070b8fddee
update operation/backup errs (#2239)
## Description

Begins updating operations/backup with the new
error handling procedures.  For backwards
compatibility, errors are currently duplicated in
the old stats.Errs and the new Errors struct.

## Does this PR need a docs update or release note?

- [x]  No 

## Type of change

- [x] 🧹 Tech Debt/Cleanup

## Issue(s)

* #1970

## Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-01-31 22:25:02 +00:00
Abin Simon
f3b2e9a632
Retry handling for delta queries in Exchange (#2328)
## Description

Added retry handling for delta queries in OneDrive. Also, bumping time timeout for graph api calls from 90s to 3m as we were seeing client timeouts for graph api calls. ~Haven't added retry for every request in exchange as I'm hoping https://github.com/alcionai/corso/issues/2287 will be a better way to handle this.~

## Does this PR need a docs update or release note?

- [x]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No 

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

## Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-01-31 21:57:38 +00:00
Keepers
498fc325a9
store errors in operation (#2237)
## Description

Adds the fault.Errors struct (now exported) to the
operations base.  stats.Errs is retained in the
backup and restore wrappers to avoid breaking
changes and allow for deserialization.  We will
continue to use the current error return until
dependencies are fully updated to use Errors.

## Does this PR need a docs update or release note?

- [x]  No 

## Type of change

- [x] 🧹 Tech Debt/Cleanup

## Issue(s)

* #1970

## Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-01-31 21:22:53 +00:00
Danny
013eeab7cb
GC: Add Service to graph/betasdk package (#2298)
## Description
Creates Beta Service for betasdk package. Abstraction loosely complies with the methods found in `graph.Servicer()`. 
Will aid when Sharepoint functionality is supported in v1.0 and `betasdk` package is removed. 
<!-- Insert PR description-->

## Does this PR need a docs update or release note?
- [x]  No 

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
- related to * #2174<issue>

## Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
2023-01-31 20:46:31 +00:00
Keepers
c270536f73
add version to clues, log cli command and flags (#2331)
## Does this PR need a docs update or release note?

- [x]  No 

## Type of change

- [x] 🧹 Tech Debt/Cleanup

## Issue(s)

* #2329 

## Test Plan

- [x] 💪 Manual
2023-01-31 20:21:01 +00:00
Danny
3a5211590a
GC: Beta Client Package (#2205)
## Description
To add needed features, Kiota-Generated Beta Connector created
<!-- Insert PR description-->

## Does this PR need a docs update or release note?

- [x]  Yes, it's included


## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* closes  #2086<issue>
* closes #2174 
* closes #2223

## Test Plan
Compilation and current CI tests will fail if the package does not align with current operational standards. 
- [x]  Unit test
- [x] 💚 E2E
2023-01-31 16:38:28 +00:00
dependabot[bot]
14be75e7ae
⬆️ Bump github.com/aws/aws-sdk-go from 1.44.189 to 1.44.190 in /src (#2327)
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.44.189 to 1.44.190.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/aws/aws-sdk-go/releases">github.com/aws/aws-sdk-go's releases</a>.</em></p>
<blockquote>
<h1>Release v1.44.190 (2023-01-30)</h1>
<h3>Service Client Updates</h3>
<ul>
<li><code>service/clouddirectory</code>: Adds new service
<ul>
<li>Enabled FIPS endpoints for GovCloud (US) regions in SDK.</li>
</ul>
</li>
<li><code>service/cloudformation</code>: Updates service API and documentation
<ul>
<li>This feature provides a method of obtaining which regions a stackset has stack instances deployed in.</li>
</ul>
</li>
<li><code>service/discovery</code>: Updates service API
<ul>
<li>Update ImportName validation to 255 from the current length of 100</li>
</ul>
</li>
<li><code>service/dlm</code>: Adds new service</li>
<li><code>service/ec2</code>: Updates service API and documentation
<ul>
<li>We add Prefix Lists as a new route destination option for LocalGatewayRoutes. This will allow customers to create routes to Prefix Lists. Prefix List routes will allow customers to group individual CIDR routes with the same target into a single route.</li>
</ul>
</li>
<li><code>service/imagebuilder</code>: Adds new service</li>
<li><code>service/kafka</code>: Adds new service</li>
<li><code>service/mediaconvert</code>: Adds new service
<ul>
<li>Enabled FIPS endpoints for GovCloud (US) regions in SDK.</li>
</ul>
</li>
<li><code>service/swf</code>: Adds new service
<ul>
<li>Enabled FIPS endpoints for GovCloud (US) regions in SDK.</li>
</ul>
</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="76684dd605"><code>76684dd</code></a> Release v1.44.190 (2023-01-30) (<a href="https://github-redirect.dependabot.com/aws/aws-sdk-go/issues/4707">#4707</a>)</li>
<li>See full diff in <a href="https://github.com/aws/aws-sdk-go/compare/v1.44.189...v1.44.190">compare view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=github.com/aws/aws-sdk-go&package-manager=go_modules&previous-version=1.44.189&new-version=1.44.190)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

You can trigger a rebase of this PR by commenting `@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


</details>
2023-01-31 16:14:29 +00:00
dependabot[bot]
0f39b23a00
⬆️ Bump @iconify/react from 4.0.1 to 4.1.0 in /website (#2326) 2023-01-31 05:16:37 +00:00
dependabot[bot]
bf404c0602
⬆️ Bump @loadable/component from 5.15.2 to 5.15.3 in /website (#2325) 2023-01-31 05:11:15 +00:00
Niraj Tolia
1fd640a754
Add docs page for fault-tolerance (#2313)
## Description

Add docs on Coso's fault tolerance and reliability

## Does this PR need a docs update or release note?

- [x]  Yes, it's included

## Type of change

- [x] 🗺️ Documentation
2023-01-31 01:53:02 +00:00
Abin Simon
da0fac20bf
Bump slash-command-dispatch and avoid running unverified commit in ok-to-test (#1756)
## Description

Bumps ok-to-test to v3. Plus fix the issue where someone might  have been able to run unverified code if they push between commenting `/ok-to-test` and the job starting.

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [x] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* https://github.com/alcionai/corso/pull/1651

## Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-01-31 01:42:04 +00:00
Keepers
48e6e8f08f
enable onedrive backup for single user (#2317)
## Description

unskips the backup load tests for a single user.
Also adds a wrapper to make toggling restore
load tests controllable for each test suite,
instead of a file-wide configuration.

## Does this PR need a docs update or release note?

- [x]  No 

## Type of change

- [x] 🤖 Test

## Issue(s)

* #2279

## Test Plan

- [x] 💚 E2E
2023-01-31 01:23:08 +00:00
ashmrtn
2458508969
Wire up GraphConnector <-> BackupOp item exclude list (#2245)
## Description

Return an item exclude list from GraphConnector to BackupOp. BackupOp does not yet pass this to kopia wrapper.

Returned list is set to nil (eventually) by all components so even if this were wired to kopia wrapper it wouldn't change the current behavior of the system

## Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No 

## Type of change

- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

## Issue(s)

* #2243

merge after:
* #2143 

## Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-01-30 19:44:06 +00:00
Keepers
03642c517f
standardize deleteContainer func in exch api (#2246)
## Description

Minor refactor that came up while hunting bugs.
For the life of me, I cannot find the reason for
read/write counts being off in event incrementals.

## Does this PR need a docs update or release note?

- [x]  No 

## Type of change

- [x] 🧹 Tech Debt/Cleanup

## Issue(s)

* #2022

## Test Plan

- [x] 💪 Manual
- [x] 💚 E2E
2023-01-30 19:24:44 +00:00
Keepers
637b1904aa
remove operation "started" (#2247)
## Description

This flag is confusing, and is better represented by
tracking errors.

## Type of change

- [x] 🧹 Tech Debt/Cleanup

## Test Plan

- [x]  Unit test
2023-01-30 18:57:45 +00:00
Niraj Tolia
ac1ff0c5cc
Add guidance on using Corso repos (#2315)
## Description

Clarify how Corso repositories and object storage buckets are related.

## Does this PR need a docs update or release note?

- [x]  No 

## Type of change

- [x] 🗺️ Documentation
2023-01-30 17:03:18 +00:00
Danny
d763d95976
BUG Fix: Restore: Operations (#2316)
BUG FIX: Adds clause to return an error if experienced during path building of a collection. 

## Description

<!-- Insert PR description-->

## Does this PR need a docs update or release note?
- [x]  No 

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🐛 Bugfix
- related to #2257
## Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
2023-01-30 16:51:32 +00:00
dependabot[bot]
27e935301c
⬆️ Bump github.com/aws/aws-sdk-go from 1.44.188 to 1.44.189 in /src (#2314)
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.44.188 to 1.44.189.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/aws/aws-sdk-go/releases">github.com/aws/aws-sdk-go's releases</a>.</em></p>
<blockquote>
<h1>Release v1.44.189 (2023-01-27)</h1>
<h3>Service Client Updates</h3>
<ul>
<li><code>service/application-autoscaling</code>: Adds new service</li>
<li><code>service/appstream</code>: Updates service waiters
<ul>
<li>Fixing the issue where Appstream waiters hang for fleet_started and fleet_stopped.</li>
</ul>
</li>
<li><code>service/elasticbeanstalk</code>: Adds new service
<ul>
<li>Enabled FIPS endpoints for GovCloud (US) regions in SDK.</li>
</ul>
</li>
<li><code>service/fis</code>: Adds new service</li>
<li><code>service/glacier</code>: Adds new service
<ul>
<li>Enabled FIPS endpoints for GovCloud (US) regions in SDK.</li>
</ul>
</li>
<li><code>service/greengrass</code>: Adds new service
<ul>
<li>Enabled FIPS endpoints for GovCloud (US) regions in SDK.</li>
</ul>
</li>
<li><code>service/greengrassv2</code>: Adds new service</li>
<li><code>service/mediatailor</code>: Updates service API and documentation</li>
<li><code>service/outposts</code>: Updates service API and documentation</li>
<li><code>service/runtime.sagemaker</code>: Updates service API and documentation</li>
<li><code>service/sagemaker</code>: Updates service API and documentation
<ul>
<li>This release supports running SageMaker Training jobs with container images that are in a private Docker registry.</li>
</ul>
</li>
<li><code>service/serverlessrepo</code>: Adds new service</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="d4386d55dd"><code>d4386d5</code></a> Release v1.44.189 (2023-01-27) (<a href="https://github-redirect.dependabot.com/aws/aws-sdk-go/issues/4706">#4706</a>)</li>
<li>See full diff in <a href="https://github.com/aws/aws-sdk-go/compare/v1.44.188...v1.44.189">compare view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=github.com/aws/aws-sdk-go&package-manager=go_modules&previous-version=1.44.188&new-version=1.44.189)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

You can trigger a rebase of this PR by commenting `@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


</details>
2023-01-30 07:36:21 +00:00
Abin Simon
414c843daf
Changelog update post release (#2312)
## Description

Updating headers in CHANGELOG now that v.0.2.0 is out.
2023-01-29 06:32:03 +00:00
Niraj Tolia
d431b99970
Remove new line check for new version (#2311)
## Description

Should fix the release process

## Does this PR need a docs update or release note?

- [x]  No 

## Type of change

- [x] 🐛 Bugfix
2023-01-29 06:02:24 +00:00
Snyk bot
37cd9b8ca2
[Snyk] Security upgrade amazonlinux from latest to 2 (#2310)
Keeping your Docker base image up-to-date means you’ll benefit from security fixes in the latest version of your chosen image.

#### Changes included in this PR 


- src/testfiles/otel_daemon/Dockerfile

We recommend upgrading to `amazonlinux:2`, as this image has only 0 known vulnerabilities. To do this, merge this pull request, then verify your application still works as expected.



Some of the most important vulnerabilities in your base image include:

| Severity                                                                                                           | Issue                                                                     | Exploit Maturity      |
| :------:                                                                                                           | :----                                                                     | :---------------      |
| ![high severity](https://res.cloudinary.com/snyk/image/upload/w_20,h_20/v1561977819/icon/h.png "high severity")   | Integer Overflow or Wraparound <br/>[SNYK-AMZN2-KRB5LIBS-3244096](https://snyk.io/vuln/SNYK-AMZN2-KRB5LIBS-3244096)   | No Known Exploit   |
| ![medium severity](https://res.cloudinary.com/snyk/image/upload/w_20,h_20/v1561977819/icon/m.png "medium severity")   | Out-of-bounds Write <br/>[SNYK-AMZN2-VIMDATA-3244105](https://snyk.io/vuln/SNYK-AMZN2-VIMDATA-3244105)   | No Known Exploit   |
| ![medium severity](https://res.cloudinary.com/snyk/image/upload/w_20,h_20/v1561977819/icon/m.png "medium severity")   | Out-of-bounds Write <br/>[SNYK-AMZN2-VIMMINIMAL-3244088](https://snyk.io/vuln/SNYK-AMZN2-VIMMINIMAL-3244088)   | No Known Exploit   |



---

**Note:** _You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs._

For more information: <img src="https://api.segment.io/v1/pixel/track?data=eyJ3cml0ZUtleSI6InJyWmxZcEdHY2RyTHZsb0lYd0dUcVg4WkFRTnNCOUEwIiwiYW5vbnltb3VzSWQiOiIyYjU3YTIwZC00N2VhLTQ2MTItODhmMC0zMjQ4ZWRjZWIzM2UiLCJldmVudCI6IlBSIHZpZXdlZCIsInByb3BlcnRpZXMiOnsicHJJZCI6IjJiNTdhMjBkLTQ3ZWEtNDYxMi04OGYwLTMyNDhlZGNlYjMzZSJ9fQ==" width="0" height="0"/>
🧐 [View latest project report](https://app.snyk.io/org/asimonalcion.ai/project/70e8c43e-1af4-4f6c-97ce-f2acbe45f2f4?utm_source&#x3D;github&amp;utm_medium&#x3D;referral&amp;page&#x3D;fix-pr)

🛠 [Adjust project settings](https://app.snyk.io/org/asimonalcion.ai/project/70e8c43e-1af4-4f6c-97ce-f2acbe45f2f4?utm_source&#x3D;github&amp;utm_medium&#x3D;referral&amp;page&#x3D;fix-pr/settings)

[//]: # 'snyk:metadata:{"prId":"2b57a20d-47ea-4612-88f0-3248edceb33e","prPublicId":"2b57a20d-47ea-4612-88f0-3248edceb33e","dependencies":[{"name":"amazonlinux","from":"latest","to":"2"}],"packageManager":"dockerfile","projectPublicId":"70e8c43e-1af4-4f6c-97ce-f2acbe45f2f4","projectUrl":"https://app.snyk.io/org/asimonalcion.ai/project/70e8c43e-1af4-4f6c-97ce-f2acbe45f2f4?utm_source=github&utm_medium=referral&page=fix-pr","type":"auto","patch":[],"vulns":["SNYK-AMZN2-KRB5LIBS-3244096","SNYK-AMZN2-VIMMINIMAL-3244088","SNYK-AMZN2-VIMDATA-3244105"],"upgrade":["SNYK-AMZN2-KRB5LIBS-3244096","SNYK-AMZN2-VIMDATA-3244105","SNYK-AMZN2-VIMMINIMAL-3244088"],"isBreakingChange":false,"env":"prod","prType":"fix","templateVariants":["updated-fix-title"],"priorityScoreList":[null,null,null]}'

---

**Learn how to fix vulnerabilities with free interactive lessons:**

 🦉 [Learn about vulnerability in an interactive lesson of Snyk Learn.](https://learn.snyk.io/?loc&#x3D;fix-pr)
v0.2.0
2023-01-28 03:24:12 +00:00
Keepers
2d7c46c6b3
add getItem unit with retries (#2305)
## Description

separates item get/retry when iterating over
items in exchange collection streams, and adds
a mocked unit test on that func.

## Does this PR need a docs update or release note?

- [x]  No 

## Type of change

- [x] 🤖 Test

## Issue(s)

* closes #2299

## Test Plan

- [x]  Unit test
2023-01-28 02:06:15 +00:00
ashmrtn
704a0f8878
Create helper functions/interfaces to retrieve next and delta links (#2308)
## Description

Create a shared set of helper functions and interfaces for getting next and delta links from Graph API responses. This helps standardize how they are handled with respect to nil/empty values and makes comparisons easier in other code

## Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No 

## Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

## Issue(s)

* #2264 

## Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2023-01-27 23:13:16 +00:00
ashmrtn
57accfc9c4
Use mocks to test getting drive info from Graph API (#2306)
## Description

Make a separate interface for fetching drive information from Graph API. Interface allows for better testing via mocks

Merges SharePoint and OneDrive code for getting drives.

Also fixes potential bug where not all drives would be fetched. This could have occurred because the previous implementation for both SharePoint and OneDrive were not checking for paginated results

Viewing by commit is recommended

## Does this PR need a docs update or release note?

- [x]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No 

## Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Test
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

## Issue(s)

* #2264

## Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-01-27 22:48:30 +00:00
Niraj Tolia
3703d3d9c7
Upgrade to Docusaurus 2.3.0 (#2303)
## Description

Upgrade to Docusaurus 2.3.0

## Does this PR need a docs update or release note?

- [x]  No 

## Type of change

- [x] 🗺️ Documentation

## Test Plan

- [x] 💪 Manual
2023-01-27 22:30:06 +00:00
Keepers
0d8eb8f4fa
refactor exchange attachment downloads (#2302)
## Description

Three changes: adds a LargeItem servicer to
the exchange api client.  Migrates attachment
downloads in mail and events out of serialize
and into the GetItem func.  Finally, utilizes the
largeItem servicer to download attachments,
instead of the standard servicer.

A follow-up PR will add mocked test cases for
these changes.

## Does this PR need a docs update or release note?

- [x]  Yes, it's included

## Type of change

- [x] 🌻 Feature

## Issue(s)

* #2299

## Test Plan

- [x] 💚 E2E
2023-01-27 22:25:39 +00:00
Georgi Matev
3e106dbac8
Warning in the help messages for pre-release hidden commands (#2307)
## Description

A basic mechanism to show a warning for the help output of pre-release commands

## Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No 

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

## Issue(s)

## Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-01-27 20:47:15 +00:00
Vaibhav Kamra
09821f2f0d
Hide sharepoint details command (#2304)
## Description

Sharepoint support is still under active development and not ready for use. Hiding the command from
usage details.

## Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No 

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [x] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

## Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-01-27 20:03:13 +00:00
Keepers
24911a358b
quick graph client refactor (#2301)
## Description

A couple minor refactors.  First, combines the
graph/service_helper.go and graph/service.go
files into a single file (just lift'n'shift, no logic
changes).  Second, renames CreateHTTPClient
to just HTTPClient.  Third, replaces the new
LargeItemClient constructor with extensible
options for the http client, starting with a
NoTimeout option.  This will provide longer
term maintenance over multiple constructors.

This change sets up QoL for building multiple services
and clients within api layer handlers, which will appear
in an immediately following PR.

## Does this PR need a docs update or release note?

- [x]  No 

## Type of change

- [x] 🧹 Tech Debt/Cleanup

## Issue(s)

* #2299

## Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-01-27 19:34:00 +00:00
Nočnica Mellifera
2aaaa7a1cb
update from Google Analytics to Google Tag Manager (#2288)
## Description

reconfigures to use google tag manager rather than direct to Google Analytics. 

## Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No 

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-01-27 18:51:26 +00:00
ashmrtn
f69dfc14e2
Wire kopia exclude list up to BackupOp (#2244)
## Description

Expose the global exclude list mechanism in kopia BackupCollections to other components. Add tests for BackupCollections testing new exclude list functionality

Currently wired to always be passed nil

## Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No 

## Type of change

- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

## Issue(s)

* #2243

merge after:
* #2143 

## Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-01-27 17:09:40 +00:00
Abin Simon
137cf5038c
Reword docs about latest binary artifact downloads (#2289)
## Description

Just some docs cleanup.

## Does this PR need a docs update or release note?

- [x]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No 

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [x] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* fixes https://github.com/alcionai/corso/issues/2278

## Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-01-27 07:31:04 +00:00
dependabot[bot]
5783464846
⬆️ Bump github.com/aws/aws-sdk-go from 1.44.187 to 1.44.188 in /src (#2293)
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.44.187 to 1.44.188.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/aws/aws-sdk-go/releases">github.com/aws/aws-sdk-go's releases</a>.</em></p>
<blockquote>
<h1>Release v1.44.188 (2023-01-26)</h1>
<h3>Service Client Updates</h3>
<ul>
<li><code>service/eventbridge</code>: Updates service documentation</li>
<li><code>service/iotfleetwise</code>: Updates service API</li>
<li><code>service/s3</code>: Updates service examples
<ul>
<li>Allow FIPS to be used with path-style URLs.</li>
</ul>
</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="61bd8adb75"><code>61bd8ad</code></a> Release v1.44.188 (2023-01-26) (<a href="https://github-redirect.dependabot.com/aws/aws-sdk-go/issues/4705">#4705</a>)</li>
<li>See full diff in <a href="https://github.com/aws/aws-sdk-go/compare/v1.44.187...v1.44.188">compare view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=github.com/aws/aws-sdk-go&package-manager=go_modules&previous-version=1.44.187&new-version=1.44.188)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

You can trigger a rebase of this PR by commenting `@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


</details>
2023-01-27 06:06:31 +00:00
Keepers
ce2c1ffbb3
clean up graph errors (#2286)
## Description

The current interface makes no sense.  This
change refactors the funcs to standard boolean
comparators.

## Does this PR need a docs update or release note?

- [x]  No 

## Type of change

- [x] 🧹 Tech Debt/Cleanup
2023-01-27 04:24:06 +00:00
ashmrtn
06efad7028
Bulk up kopia wrapper error checking in BackupOp (#2281)
## Description

Expand error checking for kopia wrapper backup. Kopia may report a number of errors for a snapshot operation but may not give error structs for them. Handle this by checking if there were > 0 errors reported and making our own error struct for this if there are.

This will keep the system from reporting a successful backup when kopia had problems reading or uploading data. Errored files in kopia do not become part of the snapshot

## Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No 

## Type of change

- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

## Issue(s)

* closes #2280 

## Test Plan

- [ ] 💪 Manual
- [ x  Unit test
- [x] 💚 E2E
2023-01-27 01:38:58 +00:00
Keepers
3cd82de23a
re-fetch file download url after expiration (#2283)
## Description

If a drive item goes over its 1 hour jwt expiration to download the backing file, re-fetch the item
and use the new download url to get the file.

## Does this PR need a docs update or release note?

- [x]  No 

## Type of change

- [x] 🌻 Feature

## Issue(s)

* #2267

## Test Plan

- [x] 💪 Manual
2023-01-27 01:20:41 +00:00
Keepers
d529d145cb
scrub pii from observe logs (#2285)
## Description

This is a quick hack to satisfy a primary case of PII scrubbing.  We expect to revisit it in the future.

## Does this PR need a docs update or release note?

- [x]  No 

## Type of change

- [x] 🌻 Feature

## Issue(s)

* #2284

## Test Plan

- [x] 💪 Manual
- [x]  Unit test
2023-01-27 00:31:57 +00:00