1915 Commits

Author SHA1 Message Date
ashmrtn
5f43287feb
Add periodic log message for loading items during restore (#3282)
#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

*

#### Test Plan

- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-05-02 21:53:37 +00:00
Keepers
c152ce8b25
move and rename common/time.go (#3215)
moves common/time.go to common/dttm/dttm.go, and renames many of the consts and funcs with names that better explain their design and intent of usage.

Only code movement, no logical changes.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-05-02 19:39:29 +00:00
ashmrtn
8cca7f12df
Create basic maintenance operation (#3225)
Add a maintenance operation to run
kopia maintenance

Using this instead of calling kopia
directly will allow us to hook into
metrics reporting in a more
consistent manner if we want metrics

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #3077

#### Test Plan

- [ ] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-05-02 18:18:40 +00:00
Keepers
e72fa49018
retry on gateway timeout (#3271)
We don't currently retry on graph API gateway timeout/bad gateway responses.  This change adds those statuses to our retryable codes.

Also adds some qol clues tracking and error formatting.

---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included

#### Type of change

- [x] 🐛 Bugfix

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-05-02 17:47:45 +00:00
Keepers
ea5be65e08
add sharepoint to sanity tests (#3204)
Add sharepoint sanity testing.

---

#### Does this PR need a docs update or release note?

- [x] 🕐 Yes, but in a later PR

#### Type of change

- [x] 🌻 Feature

#### Issue(s)

* #3135

#### Test Plan

- [x] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2023-05-02 17:11:46 +00:00
dependabot[bot]
4ff94cab3f
⬆️ Bump github.com/aws/aws-sdk-go from 1.44.252 to 1.44.254 in /src (#3276)
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.44.252 to 1.44.254.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/aws/aws-sdk-go/releases">github.com/aws/aws-sdk-go's releases</a>.</em></p>
<blockquote>
<h1>Release v1.44.254 (2023-05-01)</h1>
<h3>Service Client Updates</h3>
<ul>
<li><code>service/compute-optimizer</code>: Updates service API and documentation</li>
<li><code>service/kms</code>: Updates service API, documentation, and examples
<ul>
<li>This release makes the NitroEnclave request parameter Recipient and the response field for CiphertextForRecipient available in AWS SDKs. It also adds the regex pattern for CloudHsmClusterId validation.</li>
</ul>
</li>
</ul>
<h1>Release v1.44.253 (2023-04-28)</h1>
<h3>Service Client Updates</h3>
<ul>
<li><code>service/appflow</code>: Updates service API and documentation</li>
<li><code>service/athena</code>: Updates service API, documentation, and paginators
<ul>
<li>You can now use capacity reservations on Amazon Athena to run SQL queries on fully-managed compute capacity.</li>
</ul>
</li>
<li><code>service/directconnect</code>: Updates service documentation
<ul>
<li>This release corrects the jumbo frames MTU from 9100 to 8500.</li>
</ul>
</li>
<li><code>service/elasticfilesystem</code>: Updates service API
<ul>
<li>This release adds PAUSED and PAUSING state as a returned value for DescribeReplicationConfigurations response.</li>
</ul>
</li>
<li><code>service/grafana</code>: Updates service API and documentation</li>
<li><code>service/iot</code>: Updates service API and documentation
<ul>
<li>This release allows AWS IoT Core users to specify a TLS security policy when creating and updating AWS IoT Domain Configurations.</li>
</ul>
</li>
<li><code>service/rekognition</code>: Updates service API and documentation
<ul>
<li>Added support for aggregating moderation labels by video segment timestamps for Stored Video Content Moderation APIs and added additional information about the job to all Stored Video Get API responses.</li>
</ul>
</li>
<li><code>service/simspaceweaver</code>: Updates service API and documentation</li>
<li><code>service/wafv2</code>: Updates service API and documentation</li>
<li><code>service/workspaces</code>: Updates service documentation
<ul>
<li>Added Windows 11 to support Microsoft_Office_2019</li>
</ul>
</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="47ebc6fcb7"><code>47ebc6f</code></a> Release v1.44.254 (2023-05-01) (<a href="https://redirect.github.com/aws/aws-sdk-go/issues/4821">#4821</a>)</li>
<li><a href="af48150b5a"><code>af48150</code></a> Release v1.44.253 (2023-04-28) (<a href="https://redirect.github.com/aws/aws-sdk-go/issues/4819">#4819</a>)</li>
<li>See full diff in <a href="https://github.com/aws/aws-sdk-go/compare/v1.44.252...v1.44.254">compare view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=github.com/aws/aws-sdk-go&package-manager=go_modules&previous-version=1.44.252&new-version=1.44.254)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

You can trigger a rebase of this PR by commenting `@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


</details>
2023-05-02 11:07:39 +00:00
neha_gupta
e4a587dc42
Update CHANGELOG.md for new release v0.7.0 (#3279)
<!-- PR description-->

Updating changelog for new release v0.7.0

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-05-02 10:21:49 +00:00
ashmrtn
b6ef2d5266
Create wrapper around kopia maintenance (#3224)
Create a wrapper function around the kopia
maintenance operation. Will allow users to
run maintenance, though they still need to
be careful not to run maintenance
concurrently if they override the username
and hostname that kopia uses

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #3077

#### Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
v0.7.0
2023-05-02 00:52:15 +00:00
Vaibhav Kamra
631e1a3b61
Remove Unit-Test dependency in release job (#3274)
Release job was skipping because Unit-Test is not being run

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [x] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
2023-05-01 21:03:01 +00:00
Keepers
3b9d2841d4
rename DetailsEntry to Entry to avoid stuttering (#3265)
#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-05-01 19:29:06 +00:00
ashmrtn
c3cc40176e
Fallback to RepoRef is LocationRef is empty (#3270)
Fixes issue where doing backup detail or a restore
on an older backup and selecting by folder in
exchange wouldn't return any results. Root cause
is that older backup versions didn't populate
LocationRef in backup details which is what we
compare against

Manually tested on an older Corso backup. Will
write automated unit tests in a bit, wanted to
get this in prior to release

---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

* closes #3269

#### Test Plan

- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-05-01 17:13:49 +00:00
dependabot[bot]
213bac60e2
⬆️ Bump tailwindcss from 3.3.1 to 3.3.2 in /website (#3266) 2023-05-01 06:17:27 +00:00
neha_gupta
dbb950a37b
create onedrive custom drive (#3227)
<!-- PR description-->

Run Sanity test with custom onedrive data

#### Does this PR need a docs update or release note?
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🐛 Bugfix
- [x] 🤖 Supportability/Tests
#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
2023-04-30 09:37:53 +00:00
ashmrtn
dcb9d81f3f
Fix progress folder display for Exchange (#3219)
---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

* closes #3013

#### Test Plan

- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-04-29 00:19:54 +00:00
ashmrtn
4c653f5a41
Only run on schedule, not merges (#3260)
#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [x] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
2023-04-28 23:52:11 +00:00
ashmrtn
619bcd6eb8
Check for new and older content URLs (#3263)
Some requests appear to be returning an older version of the content URL key. This was causing corso backup failures. This commit expands the logic to check for both the new and old keys for the URL.

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

* closes #3262 

#### Test Plan

- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-04-28 23:06:10 +00:00
ashmrtn
6e3403f717
Always attempt to upload test log (#3259)
#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [x] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
2023-04-28 19:11:44 +00:00
dependabot[bot]
06f4ed48d0
⬆️ Bump github.com/aws/aws-sdk-go from 1.44.249 to 1.44.252 in /src (#3255)
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.44.249 to 1.44.252.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/aws/aws-sdk-go/releases">github.com/aws/aws-sdk-go's releases</a>.</em></p>
<blockquote>
<h1>Release v1.44.252 (2023-04-27)</h1>
<h3>Service Client Updates</h3>
<ul>
<li><code>service/ec2</code>: Updates service API and documentation
<ul>
<li>This release adds support for AMD SEV-SNP on EC2 instances.</li>
</ul>
</li>
<li><code>service/emr-containers</code>: Updates service API and documentation</li>
<li><code>service/guardduty</code>: Updates service API and documentation
<ul>
<li>Added API support to initiate on-demand malware scan on specific resources.</li>
</ul>
</li>
<li><code>service/iotdeviceadvisor</code>: Updates service API and documentation</li>
<li><code>service/kafka</code>: Updates service API, documentation, and paginators</li>
<li><code>service/lambda</code>: Updates service API
<ul>
<li>Add Java 17 (java17) support to AWS Lambda</li>
</ul>
</li>
<li><code>service/marketplace-catalog</code>: Updates service paginators</li>
<li><code>service/osis</code>: Updates service documentation</li>
<li><code>service/qldb</code>: Updates service documentation</li>
<li><code>service/sagemaker</code>: Updates service API and documentation
<ul>
<li>Added ml.p4d.24xlarge and ml.p4de.24xlarge as supported instances for SageMaker Studio</li>
</ul>
</li>
<li><code>service/xray</code>: Adds new service
<ul>
<li>Updated X-Ray documentation with Resource Policy API descriptions.</li>
</ul>
</li>
</ul>
<h1>Release v1.44.251 (2023-04-26)</h1>
<h3>Service Client Updates</h3>
<ul>
<li><code>service/osis</code>: Updates service API, documentation, paginators, and examples</li>
</ul>
<h1>Release v1.44.250 (2023-04-25)</h1>
<h3>Service Client Updates</h3>
<ul>
<li><code>service/chime-sdk-messaging</code>: Updates service API and documentation</li>
<li><code>service/connect</code>: Updates service API, documentation, and paginators</li>
<li><code>service/datasync</code>: Updates service API, documentation, and paginators</li>
<li><code>service/ds</code>: Updates service API and documentation
<ul>
<li>New field added in AWS Managed Microsoft AD DescribeSettings response and regex pattern update for UpdateSettings value.  Added length validation to RemoteDomainName.</li>
</ul>
</li>
<li><code>service/pinpoint</code>: Updates service API, documentation, and examples
<ul>
<li>Adds support for journey runs and querying journey execution metrics based on journey runs. Adds execution metrics to campaign activities. Updates docs for Advanced Quiet Time.</li>
</ul>
</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="b5c933283d"><code>b5c9332</code></a> Release v1.44.252 (2023-04-27) (<a href="https://redirect.github.com/aws/aws-sdk-go/issues/4818">#4818</a>)</li>
<li><a href="3576b9b771"><code>3576b9b</code></a> Release v1.44.251 (2023-04-26) (<a href="https://redirect.github.com/aws/aws-sdk-go/issues/4814">#4814</a>)</li>
<li><a href="ae7f3e0d9f"><code>ae7f3e0</code></a> Release v1.44.250 (2023-04-25) (<a href="https://redirect.github.com/aws/aws-sdk-go/issues/4813">#4813</a>)</li>
<li>See full diff in <a href="https://github.com/aws/aws-sdk-go/compare/v1.44.249...v1.44.252">compare view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=github.com/aws/aws-sdk-go&package-manager=go_modules&previous-version=1.44.249&new-version=1.44.252)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

You can trigger a rebase of this PR by commenting `@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


</details>
2023-04-28 17:31:45 +00:00
ashmrtn
c698c6e73e
Type out path instead of using env (#3254)
Env inside of an env declaration isn't allowed it seems.

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [x] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
2023-04-28 16:48:56 +00:00
Vaibhav Kamra
084b519766
Introduce a UserCompatNoInfo call (#3257)
Instead of returning `Info` in `User`, callers should call the `GetUserInfo` API to get extended user information.

This allows the caller to decide whether info is needed *and* control parallelism for these queries.

This change introduces a `UserCompatNoInfo` call that allows us to make the switch without breaking existing `UserCompat` references. We will remove this API after callers have switched.

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [ ]  Unit test
- [x] 💚 E2E
2023-04-28 15:22:55 +00:00
ashmrtn
40d8c45ec7
Fixup archive upload for sanity test (#3251)
* now always upload logs
* don't use relative paths for things (archive doesn't like it)
* build variables off each other
* upload whole log file output directory
* fix upload action to use the right directory (it doesn't pay attention to working-directory)

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [x] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Test Plan

- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E

tested locally with nektos act
2023-04-28 00:29:24 +00:00
ashmrtn
f2f547f60a
Update golangci-lint to 1.52.2 (#3253)
---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [x] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Test Plan

- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-04-27 23:49:09 +00:00
ashmrtn
6ac8c9f331
Allow overriding the default username/hostname in kopia (#3223)
This allows setting the username/hostname
kopia will use for maintenance. This
information is recorded in the kopia config
file during connect and reused during open

If no values are given, kopia pulls the
values from the OS

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #3077

#### Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-04-27 22:47:35 +00:00
ashmrtn
754e14d7a6
Don't use RepoRef in selector reduction (#3236)
A few high-level things of note:
* things will no longer match on folder ID. Folder IDs weren't displayed
  to the user via CLI and SDK consumers have no insight into folder IDs
  so this shouldn't be an issue
* OneDrive and SharePoint match on ParentPath (derived from
  LocationRef). ParentPath *does not* include root: in the path

Not matching on folder ID should be the only user-visible change in
this PR

First commit contains the required logic changes. All other changes
are test updates

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

* closes #3194

#### Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-04-27 22:01:34 +00:00
Keepers
1cd592d216
minor errs cleanup (#3234)
some code cleanup that happened while investigating a bug repro

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-04-27 16:57:38 +00:00
ashmrtn
b8e868c05b
Return SharePoint exclude information (#3242)
Allows proper deletion of files and also fixes file moves.

---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #3240

#### Test Plan

- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-04-27 16:25:12 +00:00
neha_gupta
ca9425a329
Don't go for snapshot deletion in case of empty backup (#3250)
<!-- PR description-->

Fix: https://github.com/alcionai/corso/issues/3050 
In case of a blank backup, don't go for snapshot deletion. 

#### Does this PR need a docs update or release note?
- [ ]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🐛 Bugfix

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* https://github.com/alcionai/corso/issues/3050

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ]  Unit test
2023-04-27 12:42:48 +00:00
dependabot[bot]
57dfa3d38a
⬆️ Bump sass from 1.62.0 to 1.62.1 in /website (#3246) 2023-04-27 06:05:23 +00:00
Keepers
85f322fe97
don't give log-file prepopulate flag a default (#3237)
Passing the default logging file into flag pre-population causes the logger to assume the flag is always passed in and never falls back to the ENV setting.  This ensures that if no flag is provided, the env log file setting is used.

---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included

#### Type of change

- [x] 🐛 Bugfix

#### Test Plan

- [x] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2023-04-27 00:54:15 +00:00
neha_gupta
c2af75da9c
documentation update for new permission (#3231)
<!-- PR description-->

Update documentation with info about required permissions.

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🗺️ Documentation

#### Issue(s)


#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
2023-04-26 21:31:45 +00:00
Keepers
f2f010b9f6
cache created folders when restoring drive items (#3220)
drive items aren't currently getting cached after creation.  Instead, we do a lookup on the folder name every time we walk the folder hierarchy toward the next child to restore.  Normally this would work fine, but apparently if the folder is named `folder`, then graph api returns nothing, causing the process to fail when repeatedly creating the folder.

---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included

#### Type of change

- [x] 🐛 Bugfix

#### Test Plan

- [x] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2023-04-26 20:53:26 +00:00
zackrossman
c28673b2f0
Helper method to expose internal errors associated with a public error category (#3233)
<!-- PR description-->

Helper method to expose internal errors associated with a public error category, useful for testing `errs.Is` scenarios. The internal error needs to be used because `errs.Is` checks if the error chain contains a ref to the internal error.

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #COR-77

#### Test Plan

<!-- How will this be tested prior to merging.-->
-  [x] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2023-04-26 20:06:06 +00:00
Abin Simon
534924f696
Don't try to fetch backup version if no compete backups available (#3230)
Regression from 62daf10213 (diff-f088d3a5f56348ddaf6fbf485c2aeaa6d8b40a860de014f7084db6daf0753829R267)


---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-04-26 19:15:56 +00:00
Vaibhav Kamra
6395dcbe39
Add parent path information to malware detected items (#3221)
This allows identifying the file location when an item is detected with malware

---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #3112 

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-04-26 17:17:52 +00:00
neha_gupta
559ad37a7e
verify data inside correct struct (#3228)
<!-- PR description-->

- check for errors in correct sub struct
- comment out few failing test cases. Working on its fix in-  https://github.com/alcionai/corso/pull/3227/files

#### Does this PR need a docs update or release note?
- [ ]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->

- [ ] 🤖 Supportability/Tests

#### Issue(s)


#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
2023-04-26 15:10:00 +00:00
Keepers
edef23bfc4
errant cleanups before incremental support (#3201)
a handful of errant cleanups to help the PR for adding incremental support to sharepoint.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #3136

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-04-25 21:09:41 +00:00
Keepers
0ae1a04097
use nanosecond folders for testing (#3205)
we've been failing on folder name collision, particularly in this package.  Probably because it's using second-granularity folder names.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🐛 Bugfix
- [x] 🤖 Supportability/Tests

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-04-25 20:33:39 +00:00
Keepers
c95a07660a
ditch userInfo middle struct (#3206)
we don't need a m365.UserInfo when we already have an api.UserInfo

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-04-25 19:05:07 +00:00
ashmrtn
8a2e63dcad
CLI connect refactor (#3213)
Move code for connecting to a repo into a common
package so that backup and restore CLI code can
both use it

This will also make it easier for maintenance
code in the future as it can reuse the same
helper

There are no logic changes in this PR, only
code movement

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)
tangentially related to
* #3077

#### Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-04-25 18:35:50 +00:00
zackrossman
51809cf6eb
Expose ErrResourceOwnerNotFound in err package (#3214)
<!-- PR description-->

Expose `ErrResourceOwnerNotFound` in `err` package to make for neat error handling

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #COR-74

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-04-25 17:04:28 +00:00
dependabot[bot]
83e6803e3a
⬆️ Bump github.com/aws/aws-sdk-go from 1.44.248 to 1.44.249 in /src (#3210)
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.44.248 to 1.44.249.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/aws/aws-sdk-go/releases">github.com/aws/aws-sdk-go's releases</a>.</em></p>
<blockquote>
<h1>Release v1.44.249 (2023-04-24)</h1>
<h3>Service Client Updates</h3>
<ul>
<li><code>service/appflow</code>: Updates service API</li>
<li><code>service/ec2</code>: Updates service API and documentation
<ul>
<li>API changes to AWS Verified Access related to identity providers' information.</li>
</ul>
</li>
<li><code>service/mediaconvert</code>: Updates service API and documentation
<ul>
<li>This release introduces a noise reduction pre-filter, linear interpolation deinterlace mode, video pass-through, updated default job settings, and expanded LC-AAC Stereo audio bitrate ranges.</li>
</ul>
</li>
<li><code>service/rekognition</code>: Updates service API and documentation
<ul>
<li>Added new status result to Liveness session status.</li>
</ul>
</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="82ec8a68e0"><code>82ec8a6</code></a> Release v1.44.249 (2023-04-24) (<a href="https://redirect.github.com/aws/aws-sdk-go/issues/4812">#4812</a>)</li>
<li>See full diff in <a href="https://github.com/aws/aws-sdk-go/compare/v1.44.248...v1.44.249">compare view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=github.com/aws/aws-sdk-go&package-manager=go_modules&previous-version=1.44.248&new-version=1.44.249)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

You can trigger a rebase of this PR by commenting `@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


</details>
2023-04-25 08:34:58 +00:00
Abin Simon
08c625b788
Show names in OneDrive restore progressbar (#3208)
Related https://github.com/alcionai/corso/pull/3168

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* https://github.com/alcionai/corso/issues/3113

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-04-25 08:08:14 +00:00
Abhishek Pandey
c38a431079
Add --disable-concurrency-limiter cli flag (#3203)
<!-- PR description-->
The concurrency limiter middleware is enabled by default for exchange backups. Adding a hidden --disable-concurrency-limiter flag to disable this middleware.

Reasons for adding this flag:
We have done limited perf testing with the concurrency limiter. In addition, our understanding of exchange concurrency limits is also a bit fuzzy. This flag acts as a killswitch in case we start to see perf regressions in prod.

I see this as a temporary flag. Ideally we would never have to use it. Also, once we have a better way to configure concurrency limiter, we can eliminate this flag.


---

#### Does this PR need a docs update or release note?

- [ ]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature


#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->

- [ ]  Unit test
2023-04-25 01:22:12 +00:00
Keepers
7b1ce22a64
enable user pn->id migrations for reporef prefix (#3200)
Enables migrations of all user-based repoRef prefixes away from the user's PrincipalName and onto the user's ID for a more stable reference.

---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included

#### Type of change

- [x] 🌻 Feature

#### Issue(s)

* #2825

#### Test Plan

- [x] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2023-04-24 22:37:11 +00:00
Keepers
b331f38654
fixup mailbox info (#3189)
#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🐛 Bugfix
- [x] 🧹 Tech Debt/Cleanup

#### Test Plan

- [x] 💪 Manual
- [x] 💚 E2E
2023-04-24 20:06:41 +00:00
Abhishek Pandey
41f742eba2
Add concurrency limiter middleware package (#3182)
<!-- PR description-->


Graph service only allows [4 concurrent requests per exchange
mailbox.](https://learn.microsoft.com/en-us/graph/throttling-limits#outlook-service-limits)
We are currently not honoring this limit in corso. This causes 429
errors with “Application is over its MailboxConcurrency limit”. This PR
introduces a concurrency limiter middleware. This middleware is
selectively added for exchange backups only.


---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic
words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E

---------

Co-authored-by: aviator-app[bot] <48659329+aviator-app[bot]@users.noreply.github.com>
2023-04-24 11:16:20 -07:00
Keepers
62daf10213
migrate onedrive using prefix collection (#3122)
#### Does this PR need a docs update or release note?

- [x] 🕐 Yes, but in a later PR

#### Type of change

- [x] 🌻 Feature

#### Issue(s)

* #2825

#### Test Plan

- [x] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2023-04-24 17:36:10 +00:00
dependabot[bot]
f5a4c3c0ba
⬆️ Bump github.com/aws/aws-sdk-go from 1.44.247 to 1.44.248 in /src (#3199)
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.44.247 to 1.44.248.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/aws/aws-sdk-go/releases">github.com/aws/aws-sdk-go's releases</a>.</em></p>
<blockquote>
<h1>Release v1.44.248 (2023-04-21)</h1>
<h3>Service Client Updates</h3>
<ul>
<li><code>service/connect</code>: Updates service API and documentation</li>
<li><code>service/ecs</code>: Updates service documentation
<ul>
<li>Documentation update to address various Amazon ECS tickets.</li>
</ul>
</li>
<li><code>service/fms</code>: Updates service API, documentation, and paginators</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="f9284e3449"><code>f9284e3</code></a> Release v1.44.248 (2023-04-21) (<a href="https://redirect.github.com/aws/aws-sdk-go/issues/4810">#4810</a>)</li>
<li><a href="42260bb91c"><code>42260bb</code></a> Fix EC2 Presigned URL customization (<a href="https://redirect.github.com/aws/aws-sdk-go/issues/4808">#4808</a>)</li>
<li>See full diff in <a href="https://github.com/aws/aws-sdk-go/compare/v1.44.247...v1.44.248">compare view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=github.com/aws/aws-sdk-go&package-manager=go_modules&previous-version=1.44.247&new-version=1.44.248)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

You can trigger a rebase of this PR by commenting `@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


</details>
2023-04-24 07:18:27 +00:00
Georgi Matev
09f52bb5b2
Move the CI user and site names to vars for better maintainability (#3191)
<!-- PR description-->

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [x] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-04-21 20:54:23 +00:00
Keepers
6405c8246e
move parallelism logs/checks upstream (#3003)
The fetch paralellism checks and logs occur on
every item streamed from GC.  This is a bit chatty,
and has been moved upstream in the process for
a more centralized behavior.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Test Plan

- [x] 💪 Manual
- [x]  Unit test
2023-04-21 20:23:41 +00:00