2102 Commits

Author SHA1 Message Date
dependabot[bot]
cd0574781a
⬆️ Bump golang.org/x/time from 0.1.0 to 0.3.0 in /src (#2936)
Bumps [golang.org/x/time](https://github.com/golang/time) from 0.1.0 to 0.3.0.
<details>
<summary>Commits</summary>
<ul>
<li><a href="2c09566ef1"><code>2c09566</code></a> rate: the state of the limiter should not be changed when the requests failed</li>
<li><a href="80b9fac54d"><code>80b9fac</code></a> x/time/rate: add rate.Sometimes, which runs a function occasionally.</li>
<li>See full diff in <a href="https://github.com/golang/time/compare/v0.1.0...v0.3.0">compare view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=golang.org/x/time&package-manager=go_modules&previous-version=0.1.0&new-version=0.3.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

You can trigger a rebase of this PR by commenting `@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


</details>> **Note**
> Automatic rebases have been disabled on this pull request as it has been open for over 30 days.
2023-05-16 23:47:22 +00:00
ashmrtn
089a96d437
Create CLI command for maintenance (#3226)
Creates hidden CLI command to run
maintenance.

Flag names and descriptions can be
updated if something else would fit
better

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #3077

#### Test Plan

- [x] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-05-16 21:03:06 +00:00
Abhishek Pandey
965427d491
Place concurrency limiter middleware after retry middlewares (#3420)
This is to safeguard against the scenario where retry handlers might spawn additional requests to graph. Such retries may cause us to exceed concurrent request limits, leading to 429s ( `Application is over its MailboxConcurrency limit`) 

This PR will ensure that such retry requests are also funneled through the concurrency limiter.

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-05-16 20:37:52 +00:00
neha_gupta
2ad32660e8
use graph client for large data set (#3355)
<!-- PR description-->
- Use graph client instead of Resty client. This brings uniformity and will help in utilising other features like Middlewares from graph client wrapper
- Moved the upload session to graph package to use Graph http-client wrapper

#### Does this PR need a docs update or release note?
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* https://github.com/alcionai/corso/issues/2300

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
2023-05-16 19:22:13 +00:00
ashmrtn
f8252b4cd3
Only remove previous deltas from metadata under some conditions (#3419)
Only allow removing previous delta tokens and not
previous paths. This ensures folder deletions will
be picked up if all items are enumerated

Updates tests to match new behavior, mostly
adjusting expectations around del list when
doNotMerge is applied

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

* closes #2334

#### Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-05-16 18:43:31 +00:00
Snyk bot
a26676c942
[Snyk] Security upgrade alpine from 3.17 to 3 (#3402)
Keeping your Docker base image up-to-date means you’ll benefit from security fixes in the latest version of your chosen image.

#### Changes included in this PR 


- build/Dockerfile

We recommend upgrading to `alpine:3`, as this image has only 0 known vulnerabilities. To do this, merge this pull request, then verify your application still works as expected.



Some of the most important vulnerabilities in your base image include:

| Severity                                                                                                                 | Priority Score / 1000  | Issue                                                                     | Exploit Maturity      |
| :------:                                                                                                                 | :--------------------  | :----                                                                     | :---------------      |
| ![medium severity](https://res.cloudinary.com/snyk/image/upload/w_20,h_20/v1561977819/icon/m.png "medium severity")   | **300**  | Out-of-bounds Read <br/>[SNYK-ALPINE317-OPENSSL-5438697](https://snyk.io/vuln/SNYK-ALPINE317-OPENSSL-5438697)   | No Known Exploit   |
| ![medium severity](https://res.cloudinary.com/snyk/image/upload/w_20,h_20/v1561977819/icon/m.png "medium severity")   | **300**  | Out-of-bounds Read <br/>[SNYK-ALPINE317-OPENSSL-5438697](https://snyk.io/vuln/SNYK-ALPINE317-OPENSSL-5438697)   | No Known Exploit   |



---

**Note:** _You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs._

For more information: <img src="https://api.segment.io/v1/pixel/track?data=eyJ3cml0ZUtleSI6InJyWmxZcEdHY2RyTHZsb0lYd0dUcVg4WkFRTnNCOUEwIiwiYW5vbnltb3VzSWQiOiJhMWFlMDZiMi0xZjZhLTQwM2EtOTgzMC01NmU4ZTRhYmE4MDkiLCJldmVudCI6IlBSIHZpZXdlZCIsInByb3BlcnRpZXMiOnsicHJJZCI6ImExYWUwNmIyLTFmNmEtNDAzYS05ODMwLTU2ZThlNGFiYTgwOSJ9fQ==" width="0" height="0"/>
🧐 [View latest project report](https://app.snyk.io/org/alcion/project/4c1165db-1d77-4278-b861-48e29b49c4e7?utm_source&#x3D;github-enterprise&amp;utm_medium&#x3D;referral&amp;page&#x3D;fix-pr)

🛠 [Adjust project settings](https://app.snyk.io/org/alcion/project/4c1165db-1d77-4278-b861-48e29b49c4e7?utm_source&#x3D;github-enterprise&amp;utm_medium&#x3D;referral&amp;page&#x3D;fix-pr/settings)

[//]: # 'snyk:metadata:{"prId":"a1ae06b2-1f6a-403a-9830-56e8e4aba809","prPublicId":"a1ae06b2-1f6a-403a-9830-56e8e4aba809","dependencies":[{"name":"alpine","from":"3.17","to":"3"}],"packageManager":"dockerfile","projectPublicId":"4c1165db-1d77-4278-b861-48e29b49c4e7","projectUrl":"https://app.snyk.io/org/alcion/project/4c1165db-1d77-4278-b861-48e29b49c4e7?utm_source=github-enterprise&utm_medium=referral&page=fix-pr","type":"auto","patch":[],"vulns":["SNYK-ALPINE317-OPENSSL-5438697"],"upgrade":["SNYK-ALPINE317-OPENSSL-5438697","SNYK-ALPINE317-OPENSSL-5438697"],"isBreakingChange":false,"env":"prod","prType":"fix","templateVariants":["updated-fix-title","priorityScore"],"priorityScoreList":[300],"remediationStrategy":"vuln"}'

---

**Learn how to fix vulnerabilities with free interactive lessons:**

 🦉 [Learn about vulnerability in an interactive lesson of Snyk Learn.](https://learn.snyk.io/?loc&#x3D;fix-pr)
2023-05-16 18:16:15 +00:00
Keepers
666813865e
case sensitive selection for item IDs (#3397)
Use a strictEquals matcher for item selection when
not matching on item names (no name match assumes
canonical system id match, which is case sensitive).

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🐛 Bugfix

#### Issue(s)

* #3313

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-05-16 18:15:18 +00:00
Abin Simon
67e055360d
Bump msgraph deps to 1.0.0 (#3345)
Bumps ms graph and all dependent libs to 1.0.0

*Still can't drop betasdk as `models.SitePageable` is still not available.*

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* Closes https://github.com/alcionai/corso/pull/3327
* Closes https://github.com/alcionai/corso/pull/3326
* Closes https://github.com/alcionai/corso/pull/3324
* Closes https://github.com/alcionai/corso/pull/3323
* Closes https://github.com/alcionai/corso/pull/3322
* Closes https://github.com/alcionai/corso/pull/3321
* Closes https://github.com/alcionai/corso/pull/3320
* Closes https://github.com/alcionai/corso/issues/3354

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2023-05-16 17:44:16 +00:00
Keepers
6febc3ce5b
add namespace to fault items (#3415)
prevent fault items and skips from clobbering eachother by adding a namespace to the fault item that defines a deduplication boundary.  This allows multiple items in a single service operation to have identical ids so long as their namespace differs.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🐛 Bugfix
- [x] 🤖 Supportability/Tests

#### Issue(s)

* #3283

#### Test Plan

- [x] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2023-05-16 17:18:51 +00:00
Keepers
5ca595b7b0
filters cleanup prep for case handling (#3395)
Some tidying up in filters before adding a case specific comparator.  Goals include:
* better test handling/coverage
* swap iota for string in comparator type
* more organized param normalization
* reduction in code duplication

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #3313

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-05-16 16:41:39 +00:00
neha_gupta
694abfab68
refactor onedrive test data creation (#3343)
<!-- PR description-->

Currently we are duplicating the Onedrive test data creation code at two places-
- in cli package 
- in test files.
Moved the common code to connector package so that both cli and test files can use the same common code.

#### Does this PR need a docs update or release note?

- [ ]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->

- [ ] 🤖 Supportability/Tests


#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* https://github.com/alcionai/corso/issues/3183

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
2023-05-16 09:35:26 +00:00
dependabot[bot]
0bd524f86d
⬆️ Bump github.com/aws/aws-sdk-go from 1.44.262 to 1.44.263 in /src (#3424)
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.44.262 to 1.44.263.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/aws/aws-sdk-go/releases">github.com/aws/aws-sdk-go's releases</a>.</em></p>
<blockquote>
<h1>Release v1.44.263 (2023-05-15)</h1>
<h3>Service Client Updates</h3>
<ul>
<li><code>service/athena</code>: Updates service API and documentation
<ul>
<li>You can now define custom spark properties at start of the session for use cases like cluster encryption, table formats, and general Spark tuning.</li>
</ul>
</li>
<li><code>service/kafka</code>: Updates service API</li>
<li><code>service/rekognition</code>: Updates service API and documentation
<ul>
<li>This release adds a new EyeDirection attribute in Amazon Rekognition DetectFaces and IndexFaces APIs which predicts the yaw and pitch angles of a person's eye gaze direction for each face detected in the image.</li>
</ul>
</li>
<li><code>service/rolesanywhere</code>: Updates service API and documentation</li>
<li><code>service/transfer</code>: Updates service API and documentation
<ul>
<li>This release introduces the ability to require both password and SSH key when users authenticate to your Transfer Family servers that use the SFTP protocol.</li>
</ul>
</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="d47c75a58a"><code>d47c75a</code></a> Release v1.44.263 (2023-05-15) (<a href="https://redirect.github.com/aws/aws-sdk-go/issues/4840">#4840</a>)</li>
<li>See full diff in <a href="https://github.com/aws/aws-sdk-go/compare/v1.44.262...v1.44.263">compare view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=github.com/aws/aws-sdk-go&package-manager=go_modules&previous-version=1.44.262&new-version=1.44.263)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

You can trigger a rebase of this PR by commenting `@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


</details>
2023-05-16 07:33:50 +00:00
dependabot[bot]
cffdb84e9c
⬆️ Bump @docusaurus/module-type-aliases from 2.4.0 to 2.4.1 in /website (#3421) 2023-05-16 06:17:52 +00:00
dependabot[bot]
6dbaa6a594
⬆️ Bump @docusaurus/preset-classic from 2.4.0 to 2.4.1 in /website (#3423) 2023-05-16 06:12:47 +00:00
dependabot[bot]
a084ce6d30
⬆️ Bump @docusaurus/core from 2.4.0 to 2.4.1 in /website (#3422) 2023-05-16 06:07:08 +00:00
Abin Simon
5a2cd41d57
Convert PageLinker to DeltaPageLinker for NonDelta pagers (#3411)
This ensures that the interface for both Delta and NonDelta requests are the same and we don't have to cast the type before getting delta url from the returned object.

Follow up to https://github.com/alcionai/corso/pull/3212#discussion_r1191511216

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-05-16 05:26:13 +00:00
Abin Simon
a55758020f
Ensure response body gets closed (#3409)
From https://pkg.go.dev/net/http#Client.Do

> If the returned error is nil, the Response will contain a non-nil
> Body which the user is expected to close. If the Body is not both read
> to EOF and closed, the Client's underlying RoundTripper (typically
> Transport) may not be able to re-use a persistent TCP connection to
> the server for a subsequent "keep-alive" request.

These were found using https://github.com/timakin/bodyclose. Not adding it as a linter for now as it was producing false positive(I think) on OneDrive item. The close was happening in a9918d2f78/src/internal/kopia/upload.go (L68) instead of right alongside it which might be why it got thrown off.

<!-- PR description-->

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-05-16 04:46:56 +00:00
Abin Simon
1961104207
Don't repeat initialization steps for sanity-test composite action (#3413)
Initialization does not have to be done for every step as they do get carried over.

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [x] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-05-16 03:49:52 +00:00
Keepers
2c7b2633a9
update release changelog (#3418)
#### Does this PR need a docs update or release note?

- [x]  Yes, it's included

#### Type of change

- [x] 🗺️ Documentation
2023-05-15 23:42:16 +00:00
ashmrtn
04a30ac662
Don't display folders in restore output (#3414)
#### Does this PR need a docs update or release note?

- [x]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

* closes #3281

#### Test Plan

- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-05-15 23:28:04 +00:00
Nočnica Mellifera
ecb1fdc40a
Incrementals blog(s), part 1 and 2 (#3053)
<!-- PR description-->
These will both go out on the 10th, they have 2 different dates to make
sure the blog orders them correctly
---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [x] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic
words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E

---------

Co-authored-by: Niraj Tolia <ntolia@users.noreply.github.com>
2023-05-15 16:05:17 -07:00
Keepers
3c26b09d71
separate log level and format config (#3398)
Separates the log level and log format.  This will allows sdk consumers using `SeedLogger` to utilize JSON
format logs without sacrificing log level controls.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🤖 Supportability/Tests

#### Test Plan

- [x] 💪 Manual
2023-05-15 22:03:37 +00:00
Keepers
97ed97c1c3
logic change for sharepoint permission support (#3333)
adds logical changes required for sharepoint permission handling, both backup and restore.

---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included

#### Type of change

- [x] 🌻 Feature

#### Issue(s)

* #3135

#### Test Plan

- [x] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
v0.8.0
2023-05-15 21:03:00 +00:00
Abin Simon
451041c147
Cleanup and simplify sanity tests (#3412)
This moves the major chunk of sanity testing code (backup -> restore
-> verify -> list -> list backup) into a separate action and use that
in the primary test instead of repeating the same thing over and over.

Successful run: https://github.com/alcionai/corso/actions/runs/4980277816/jobs/8912982952

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-05-15 16:12:17 +00:00
Keepers
a9918d2f78
add repoRef pii transformer (#3383)
adds a transformer func in path/elements.go to safely log a repoRef (or any other dir ref) without needing to conceal the entire string.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Test Plan

- [x]  Unit test
2023-05-12 21:43:16 +00:00
Keepers
60f6d4a035
renaming/cleanup for sharepoint perms pt.1 (#3330)
No logic changes, just renaming and minor cleanup.

PRs to follow:
1. collect various maps in onedrive collections into a single cache.
2. logic changes and tests to produce sharepoint permissions backup/restore e2e.
3. extend permission identity-type retention.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #3135

#### Test Plan

- [x]  Unit test
2023-05-12 21:23:52 +00:00
Keepers
695f8060da
insert restoreEnd event into deferred func (#3400)
move the end-of-op notification event to a defer within backup/restore.Do(). This ensures we'll get the end event data even in case of failure.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🐛 Bugfix
- [x] 🤖 Supportability/Tests

#### Issue(s)

* #3388

#### Test Plan
2023-05-12 19:18:53 +00:00
Abin Simon
5995e7aff6
Fix json tag for modified time in sharepoint info (#3353)
Small typo fix for metadata store.

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-05-12 17:25:40 +00:00
Abin Simon
729543999f
Remove all uses of CORSO_URL_LOGGING (#3406)
https://github.com/alcionai/corso/pull/3212#discussion_r1191768302

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-05-12 16:52:40 +00:00
Keepers
4dc9bf9d64
add known issue about deleted library restore (#3390)
#### Does this PR need a docs update or release note?

- [x]  Yes, it's included

#### Type of change

- [x] 🗺️ Documentation

#### Issue(s)

* #3382
2023-05-12 16:02:14 +00:00
Abin Simon
f8aa37b822
Add non delta pagers to exchange (#3212)
When the user's mailbox is full, we cannot make use of delta apis. This adds initial changes needed to create separate delta and non delta pagers for all of exchange.

*I would suggest looking commit wise when reviewing the PR.*

---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-05-12 15:56:49 +00:00
dependabot[bot]
4274de2b73
⬆️ Bump github.com/aws/aws-sdk-go from 1.44.261 to 1.44.262 in /src (#3404)
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.44.261 to 1.44.262.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/aws/aws-sdk-go/releases">github.com/aws/aws-sdk-go's releases</a>.</em></p>
<blockquote>
<h1>Release v1.44.262 (2023-05-11)</h1>
<h3>Service Client Updates</h3>
<ul>
<li><code>service/connect</code>: Updates service documentation</li>
<li><code>service/elasticache</code>: Updates service API and documentation
<ul>
<li>Added support to modify the cluster mode configuration for the existing ElastiCache ReplicationGroups. Customers can now modify the configuration from cluster mode disabled to cluster mode enabled.</li>
</ul>
</li>
<li><code>service/es</code>: Updates service API and documentation
<ul>
<li>This release fixes DescribePackages API error with null filter value parameter.</li>
</ul>
</li>
<li><code>service/health</code>: Updates service documentation
<ul>
<li>Add support for regional endpoints</li>
</ul>
</li>
<li><code>service/ivs-realtime</code>: Updates service API, documentation, and paginators</li>
<li><code>service/omics</code>: Updates service API, documentation, and paginators</li>
<li><code>service/opensearch</code>: Updates service API</li>
<li><code>service/route53resolver</code>: Adds new service</li>
<li><code>service/support</code>: Updates service API and documentation
<ul>
<li>This release adds 2 new Support APIs, DescribeCreateCaseOptions and DescribeSupportedLanguages. You can use these new APIs to get available support languages.</li>
</ul>
</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="88d6da1d05"><code>88d6da1</code></a> Release v1.44.262 (2023-05-11) (<a href="https://redirect.github.com/aws/aws-sdk-go/issues/4837">#4837</a>)</li>
<li>See full diff in <a href="https://github.com/aws/aws-sdk-go/compare/v1.44.261...v1.44.262">compare view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=github.com/aws/aws-sdk-go&package-manager=go_modules&previous-version=1.44.261&new-version=1.44.262)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

You can trigger a rebase of this PR by commenting `@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


</details>
2023-05-12 10:12:22 +00:00
Keepers
1b417af5bb
minor updates to the path.Path interface (#3384)
Swaps path.Append from a single item method to accept a variadic list of string elements.  Since 95% of all calls to path.Append were items, also adds a shorthand
AppendItem func to the interface for easy clarity. Finally, adds a Last() method to elements for getting the last element in the slice.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-05-12 01:29:17 +00:00
ashmrtn
674d3eec91
Switch to folder IDs in Exchange (#3373)
Store all folders from Exchange by folder ID
in kopia. Remove the old duplicate folder
name stop-gap measure as well since it's no
longer required

Update tests to check LocationPath instead
of FullPath since LocationPath still has
display name info

---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

- [x] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

* closes #3197

#### Test Plan

- [x] 💪 Manual
- [ ]  Unit test
- [x] 💚 E2E
2023-05-11 23:07:38 +00:00
Keepers
6c410c298c
consts for drives, root: (#3385)
#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-05-11 22:46:51 +00:00
Keepers
caea3ab6da
make the linter happy (#3394)
#### Type of change

- [x] 🧹 Tech Debt/Cleanup
2023-05-11 20:38:33 +00:00
ashmrtn
97ca68fba1
Re-add GitHub Actions linting (#3391)
Move linting into the main CI workflow

Split into a different PR so that the file checker
gets updated and we can actually see if this is
working as intended

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [x] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

* closes #3389

#### Test Plan

- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-05-11 19:05:15 +00:00
Keepers
cf2aa9013a
refine rate limiter: per-query token consumption (#3358)
Now that the rate limiter is split by service, we can further split by per-query token consumption.  Two primary service cases exist: all exchange queries assume to cost a single token (for now).  Drive-service queries are split between permissions (5), default cost (2), and single-item or delta gets (1).

---

#### Does this PR need a docs update or release note?

- [x] 🕐 Yes, but in a later PR

#### Type of change

- [x] 🌻 Feature

#### Issue(s)

* #2951

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-05-11 18:23:14 +00:00
dependabot[bot]
aae991686d
⬆️ Bump github.com/aws/aws-sdk-go from 1.44.260 to 1.44.261 in /src (#3387)
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.44.260 to 1.44.261.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/aws/aws-sdk-go/releases">github.com/aws/aws-sdk-go's releases</a>.</em></p>
<blockquote>
<h1>Release v1.44.261 (2023-05-10)</h1>
<h3>Service Client Updates</h3>
<ul>
<li><code>service/elasticmapreduce</code>: Updates service API and documentation
<ul>
<li>EMR Studio now supports programmatically executing a Notebooks on an EMR on EKS cluster.  In addition, notebooks can now be executed by specifying its location in S3.</li>
</ul>
</li>
<li><code>service/rds</code>: Updates service API, documentation, waiters, paginators, and examples
<ul>
<li>Amazon Relational Database Service (RDS) updates for the new Aurora I/O-Optimized storage type for Amazon Aurora DB clusters</li>
</ul>
</li>
<li><code>service/swf</code>: Updates service API and documentation
<ul>
<li>This release adds a new API parameter to exclude old history events from decision tasks.</li>
</ul>
</li>
</ul>
<h3>SDK Bugs</h3>
<ul>
<li><code>service/sms</code>: Remove deprecated services (SMS) integration tests.
<ul>
<li>SMS integration tests will fail because SMS deprecated their service.</li>
</ul>
</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="c741f7708a"><code>c741f77</code></a> Release v1.44.261 (2023-05-10) (<a href="https://redirect.github.com/aws/aws-sdk-go/issues/4836">#4836</a>)</li>
<li><a href="225097cc88"><code>225097c</code></a> remove deprecated service AWS SMS integ tests (<a href="https://redirect.github.com/aws/aws-sdk-go/issues/4834">#4834</a>)</li>
<li>See full diff in <a href="https://github.com/aws/aws-sdk-go/compare/v1.44.260...v1.44.261">compare view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=github.com/aws/aws-sdk-go&package-manager=go_modules&previous-version=1.44.260&new-version=1.44.261)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

You can trigger a rebase of this PR by commenting `@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


</details>
2023-05-11 17:36:02 +00:00
ashmrtn
ebbf8aef75
More workflow linting changes (#3380)
Pickup a few missed things to get linting running
on github actions changes

Lint job is re-added in #3391 which will merge
after this one does so we can verify it's working

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [x] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issues

* #3389 

#### Test Plan

- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-05-11 17:17:54 +00:00
Keepers
0202207f3e
add nil pointer guard to resp.Headers in middleware (#3393)
#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🐛 Bugfix

#### Test Plan

- [x] 💚 E2E
2023-05-11 16:38:13 +00:00
neha_gupta
245d3ee089
treat / as root for restore of onedrive and sharepoint (#3328)
<!-- PR description-->

passing '/' will select anything for backup details and restore for onedrive and sharepoint

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #https://github.com/alcionai/corso/issues/3252
#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-05-11 05:34:13 +00:00
Keepers
f2f76d932d
release the sensitive-info flag (#3369)
Not 100% happy with the flag name, and am open to suggestions.

---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included

#### Type of change

- [x] 🌻 Feature
- [x] 🗺️ Documentation

#### Test Plan

- [x] 💪 Manual
- [x]  Unit test
2023-05-11 04:35:53 +00:00
ashmrtn
3be3b72d0a
Combine collection structs (#3375)
They both implement the same underlying functionality, just in slightly different ways. Combine them so there's less code duplication.

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

#### Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-05-11 03:54:49 +00:00
Keepers
e7d2aeac5d
add drive-level tombstone for deleted drives (#3381)
adds a tombstone collection for any drive that has been completely deleted (or that surfaced from a prior backup, but does not exist in the current) from the driveish account.
---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included

#### Type of change

- [x] 🐛 Bugfix

#### Issue(s)

* #3379

#### Test Plan

- [x]  Unit test
2023-05-11 03:20:28 +00:00
ashmrtn
2e4fc71310
Add restore path generation code (#3362)
In preparation for switching to folder IDs,
add logic to generate the restore path based
on prefix information from the RepoRef and
LocationRef of items

Contains fallback code (and tests) to handle
older details versions that may not have had
LocationRef

Manually tested restore from old backup that
didn't have any LocationRef information

Manually tested restore checking that
calendar names are shown instead of IDs in
progress bar

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #3197
* fixes #3218

#### Test Plan

- [x] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-05-11 02:49:32 +00:00
Keepers
c5b388a721
add indeets test helper, implement in exchange op (#3295)
Adds a helper for building expected details entries and checking
them after a backup.  Implements the helper in the exchange backup
tests in operations/backup integration.  Will follow with a onedrive
implementation.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🤖 Supportability/Tests

#### Issue(s)

* #3240

#### Test Plan

- [x] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2023-05-11 02:28:18 +00:00
Keepers
c0725b9cf9
some quick logging and error naming updates (#3348)
#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #3344

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-05-11 02:03:31 +00:00
Keepers
522d6d2206
fix up per-service token constraints (#3378)
#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🐛 Bugfix

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-05-11 01:22:05 +00:00
Keepers
833216c8ae
normalize folders restored in testing (#3363)
Normalize all test folders to use a constant prefix: Corso_Test.  This cleans up and centralizes all per- test variations on the restore destination.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🤖 Supportability/Tests

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-05-11 01:02:30 +00:00