1651 Commits

Author SHA1 Message Date
Abin Simon
8b21e25a82
Mark v0.4.0 as beta (#2592)
We are now in beta! 🎊
2023-02-21 04:27:10 +00:00
ashmrtn
ed6335e590
New restore path for items with name in metadata file (#2495)
## Description

Add code to restore items that have their file name stored in the corso metadata file instead of as the name of the kopia file.

This code is not executed as the conditional for it will not trigger (comparison to max int)

## Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No 

## Type of change

- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

## Issue(s)

* #1535 

## Test Plan

- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-02-21 04:26:25 +00:00
ashmrtn
4565c9f33b
Begin storing OneDrive file names in metadata (#2494)
## Description

This starts to store the name of the OneDrive file/directory in the corso metadata file for the OneDrive file/directory. This data is not yet used anywhere

## Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No 

## Type of change

- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

## Issue(s)

* #1535

## Test Plan

- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-02-21 03:12:16 +00:00
Keepers
d7087d7461
add awareness of missing folders (#2578)
## Does this PR need a docs update or release note?

- [x]  Yes, it's included

## Type of change

- [x] 🗺️ Documentation
## Issue(s)

* #2567
2023-02-20 22:22:40 +00:00
Danny
7ba3be5242
CLI: SharePoint --data flag and wildcards (#2580)
## Description
Enables feature SharePoint to use "wildcard" to select all available sites with the ability to only backup selected categories.

Example 
```bash
./corso backup create sharepoint --web-url "*" --data libraries
```



<!-- Insert PR description-->

## Does this PR need a docs update or release note?
- [x]  No 

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🐛 Bugfix


## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* closes #2579<issue>

## Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
2023-02-20 21:21:27 +00:00
Keepers
960cb45ca8
write test logs to stdout (#2568)
stderr logging writes all our logs to the terminal (ie, what shows up in github workflows) instead of the logging file artifact.  On failures with thousands of lines of logs, this makes searching for the error source nearly impossible.  This change ensures we have clean outputs in github workflows, and logs stored in artifact files for extensive review.
2023-02-20 19:49:50 +00:00
ashmrtn
cf62c6283c
Consolidate logic to see if Exchange/OneDrive user exists (#2508)
## Description

Use the check to see if a user has access to OneDrive/Exchange to also see if the user exists. This does not stop the population of GraphConnector users every time an instance is created, but does move us one step closer to that

Manually tested the following cases:
* backup "*"
* backup single existing user without error
* backup multiple existing users without error
* error when attempting to backup user that doesn't exist in the same domain
* error when attempting to backup user that doesn't exist in a different domain

Sample error output (progress disabled):
```text
No backups available
Error: 1 error occurred:
        * Failed to run Exchange backup for user foo@other.onmicrosoft.com: doing backup: producing backup data collections: resource owner [foo@other.onmicrosoft.com] not found within tenant
```

## Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No 

## Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

## Issue(s)

* #1547 

## Test Plan

- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-02-20 17:50:47 +00:00
Abin Simon
a86453f138
Add owner in json output for backup list (#2575)
## Description

This adds information about owner of the backup to `--json` output of backup list. Previously we were showing owner in the normal backup list command, but not when asked to as json.

## Does this PR need a docs update or release note?

- [x]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No 

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-02-20 14:50:28 +00:00
Abin Simon
8096025c55
Print api response in logs for 4xx errors (#2544)
## Description

Print api response in logs for 4xx errors from GC. More info for things like 400 Bad Request could be useful.

## Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No 

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

## Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-02-20 09:14:23 +00:00
neha_gupta
2b8e1ad2d9
Update CHANGELOG.md (#2574)
## Description

Updating changelog as v0.0.4 is out.

## Does this PR need a docs update or release note?
- [ ]  No 

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🗺️ Documentation

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

## Test Plan
- [ ] 💪 Manual
2023-02-20 08:04:27 +00:00
dependabot[bot]
0c76903dc4
⬆️ Bump sass from 1.58.1 to 1.58.3 in /website (#2572) 2023-02-20 06:23:27 +00:00
dependabot[bot]
d2d2fe1197
⬆️ Bump mermaid from 9.3.0 to 9.4.0 in /website (#2571) 2023-02-20 06:18:18 +00:00
Keepers
5707036b7c
add clues/fault to sharepoint api (#2507)
## Does this PR need a docs update or release note?

- [x]  No 

## Type of change

- [x] 🧹 Tech Debt/Cleanup

## Issue(s)

* #1970

## Test Plan

- [x]  Unit test
- [x] 💚 E2E
v0.4.0
2023-02-18 20:42:48 +00:00
Keepers
b1ff20d36c
add clues/fault to sharepoint collections (#2506)
## Does this PR need a docs update or release note?

- [x]  No 

## Type of change

- [x] 🧹 Tech Debt/Cleanup

## Issue(s)

* #1970

## Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-02-18 19:51:51 +00:00
Abin Simon
7e3532832e
More fine grained error messages for retries (#2545)
## Description

This should help in figuring out more retry failures.

## Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No 

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

## Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-02-18 17:16:42 +00:00
Niraj Tolia
6b79355948
Move Corso to beta (#2566)
## Description

Move Corso out of alpha

## Does this PR need a docs update or release note?

- [x]  No 

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🗺️ Documentation
2023-02-18 15:35:59 +00:00
ashmrtn
e215462518
Add log messages about metadata saved/used (#2551)
## Description

Record the number of entries in the saved metadata. This could help catch odd situations like not finding anything to backup.

## Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No 

## Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

## Issue(s)

* #2550

## Test Plan

- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-02-18 06:25:05 +00:00
dependabot[bot]
bf12428738
⬆️ Bump github.com/aws/aws-sdk-go from 1.44.202 to 1.44.204 in /src (#2564)
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.44.202 to 1.44.204.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/aws/aws-sdk-go/releases">github.com/aws/aws-sdk-go's releases</a>.</em></p>
<blockquote>
<h1>Release v1.44.204 (2023-02-17)</h1>
<h3>Service Client Updates</h3>
<ul>
<li><code>service/apprunner</code>: Updates service API</li>
<li><code>service/glue</code>: Updates service API and documentation
<ul>
<li>Release of Delta Lake Data Lake Format for Glue Studio Service</li>
</ul>
</li>
</ul>
<h1>Release v1.44.203 (2023-02-16)</h1>
<h3>Service Client Updates</h3>
<ul>
<li><code>service/elasticmapreduce</code>: Updates service API and documentation
<ul>
<li>This release provides customers the ability to define a timeout period for procuring capacity during a resize operation for Instance Fleet clusters. Customers can specify this timeout using the ResizeSpecifications parameter supported by RunJobFlow, ModifyInstanceFleet and AddInstanceFleet APIs.</li>
</ul>
</li>
<li><code>service/grafana</code>: Updates service API and documentation</li>
<li><code>service/ivs</code>: Updates service documentation</li>
<li><code>service/wafv2</code>: Updates service documentation</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="ad756cba2a"><code>ad756cb</code></a> Release v1.44.204 (2023-02-17) (<a href="https://github-redirect.dependabot.com/aws/aws-sdk-go/issues/4730">#4730</a>)</li>
<li><a href="8f70ea398e"><code>8f70ea3</code></a> add go 1.20 support (<a href="https://github-redirect.dependabot.com/aws/aws-sdk-go/issues/4724">#4724</a>)</li>
<li><a href="ec9362abbd"><code>ec9362a</code></a> Release v1.44.203 (2023-02-16) (<a href="https://github-redirect.dependabot.com/aws/aws-sdk-go/issues/4727">#4727</a>)</li>
<li>See full diff in <a href="https://github.com/aws/aws-sdk-go/compare/v1.44.202...v1.44.204">compare view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=github.com/aws/aws-sdk-go&package-manager=go_modules&previous-version=1.44.202&new-version=1.44.204)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

You can trigger a rebase of this PR by commenting `@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


</details>
2023-02-18 05:46:46 +00:00
Vaibhav Kamra
c3f5e50b9f
Fix how ErrrorData is marshaled (#2565)
## Description

The error type does not get marshaled which can lead to errors when we try to un-marshal a manifest
that had errors stored in it.

Repro here: https://go.dev/play/p/tgj8oq5CGFd

For now - this disables JSON marshaling and also fixes the error un-marshaling. I have added a regression test
to verify both behaviors.

As a follow-up, I believe we can implement a custom marshaler.

## Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No 

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-02-18 01:43:59 +00:00
Keepers
207232e8d9
add clues/fault to sharepoint restores (#2505)
## Does this PR need a docs update or release note?

- [x]  No 

## Type of change

- [x] 🧹 Tech Debt/Cleanup

## Issue(s)

* #1970

## Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-02-17 22:05:32 +00:00
Vaibhav Kamra
daad056d7e
Pull in fix for concurrent map updates in clues (#2562)
## Description

Pulls in fix for a panic caused by concurrent map updates in the clues pkg

## Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No 

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2023-02-17 21:24:06 +00:00
Danny
62d652764a
GC: Ptr clean-up and Invalid NextLink logging (#2547)
## Description
- Use of ptr package for next link and delta links. 
- Added logging to verify the presence of receiving invalid next links from M365.


<!-- Insert PR description-->

## Does this PR need a docs update or release note?

- [x]  No 

- Related to #2520
## Type of change
- [x] 🧹 Tech Debt/Cleanup

## Test Plan

<!-- How will this be tested prior to merging.-->
- [x]  Unit test
2023-02-17 19:20:21 +00:00
Keepers
d395311821
error logging around malformed urls (#2561)
adds debug logging around url construction
2023-02-17 18:16:45 +00:00
Keepers
c92b70e000
wrap up clues and fault additions to exchange (#2504)
## Does this PR need a docs update or release note?

- [x]  No 

## Type of change

- [x] 🧹 Tech Debt/Cleanup

## Issue(s)

* #1970

## Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-02-17 17:08:08 +00:00
Niraj Tolia
7c8c464fea
Fix Corso panics in Docker images (#2543)
## Description

The `KOPIA_CACHE_DIRECTORY` env variable causes Corso to panic even if `$CORSO_HOME/kopia/cache` exists.
Removing it fixes the builds but I am unsure of any side effects.

## Does this PR need a docs update or release note?

- [x]  Yes, it's included

## Type of change

- [x] 🐛 Bugfix

## Issue(s)

* #2529 

## Test Plan

- [x] 💪 Manual
2023-02-17 05:29:58 +00:00
ashmrtn
0ff218ab3e
Hook into error callback in kopia (#2555)
## Description

Using this over extracting errors from the snapshot allows us to see all errors. The errors that each directory tracks are truncated when the directory is serialized for persistence if there're too many.

With the current `fault.Errors` behavior this cases the backup to fail if there's any errors
kopia ignored. This is consistent with behavior in the backup op that checks for any errors.

Unclear what the CLI output and log messages will be for these errors as they are grouped
into all other errors reported through the fault package

## Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No 

## Type of change

- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

## Issue(s)

* #2548

## Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-02-17 04:32:44 +00:00
ashmrtn
7b9ba935e8
log and continue if kopia can't find expected base snapshot directory (#2552)
## Description

This allows corso to continue making a snapshot even if the expected directory in the base is not found. The fact that the entry wasn't found will be logged

This should be safe so long as backups that had any sort of error, including "non-catastrophic" errors are marked as failed. Marking those as failed ensures that corso is using a matched set of metadata and data in the base snapshot rather than a partial snapshot

## Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [x] 🕐 Yes, but in a later PR
- [ ]  No 

## Type of change

- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

## Issue(s)

* #2550

## Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-02-17 03:35:55 +00:00
ashmrtn
7995640109
Disable kopia optimization that lets us reuse a base directory (#2522)
## Description

Just handing the base directory to kopia doesn't let us see which items have been added which means we can't properly merge backup details later on.

Future implementations could do backup details merging based on path prefixes which could fix this problem.

## Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No 

## Type of change

- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

## Issue(s)

* closes #2521

## Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-02-16 19:57:48 +00:00
Keepers
28ad304bb7
update items to accept ctx, fault.Errors (#2493)
## Description

In order for corso to track recoverable errors,
we need to pass a fault.Errors struct into the
items stream.  As long as we're doing that, we
might as well pass along the available ctx as well.

## Does this PR need a docs update or release note?

- [x]  No 

## Type of change

- [x] 🧹 Tech Debt/Cleanup

## Issue(s)

* #1970

## Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-02-16 19:09:20 +00:00
dependabot[bot]
fffa633971
⬆️ Bump github.com/aws/aws-sdk-go from 1.44.201 to 1.44.202 in /src (#2538)
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.44.201 to 1.44.202.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/aws/aws-sdk-go/releases">github.com/aws/aws-sdk-go's releases</a>.</em></p>
<blockquote>
<h1>Release v1.44.202 (2023-02-15)</h1>
<h3>Service Client Updates</h3>
<ul>
<li><code>service/cloudtrail</code>: Updates service API
<ul>
<li>This release adds an InsufficientEncryptionPolicyException type to the StartImport endpoint</li>
</ul>
</li>
<li><code>service/elasticfilesystem</code>: Updates service documentation
<ul>
<li>Documentation update for EFS to support IAM best practices.</li>
</ul>
</li>
<li><code>service/frauddetector</code>: Updates service API, documentation, and paginators</li>
<li><code>service/glue</code>: Updates service API and documentation
<ul>
<li>Fix DirectJDBCSource not showing up in CLI code gen</li>
</ul>
</li>
<li><code>service/privatenetworks</code>: Updates service API and documentation</li>
<li><code>service/rds</code>: Updates service API, documentation, waiters, paginators, and examples
<ul>
<li>Database Activity Stream support for RDS for SQL Server.</li>
</ul>
</li>
<li><code>service/wafv2</code>: Updates service API and documentation</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="6cdd617804"><code>6cdd617</code></a> Release v1.44.202 (2023-02-15) (<a href="https://github-redirect.dependabot.com/aws/aws-sdk-go/issues/4726">#4726</a>)</li>
<li>See full diff in <a href="https://github.com/aws/aws-sdk-go/compare/v1.44.201...v1.44.202">compare view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=github.com/aws/aws-sdk-go&package-manager=go_modules&previous-version=1.44.201&new-version=1.44.202)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

You can trigger a rebase of this PR by commenting `@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


</details>
2023-02-16 18:32:57 +00:00
Abin Simon
ce37bc039f
Revert "⬆️ Bump docker/build-push-action from 3 to 4" (#2540)
Reverts alcionai/corso#2402
https://github.com/alcionai/corso/issues/2539
2023-02-16 16:55:22 +00:00
Keepers
bb69eeae4c
some fault err explanation cleanup (#2464)
## Type of change

- [x] 🧹 Tech Debt/Cleanup

## Issue(s)

* #1970

## Test Plan

- [x] 💪 Manual
2023-02-16 15:31:27 +00:00
Niraj Tolia
ffcc248398
Correctly add CA certificates (#2542)
## Description

This ensures that the CA certificates are not from some random version of Alpine but from the version we are using as a base image.

## Does this PR need a docs update or release note?

- [x]  No 

## Type of change

- [x] 🐛 Bugfix
- [x] 🧹 Tech Debt/Cleanup

## Issue(s)

* #2529 

## Test Plan

- [x] 💪 Manual
2023-02-16 07:40:00 +00:00
Abin Simon
727416c6b0
Remove redundant err check (#2541)
## Description

<!-- Insert PR description-->

## Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No 

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

## Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-02-16 07:34:58 +00:00
Abin Simon
2a66097e15
Check for err in repo connect (#2537)
## Description

Add a missing err check to repo connect. Follow up to https://github.com/alcionai/corso/pull/2502 .

## Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No 

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

## Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-02-16 05:44:32 +00:00
Abin Simon
55825afdbd
Deletion related fixes for OneDrive delta incrementals (#2518)
## Description

- parentReference for deleted items will not have path.
- GC was not responding with deleted state
- Deleted items are not streamed by kopia

Borrows some changes from https://github.com/alcionai/corso/pull/2503/files as that might be delayed with on merge. Will rebase that if this gets merged fist.

## Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [x] 🕐 Yes, but in a later PR
- [ ]  No 

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* https://github.com/alcionai/corso/issues/2117
* fixes https://github.com/alcionai/corso/issues/2517

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-02-16 04:19:50 +00:00
ashmrtn
476a931ccb
Update tests in kopia package (#2514)
## Description

Use the new suite.Suite wrappers in tester package. Also update suite.Run and some nolint directives.

## Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No 

## Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Test
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

## Issue(s)

* #2373

## Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-02-16 02:22:43 +00:00
ashmrtn
034ee0df39
Use suite wrapper in internal/common package (#2527)
## Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No 

## Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Test
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

## Issue(s)

* #2373

## Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-02-16 01:10:13 +00:00
ashmrtn
efe1b6292d
Extra test for some recursive delete behavior (#2478)
## Description

Folders replaced after a delete should not have their subfolders reparented to the new folder.

## Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No 

## Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

## Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-02-15 21:14:32 +00:00
Keepers
7187b969e4
centralize observe channel listener handling (#2523)
## Description

To ensure that the observe channel handling
doesn't accidentally spawn unkillable routines,
adds a centralized channel listener func for
standard channel management in observe
progress bars.

## Does this PR need a docs update or release note?

- [x]  No 

## Type of change

- [x] 🐛 Bugfix

## Test Plan

- [x] 💪 Manual
- [x]  Unit test
2023-02-15 20:20:07 +00:00
Abin Simon
e60ae2351f
Set proper state and paths for moves in OneDrive (#2501)
## Description

Ties up the final piece of https://github.com/alcionai/corso/issues/2123. Add handling of moves for folder / files in delta response.

## Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [x] 🕐 Yes, but in a later PR
- [ ]  No 

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* https://github.com/alcionai/corso/issues/2123

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-02-15 18:27:45 +00:00
neha_gupta
123aec2bd8
Handle repo connect event on - repo connect command (#2502)
## Description

Create a `repo connect` event on CLI's repo connect command 

## Does this PR need a docs update or release note?
- [ ]  No 

## Type of change
- [x] 🌻 Feature

## Issue(s)
https://github.com/alcionai/corso/issues/1762

## Test Plan
- [x] 💪 Manual
2023-02-15 23:38:19 +05:30
dependabot[bot]
8d7c799204
⬆️ Bump github.com/aws/aws-sdk-go from 1.44.200 to 1.44.201 in /src (#2515)
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.44.200 to 1.44.201.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/aws/aws-sdk-go/releases">github.com/aws/aws-sdk-go's releases</a>.</em></p>
<blockquote>
<h1>Release v1.44.201 (2023-02-14)</h1>
<h3>Service Client Updates</h3>
<ul>
<li><code>service/appconfig</code>: Updates service API and documentation</li>
<li><code>service/datasync</code>: Updates service API and documentation</li>
<li><code>service/ec2</code>: Updates service API and documentation
<ul>
<li>With this release customers can turn host maintenance on or off when allocating or modifying a supported dedicated host. Host maintenance is turned on by default for supported hosts.</li>
</ul>
</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="0df2f81e6d"><code>0df2f81</code></a> Release v1.44.201 (2023-02-14) (<a href="https://github-redirect.dependabot.com/aws/aws-sdk-go/issues/4725">#4725</a>)</li>
<li><a href="cc5f9c0266"><code>cc5f9c0</code></a> remove failing Go versions on macos-latest (<a href="https://github-redirect.dependabot.com/aws/aws-sdk-go/issues/4655">#4655</a>)</li>
<li><a href="5c5783001c"><code>5c57830</code></a> Update processcreds.CredentialProcessResponse visibility to public (<a href="https://github-redirect.dependabot.com/aws/aws-sdk-go/issues/4621">#4621</a>)</li>
<li><a href="8e62f0aa75"><code>8e62f0a</code></a> Fix expect header value to '100-continue' (<a href="https://github-redirect.dependabot.com/aws/aws-sdk-go/issues/4722">#4722</a>)</li>
<li>See full diff in <a href="https://github.com/aws/aws-sdk-go/compare/v1.44.200...v1.44.201">compare view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=github.com/aws/aws-sdk-go&package-manager=go_modules&previous-version=1.44.200&new-version=1.44.201)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

You can trigger a rebase of this PR by commenting `@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


</details>
2023-02-15 17:19:02 +00:00
Keepers
26d286138b
add fault & clues to exchange api (#2492)
## Does this PR need a docs update or release note?

- [x]  No 

## Type of change

- [x] 🧹 Tech Debt/Cleanup

## Issue(s)

* #1970

## Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-02-15 17:01:04 +00:00
Danny
e70bf25f6e
GC: Restore: Libraries Workflow (#2511)
## Description
Due to recent changes with `Incrementals` restore of Library categories stopped working. This PR will allow for files that do not require special folder permissions to be restored. Changes involve the folder naming format that was coming from the CLI
<!-- Insert PR description-->

## Does this PR need a docs update or release note?

- [x]  No 

## Type of change

- [x] 🐛 Bugfix


## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* closes  #2489<issue>

## Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
2023-02-15 13:30:03 +00:00
ashmrtn
28fcafe3b2
Consolidate/move OneDrive GraphConnector tests (#2487)
## Description

Now that we're also testing permissions behavior the OneDrive tests have gotten kind of large. This PR consolidates some of the code for tests and moves the OneDrive tests to their own file

It also sets things up so it's easier to add tests for different backup layouts in the future

Viewing by commit may make reviews easier

## Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No 

## Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

## Issue(s)

* #2447

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-02-14 21:27:25 +00:00
Keepers
570ce85656
add clues/fault to exchange restore (#2491)
## Does this PR need a docs update or release note?

- [ ]  No 

## Type of change

- [x] 🧹 Tech Debt/Cleanup

## Issue(s)

* #1970

## Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-02-14 19:33:07 +00:00
ashmrtn
31c9195a12
Proof of concept wrapper for suite.Suite (#2410)
## Description

Proof of concept for how to wrap suite.Suite and insert our own logic for determining if a test should be skipped or not.

Adds a new env var for running only integration tests

May require future work to add unit tests with the same (non-default) labels as their integration tests

## Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No 

## Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Test
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

## Issue(s)

* #2373

## Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-02-14 17:56:26 +00:00
neha_gupta
2e128726c0
add retry middleware (#2406)
## Description

Added retry middleware which handles retrial of all HTTP request if,
- internalServerError 
- request timeout
 
with default no of retrials being 3.

## Does this PR need a docs update or release note?
- [ ]  No 

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🧹 Tech Debt/Cleanup

## Issue(s)
* https://github.com/alcionai/corso/issues/2287

## Test Plan
- [x] 💪 Manual
2023-02-14 22:58:16 +05:30
dependabot[bot]
b646f502ef
⬆️ Bump github.com/aws/aws-sdk-go from 1.44.199 to 1.44.200 in /src (#2500)
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.44.199 to 1.44.200.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/aws/aws-sdk-go/releases">github.com/aws/aws-sdk-go's releases</a>.</em></p>
<blockquote>
<h1>Release v1.44.200 (2023-02-13)</h1>
<h3>Service Client Updates</h3>
<ul>
<li><code>service/account</code>: Updates service API, documentation, and paginators</li>
<li><code>service/appconfigdata</code>: Updates service API and documentation</li>
<li><code>service/snowball</code>: Updates service API and documentation
<ul>
<li>Adds support for EKS Anywhere on Snowball. AWS Snow Family customers can now install EKS Anywhere service on Snowball Edge Compute Optimized devices.</li>
</ul>
</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="d0700c118b"><code>d0700c1</code></a> Release v1.44.200 (2023-02-13) (<a href="https://github-redirect.dependabot.com/aws/aws-sdk-go/issues/4723">#4723</a>)</li>
<li>See full diff in <a href="https://github.com/aws/aws-sdk-go/compare/v1.44.199...v1.44.200">compare view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=github.com/aws/aws-sdk-go&package-manager=go_modules&previous-version=1.44.199&new-version=1.44.200)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

You can trigger a rebase of this PR by commenting `@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


</details>
2023-02-14 06:59:10 +00:00