1148 Commits

Author SHA1 Message Date
Keepers
eacb638099
do not wait on deleted collections (#1953)
## Type of change

- [x] 🐛 Bugfix

## Issue(s)

* #1950

## Test Plan

- [x] 💪 Manual
v0.0.4
2022-12-24 00:10:49 +00:00
Keepers
442686e51e
distinguish added and removed jobs (#1952)
## Description

In order to track deleted items, we need to build
separate job sets when aggregating item IDs:
one for added items, and one for removed.

## Does this PR need a docs update or release note?

- [x]  No 

## Type of change

- [x] 🐛 Bugfix

## Issue(s)

* #1777

## Test Plan

- [x] 💪 Manual
2022-12-23 23:53:57 +00:00
ashmrtn
843772ec50
Don't merge directories if they're marked as new (#1940)
## Description

If we have a collection marked as new and we have a base directory, do not merge the base with the collection.

## Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No 

## Type of change

- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

* #1740 

## Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-12-23 23:27:51 +00:00
ashmrtn
76ebd7254d
Source resource owner from prev path if needed (#1949)
## Description

Deleted collections have full path set to nil but have their previous path populated. When gathering information for statistics reporting, check that we have a non-nil value to get resource owner from.

## Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No 

## Type of change

- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

* closes #1948 

## Test Plan

- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2022-12-23 23:13:32 +00:00
Keepers
73c4937456
use prevPath in data.Collection (#1943)
## Type of change

- [x] 🐛 Bugfix

## Issue(s)

* #1777

## Test Plan

- [x] 💪 Manual
2022-12-23 21:48:37 +00:00
Abin Simon
a1444f529e
Make snapshot release in goreleaser if from main (#1899)
## Description

This makes sure we switch between `--snapshot` and regular release based of we are doing it from main branch.

## Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No 

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [x] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* fixes https://github.com/alcionai/corso/issues/1873

## Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2022-12-23 21:11:49 +00:00
ashmrtn
a9da26d3cc
Test for deleting and making a new collection at the same path (#1888)
## Description

<!-- Insert PR description-->

## Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No 

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2022-12-23 20:54:53 +00:00
Keepers
b575f2ff68
check for conflicts on tombstone collections (#1927)
## Description

Ensure that we do not generate both a regular
collection and a tombstone for the same id.

## Does this PR need a docs update or release note?

- [x]  No 

## Type of change

- [x] 🐹 Trivial/Minor

## Issue(s)

* #1923

## Test Plan

- [x] 💚 E2E
2022-12-23 20:38:16 +00:00
Keepers
9e9cb43b58
retain kopia-assist when not on incrementals (#1941)
## Description

Adds a flag to the BackupCollections interface
that identifies whether the caller is running an
incremental backup or not.  If they are, kopia
will utilize the previous base snapshots when
building the directory tree.

## Does this PR need a docs update or release note?

- [x]  No 

## Type of change

- [x] 🌻 Feature

## Issue(s)

* #1901

## Test Plan

- [x] 💚 E2E
2022-12-23 20:14:05 +00:00
ashmrtn
a4791af7bf
Remove some dead code in upload.go (#1920)
## Description

Code was missed when switching to having BackupOp supply the OwnersCats for a backup.

## Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No 

## Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [x] 🐹 Trivial/Minor

## Issue(s)

* #1781 

## Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-12-23 19:57:49 +00:00
ashmrtn
35c1b10912
Wire details merging during BackupOp.Run (#1905)
## Description

Actually call the details merge function while performing a backup. If no base snapshots or `len(toMerge) == 0` then turns into a noop.

This PR contains a little extra refactoring to keep from initializing a struct multiple times. However, long-term we should probably refactor BackupOp to better fit the fact that we are now using interfaces as inputs to many functions for the sake of testing

## Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No 

## Type of change

- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

* closes #1800 

## Test Plan

- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2022-12-23 19:14:31 +00:00
Niraj Tolia
ff1b54b551
Improve .md code color (#1918)
## Description

Inherit from top-level color schemes so that the text color is readable in both dark and light modes

## Does this PR need a docs update or release note?

- [x]  No 

## Type of change

- [x] 🐛 Bugfix
- [x] 🗺️ Documentation
2022-12-23 18:01:41 +00:00
ashmrtn
91b9f2e825
Propagate options to repo and operations (#1932)
## Description

Make sure the options struct is passed through the entire stack so we actually have access to flag values.

## Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No 

## Type of change

- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [x] 🐹 Trivial/Minor

## Issue(s)

* closes #1931 

## Test Plan

- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2022-12-23 17:23:06 +00:00
dependabot[bot]
37f1a3eca5
⬆️ Bump github.com/aws/aws-sdk-go from 1.44.165 to 1.44.166 in /src (#1929)
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.44.165 to 1.44.166.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/aws/aws-sdk-go/releases">github.com/aws/aws-sdk-go's releases</a>.</em></p>
<blockquote>
<h1>Release v1.44.166 (2022-12-22)</h1>
<h3>Service Client Updates</h3>
<ul>
<li><code>service/compute-optimizer</code>: Updates service API, documentation, and paginators</li>
<li><code>service/connect</code>: Updates service API and documentation</li>
<li><code>service/iotdeviceadvisor</code>: Updates service API and documentation</li>
<li><code>service/kinesis-video-webrtc-storage</code>: Updates service API and documentation</li>
<li><code>service/rds</code>: Updates service API, documentation, waiters, paginators, and examples
<ul>
<li>Add support for managing master user password in AWS Secrets Manager for the DBInstance and DBCluster.</li>
</ul>
</li>
<li><code>service/secretsmanager</code>: Updates service documentation
<ul>
<li>Documentation updates for Secrets Manager</li>
</ul>
</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="06491f505c"><code>06491f5</code></a> Release v1.44.166 (2022-12-22) (<a href="https://github-redirect.dependabot.com/aws/aws-sdk-go/issues/4670">#4670</a>)</li>
<li>See full diff in <a href="https://github.com/aws/aws-sdk-go/compare/v1.44.165...v1.44.166">compare view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=github.com/aws/aws-sdk-go&package-manager=go_modules&previous-version=1.44.165&new-version=1.44.166)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

You can trigger a rebase of this PR by commenting `@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


</details>
2022-12-23 06:41:45 +00:00
Niraj Tolia
9927ec3130
Fix error introduced by search/replace (#1928)
## Description

Fix Azure client ID docs description

## Does this PR need a docs update or release note?

- [x]  No 

## Type of change

- [x] 🐛 Bugfix
- [x] 🗺️ Documentation
2022-12-23 04:37:04 +00:00
ashmrtn
62c8f8c6ea
Update details merge function for details Builder and new location of folder population (#1926)
## Description

Folder population now done when merging items in BackupOp. Also add more tests to make sure folders are actually populated properly in Details

## Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No 

## Type of change

- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

* #1800 

## Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-12-23 02:04:14 +00:00
ashmrtn
c3dbd5e0a8
Handle DoNotMergeItems in hierarchy merge (#1922)
## Description

The DoNotMergeItems flag denotes situations where we want to propagate changes to the hierarchy but do not want to source items from the base for a specific directory. As of now, the only time we expect to encounter this situation is when a delta token expires in M365 and we need to pull all the items for the container again. By setting DoNotMergeItems, a collection can propagate things like rename to its subtree while avoiding zombie items that would have appeared if there was a deletion in the container and the container was enumerated again

## Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No 

## Type of change

- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

* #1914 

## Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-12-23 01:40:26 +00:00
ashmrtn
bc56d38970
Functions to merge a set of backup details (#1904)
## Description

Go through the provided bases, load their backup details, and check if
any of the items in them need to be merged into the details for the
current backup.

Has a small amount of logic to treat moved items as updated.

Also include all the tests

Viewing by commit may be useful

## Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No 

## Type of change

- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

* #1800 

## Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-12-23 01:18:08 +00:00
Keepers
b6a2227acc
reset container if deleted in flight, not delete (#1924)
## Does this PR need a docs update or release note?

- [x]  No 

## Type of change

- [x] 🐹 Trivial/Minor

## Issue(s)

* #1923

## Test Plan

- [x] 💚 E2E
2022-12-23 00:46:36 +00:00
Keepers
d0093fdb17
track tombstones by id, not path (#1917)
## Description

Fixes a bug where we could incorrectly mark a previousPath as both
deleted and moved/created at the same time.

## Does this PR need a docs update or release note?

- [x]  No 

## Type of change

- [x] 🐛 Bugfix

## Issue(s)

* #1780

## Test Plan

- [x] 💚 E2E
2022-12-23 00:20:16 +00:00
ashmrtn
04851f45dc
Lift'n'shift of existing code (#1925)
## Description

Factor out code to get a set of FolderEntries based on some path information. This code will be used in BackupOp when merging item details.

## Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No 

## Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [x] 🐹 Trivial/Minor

## Issue(s)

* #1800 

## Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-12-22 23:55:17 +00:00
Keepers
168ba1b88c
set nomerge on delta reset (#1921)
## Description

Flag doNotMergeItems for exchange collections
where the delta token was marked as reset (ie,
expired or otherwise unusable).  Do this instead
of making a (delete, new) collection tuple for
that path.


## Does this PR need a docs update or release note?

- [x]  No 

## Type of change

- [x] 🌻 Feature

## Issue(s)

* #1914

## Test Plan

- [x] 💚 E2E
2022-12-22 22:52:47 +00:00
ashmrtn
6b689b76c1
Produce map of items that need backup details entries (#1892)
## Description

Generate a `map[old ShortRef]new path.Path` for every item sourced from a base snapshot during backup. Return this information at the end of BackupCollections so that callers can use it to merge backup details if desired.

This PR does a few auxiliary things as well:
* thread current and previous path through hierarchy merging
* add extra field to info tracked by corsoProgress to hold previous path
* provide current and previous paths to streamBaseItems function

## Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No 

## Type of change

- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

* #1800 

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-12-22 22:29:15 +00:00
Keepers
aacb013b60
add doNotMerge func to collections (#1919)
## Description

Adds a new func to the data.Collection iface:
DoNotMergeItems() tells kopia to skip the
retention of items from prior snapshots when
generating the new snapshot for this collection.
A primary use case for this flag is when a delta
token expires, preventing an incremental lookup
and forcing gc to re-discover all items in the
container.

## Does this PR need a docs update or release note?

- [x]  No 

## Type of change

- [x] 🌻 Feature

## Issue(s)

* #1914

## Test Plan

- [x]  Unit test
2022-12-22 21:27:11 +00:00
Keepers
402e6b5139
handle invalid or expired deltas (#1882)
## Description

If a delta token expires or is otherwise invalid,
the backup should fall back to the same
behavior as if the collection were new.  It must
collect the full delta of information, and create
a collection with New state.  If the previous
container moved between the two deltas,
it should be marked for deletion with a tomb-
stone collection.

## Does this PR need a docs update or release note?

- [x]  No 

## Type of change

- [x] 🌻 Feature

## Issue(s)

* #1780

## Test Plan

- [x] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2022-12-22 20:54:00 +00:00
ashmrtn
f37ee2b942
Factor out function to retrieve backup details (#1894)
## Description

Given a backup ID pull the backup details for the backup. This function will also be used during incremental backups as the backup details from the base snapshot(s) will need merged with the details generated by backing up new/changed items.

## Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No 

## Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [x] 🐹 Trivial/Minor

## Issue(s)

* #1800 

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [ ]  Unit test
- [x] 💚 E2E
2022-12-22 20:35:10 +00:00
ashmrtn
5243dddcbf
Stricter checking for how hierarchies are merged (#1909)
## Description

Add some extra error checking for how the hierarchy can evolve during merging in kopia.Wrapper. Add more tests to solidify this behavior as well.

## Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No 

## Type of change

- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

* closes #1884 

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-12-22 20:13:54 +00:00
Niraj Tolia
741b36da98
Adding missing docs for deleting OD/EX (#1902)
## Description

There were missed as the addition process to docs is manual

## Does this PR need a docs update or release note?

- [x]  No 

## Type of change

- [x] 🐛 Bugfix
- [x] 🗺️ Documentation
2022-12-22 17:30:10 +00:00
Keepers
bdd7a456bc
do not return errs from tester setup funcs (#1886)
## Type of change

- [x] 🤖 Test
- [x] 🐹 Trivial/Minor

## Test Plan

- [x]  Unit test
- [x] 💚 E2E
2022-12-22 16:11:02 +00:00
dependabot[bot]
460bf7d096
⬆️ Bump github.com/aws/aws-sdk-go from 1.44.164 to 1.44.165 in /src (#1911)
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.44.164 to 1.44.165.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/aws/aws-sdk-go/releases">github.com/aws/aws-sdk-go's releases</a>.</em></p>
<blockquote>
<h1>Release v1.44.165 (2022-12-21)</h1>
<h3>Service Client Updates</h3>
<ul>
<li><code>service/connect</code>: Updates service API</li>
<li><code>service/connectparticipant</code>: Updates service API</li>
<li><code>service/license-manager-linux-subscriptions</code>: Adds new service</li>
<li><code>service/macie2</code>: Updates service API</li>
<li><code>service/sagemaker</code>: Updates service API and documentation
<ul>
<li>This release enables adding RStudio Workbench support to an existing Amazon SageMaker Studio domain. It allows setting your RStudio on SageMaker environment configuration parameters and also updating the RStudioConnectUrl and RStudioPackageManagerUrl parameters for existing domains</li>
</ul>
</li>
<li><code>service/scheduler</code>: Updates service API</li>
<li><code>service/ssm</code>: Updates service documentation
<ul>
<li>Doc-only updates for December 2022.</li>
</ul>
</li>
<li><code>service/support</code>: Updates service documentation
<ul>
<li>Documentation updates for the AWS Support API</li>
</ul>
</li>
<li><code>service/transfer</code>: Updates service API and documentation
<ul>
<li>This release adds support for Decrypt as a workflow step type.</li>
</ul>
</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="e8457468d2"><code>e845746</code></a> Release v1.44.165 (2022-12-21) (<a href="https://github-redirect.dependabot.com/aws/aws-sdk-go/issues/4669">#4669</a>)</li>
<li>See full diff in <a href="https://github.com/aws/aws-sdk-go/compare/v1.44.164...v1.44.165">compare view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=github.com/aws/aws-sdk-go&package-manager=go_modules&previous-version=1.44.164&new-version=1.44.165)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

You can trigger a rebase of this PR by commenting `@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


</details>
2022-12-22 10:15:05 +00:00
dependabot[bot]
e4a8f56680
⬆️ Bump github.com/microsoft/kiota-http-go from 0.10.0 to 0.11.0 in /src (#1912)
Bumps [github.com/microsoft/kiota-http-go](https://github.com/microsoft/kiota-http-go) from 0.10.0 to 0.11.0.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/microsoft/kiota-http-go/releases">github.com/microsoft/kiota-http-go's releases</a>.</em></p>
<blockquote>
<h2>v0.11.0</h2>
<h3>Changed</h3>
<ul>
<li>Fixed a bug where retry handling wouldn't rewind the request body before retrying.</li>
</ul>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a href="https://github.com/microsoft/kiota-http-go/blob/main/CHANGELOG.md">github.com/microsoft/kiota-http-go's changelog</a>.</em></p>
<blockquote>
<h2>[0.11.0] - 2022-12-20</h2>
<h3>Changed</h3>
<ul>
<li>Fixed a bug where retry handling wouldn't rewind the request body before retrying.</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="6f459d630f"><code>6f459d6</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/microsoft/kiota-http-go/issues/50">#50</a> from meain/retry-seek</li>
<li><a href="9d01b549b3"><code>9d01b54</code></a> - adds a changelog entry for the retry handling seek</li>
<li><a href="33cb28b298"><code>33cb28b</code></a> Seek req.Body to 0 before retry</li>
<li><a href="fe2d2fbdb0"><code>fe2d2fb</code></a> Bump github.com/microsoft/kiota-abstractions-go from 0.15.0 to 0.15.1 (<a href="https://github-redirect.dependabot.com/microsoft/kiota-http-go/issues/49">#49</a>)</li>
<li>See full diff in <a href="https://github.com/microsoft/kiota-http-go/compare/v0.10.0...v0.11.0">compare view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=github.com/microsoft/kiota-http-go&package-manager=go_modules&previous-version=0.10.0&new-version=0.11.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

You can trigger a rebase of this PR by commenting `@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


</details>
2022-12-22 09:46:37 +00:00
Vaibhav Kamra
3141b181b6
Address review feedback from #1896 (#1913)
## Description

- Adds accessor methods for size/modified
- Do not initialize folder in test with a modified time

## Does this PR need a docs update or release note?

- [x]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No 

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [x] 🐹 Trivial/Minor

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-12-22 06:13:30 +00:00
Keepers
8fa440e277
populate sharepoint selectors with scopes (#1910)
## Description

On backup create, sharepoint selectors also
need scopes, or else they do nothing.

## Does this PR need a docs update or release note?

- [x]  No 

## Type of change

- [x] 🐛 Bugfix

## Issue(s)

* #1908

## Test Plan

- [x] 💪 Manual
- [x]  Unit test
2022-12-22 01:25:00 +00:00
ashmrtn
d3573d12dc
Allow updating portions of ItemInfo for some types (#1883)
## Description

Add helper functions that allows updating ItemInfo given a new path.Path. This is required because some items embed portions of the path in their details.

## Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No 

## Type of change

- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

* #1800 

## Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-12-22 00:09:40 +00:00
ashmrtn
df086bd6f4
Move OneDrive path helper func to path package (#1880)
## Description

Later code in details package will need access to this function so it can update item details. However, importing the onedrive package in the details package leads to an import cycle. Moving it to the path package breaks the cycle while still allowing the code to be accessed.

## Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No 

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [x] 🐹 Trivial/Minor

## Issue(s)

* #1800 

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-12-21 23:32:47 +00:00
Keepers
5b6b60de60
enable incrementals with feature flags (#1903)
## Description

Allows the usage of incrementals via feature-
flag controls.  Feature flags can be enabled
on a per-flag basis in either the cli (the flag
is hidden from users, normally) or through the
sdk by directly toggling the flag property
in the control.Options.

## Does this PR need a docs update or release note?

- [x]  No 

## Type of change

- [x] 🌻 Feature

## Issue(s)

* #1901

## Test Plan

- [x] 💪 Manual
2022-12-21 21:31:48 +00:00
Vaibhav Kamra
a45aeda4a2
Update folder size and modified time in details (#1881)
## Description

Caches folder info added during details construction in the details builder and keeps the size/modified time
updated as newer items are added.

As part of this, this PR refactors the details package to separate out building `details.Details` from the in-memory representation and model.

## Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [x] 🕐 Yes, but in a later PR
- [ ]  No 

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #1850 

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-12-21 19:17:24 +00:00
Danny
b59e5849ab
GC: Sharepoint selector added to the helper set. (#1900)
## Description
Adds SharePoint service to the GC helper set. 
<!-- Insert PR description-->

## Does this PR need a docs update or release note?
- [x]  No 

## Type of change

- [x] 🤖 Test

## Test Plan

- [x]  Unit test
2022-12-21 18:55:38 +00:00
Keepers
752ff20c6b
require resource owner set on selector create (#1887)
## Description

selector creation now includes a parameter for
a slice of resource owners (users or sites).  This
is step one in migrating resource owner lists out
of scopes and into the selector.  next step is to
have the selector utilize the primary list instead
of the per-scope list.

## Does this PR need a docs update or release note?

- [x]  No 

## Type of change

- [x] 🌻 Feature

## Issue(s)

* #1617

## Test Plan

- [x]  Unit test
- [x] 💚 E2E
2022-12-21 18:38:41 +00:00
Keepers
a13f1cc3a6
remove common maps, add golang maps (#1885)
## Description

replaces the common/maps.go code with the
golang experimental maps package. The
maps package provides standard api
such as Keys, Values, Copy, and Clone.

https://pkg.go.dev/golang.org/x/exp/maps

## Does this PR need a docs update or release note?

- [x]  No 

## Type of change

- [x] 🐹 Trivial/Minor

## Test Plan

- [x]  Unit test
2022-12-21 18:08:26 +00:00
dependabot[bot]
e31267b579
⬆️ Bump github.com/aws/aws-sdk-go from 1.44.163 to 1.44.164 in /src (#1898)
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.44.163 to 1.44.164.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/aws/aws-sdk-go/releases">github.com/aws/aws-sdk-go's releases</a>.</em></p>
<blockquote>
<h1>Release v1.44.164 (2022-12-20)</h1>
<h3>Service Client Updates</h3>
<ul>
<li><code>service/batch</code>: Updates service API and documentation
<ul>
<li>Adds isCancelled and isTerminated to DescribeJobs response.</li>
</ul>
</li>
<li><code>service/ec2</code>: Updates service API, documentation, and paginators
<ul>
<li>Adds support for pagination in the EC2 DescribeImages API.</li>
</ul>
</li>
<li><code>service/lookoutequipment</code>: Updates service API and documentation</li>
<li><code>service/medialive</code>: Updates service API and documentation
<ul>
<li>This release adds support for two new features to AWS Elemental MediaLive. First, you can now burn-in timecodes to your MediaLive outputs. Second, we now now support the ability to decode Dolby E audio when it comes in on an input.</li>
</ul>
</li>
<li><code>service/nimble</code>: Updates service API, documentation, and paginators</li>
<li><code>service/resource-explorer-2</code>: Updates service documentation</li>
<li><code>service/route53domains</code>: Updates service API and documentation
<ul>
<li>Use Route 53 domain APIs to change owner, create/delete DS record, modify IPS tag, resend authorization. New: AssociateDelegationSignerToDomain, DisassociateDelegationSignerFromDomain, PushDomain, ResendOperationAuthorization. Updated: UpdateDomainContact, ListOperations, CheckDomainTransferability.</li>
</ul>
</li>
<li><code>service/sagemaker</code>: Updates service API and documentation
<ul>
<li>Amazon SageMaker Autopilot adds support for new objective metrics in CreateAutoMLJob API.</li>
</ul>
</li>
<li><code>service/transcribe</code>: Updates service API</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="bda57440af"><code>bda5744</code></a> Release v1.44.164 (2022-12-20) (<a href="https://github-redirect.dependabot.com/aws/aws-sdk-go/issues/4668">#4668</a>)</li>
<li>See full diff in <a href="https://github.com/aws/aws-sdk-go/compare/v1.44.163...v1.44.164">compare view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=github.com/aws/aws-sdk-go&package-manager=go_modules&previous-version=1.44.163&new-version=1.44.164)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

You can trigger a rebase of this PR by commenting `@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


</details>
2022-12-21 17:32:20 +00:00
dependabot[bot]
358f45d229
⬆️ Bump sass from 1.57.0 to 1.57.1 in /website (#1897) 2022-12-21 05:14:10 +00:00
Keepers
ce7fb30a7f
move exchange data collections to exchange (#1818)
## Description

moves the DataCollections producer out of
collections and into exchange, along with the
integration tests. The only changes are the
code shuffles, passing down required values,
and the unexporting of funcs that were only
exported for the old design.

## Does this PR need a docs update or release note?

- [x]  No 

## Type of change

- [x] 🐹 Trivial/Minor

## Issue(s)

* #1727

## Test Plan

- [x]  Unit test
- [x] 💚 E2E
2022-12-20 22:46:06 +00:00
Keepers
f202007843
debug logs only if testing with -v (#1870)
## Description

Log tests at info level normally.  If debug logs
are wanted, the tester can call -v.

## Type of change

- [x] 🐹 Trivial/Minor
2022-12-20 22:23:28 +00:00
Danny
fe27fea2ae
GC: Backup: SharePoint List Integration with Tests (#1788)
## Description
Framework for the SharePoint backup workflow. 

### Special Instructions for Review
Ensure that the information used to build `path.Path` are in line with future PRs (@ashmrtn )
## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature

## Issue(s)

* closes #1403
* closes #1474<issue>
* closes #1795

## Test Plan
- SharePoint Integration checked:
  - `src/internal/connector/data_collections_test.go`
  - `TestSharePointDataCollection()`
- SharePoint List basics checked:  
  - `src/internal/connector/sharepoint/collection_test.go`
- SharePoint Operational Backup:
  - `src/internal/operations/backup_test.go`
  - `TestBackup_Run_sharePoint() ` verified as operational during testing
- [x]  Unit test
2022-12-20 21:30:20 +00:00
Niraj Tolia
d383a2d568
Create a security policy (#1879)
## Description

This helps with meeting community standards too.

## Does this PR need a docs update or release note?

- [x]  No 

## Type of change

- [x] 🗺️ Documentation
2022-12-20 19:58:11 +00:00
Niraj Tolia
0591ca1ba2
Add info on getting logs to docs and the issue template (#1876)
## Description

Make it clear what logs we want

## Does this PR need a docs update or release note?

- [x]  Yes, it's included

## Type of change

- [x] 🗺️ Documentation
2022-12-20 19:09:19 +00:00
Niraj Tolia
ee166903e4
Fix typo in bug report template (#1875)
## Description

s/addition/additional

## Does this PR need a docs update or release note?

- [x]  No 

## Type of change

- [x] 🐛 Bugfix
2022-12-20 17:39:23 +00:00
Niraj Tolia
61f4e97f27
Add GitHub form for bug reports (#1871)
## Description

This asks the users for bug reports, Corso and OS versions, and logs.

## Does this PR need a docs update or release note?

- [x]  No
2022-12-20 16:47:53 +00:00
Keepers
569b5c44b2
log non-2xx graph api calls (#1860)
## Description

Adds logging for non-2xx http communication
within the graph API client.  Logging req/resp
metadata is turned off by default, and requires
either a cli flag or os env var to enable.

## Does this PR need a docs update or release note?

- [x]  No 

## Type of change

- [x] 🤖 Test

## Issue(s)

* #1858

## Test Plan

- [x] 💚 E2E
2022-12-20 16:47:03 +00:00