454 Commits

Author SHA1 Message Date
Georgi Matev
3a1eb1efd2
Add Examples section to CLI usage and docs (#1050)
## Description

Adds Examples to CLI usage output and the auto-gen docs

Also cleans up:
* OneDrive flags and flags help
* Addition of OneDrive generated files to docs 


## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [x] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* Fixes #529  

## Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2022-10-05 07:06:32 +00:00
ashmrtn
3f0951dea2
Higher resolution on test restore folder names (#1040)
## Description

\~Only affects folders created by tests\~
Create and use helper function that sets the restore destination folders to have microsecond granularity. This is to avoid naming collisions with second granularity (of which I've observed at least once)

Switches all tests for all services/categories to use OneDrive DateTime format+microseconds

Manually verified that a manually created folder with the given timestamp format was removed by the CI purge script

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

* closes #1036

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-10-04 23:52:13 +00:00
Keepers
630d74bee7
add NoData operation status (#1042)
## Description

Adds a NoData status to operations in the event
that a backup (or, possibly in the future, restore)
finishes processing without having any items to
store or restore.

## Type of change

- [x] 🌻 Feature

## Issue(s)

* #1000

## Test Plan

- [x] 💪 Manual
- [x]  Unit test
2022-10-04 23:08:17 +00:00
ashmrtn
edd84e690d
Version data stored in kopia (#1028)
## Description

As there's not a 1:1 relationship between backup details items and data stored in kopia as files, version all the data stored in kopia files. This will allow us to change the serialized format of this data in the future independently of the version in the backup details models. This is also less error prone than associating the version in the models with the layout of the kopia file data

See this [comment](https://github.com/alcionai/corso/issues/284#issuecomment-1255346379) for more context on why kopia layout versioning is useful and a sketch of the implemented solution

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

* #284 

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-10-04 22:00:03 +00:00
ashmrtn
116d6e9df6
Multi folder backup of contacts and events (#1016)
## Description

Test ability to backup multiple disjoint folders in contacts and events

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

* #913

merge after
* #995 

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-10-04 21:18:04 +00:00
Keepers
3719c36bce
print restore details following successful op (#1030)
## Description

Now that GC is tracking the details entries during
a restore procedure, it can display the results in
the cli.  Due to an absence of itemInfo, only the
item shortRef is displayed.  But we can expand
on that from here.

## Type of change

- [x] 🌻 Feature

## Issue(s)

* #977

## Test Plan

- [x] 💪 Manual
- [x] 💚 E2E
2022-10-04 20:43:38 +00:00
ashmrtn
f8a6d33466
Add test for idempotent repo creation (#1037)
## Description

Regression test to ensure that hidden --succeed-if-exists flag works.

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

* #913 

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-10-04 19:41:41 +00:00
Keepers
67215c9bf1
field NotFound err in kopia, handle in cli (#1027)
## Description

Replace the `manifest.NotFound` error production
in kopia wrapper with a local version of the error. This allows the cli to catch the corso error and
write a clearer response to the end user.

## Type of change

- [x] 🌻 Feature

## Issue(s)

* #976 

## Test Plan

- [x] 💪 Manual
- [x] 💚 E2E
2022-10-04 18:27:23 +00:00
Keepers
d0560500d2
add deets to gc restore ops (#1029)
## Description

In order for a restore operation to accurately print
out the list of items that were restored, it needs to
track item details within the GC restoration like is
done within kopia in backup.   These details will
not be stored in a modelStore.  But they will get
printed out on the CLI for end users.

Part 1 of multiple.

## Type of change

- [x] 🌻 Feature

## Issue(s)

* #977

## Test Plan

- [x] 💪 Manual
- [x] 💚 E2E
2022-10-04 17:56:34 +00:00
ashmrtn
dd34ecd5f7
Test CLI init/connect and without s3 prefix on bucket (#1033)
## Description

Regression test for bucket prefix trimming

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

* #913 
* #975 

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-10-04 16:45:09 +00:00
Vaibhav Kamra
3df3d68a3e
OneDrive details UX improvements (#1034)
## Description

- Rename `LastModified` to `Modified` in OneDrive backup details
- Humanize file size units

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [x] 🐹 Trivial/Minor

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #627 

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-10-04 15:44:47 +00:00
Danny
c6a9d2feb6
GC Status Progress Detail Extension (#1001)
## Description
Feature to show the number of bytes and additional details for backup / restore progress from GraphConncector
Full description [here](https://www.notion.so/alcion/Corso-Testing-Notes-b9867ac719d8459d8b46dbc7b07b33e0#da3869278b434b0398e7d245554b609b)
<!-- Insert PR description-->

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* closes #559<issue>

## Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual

Feature Output changed to:

```go
graph_connector_test.go:137: Action: Backup performed on 5 of 5 objects (227.63KB) within 1 directories. Downloaded from Inbox
        
graph_connector_test.go:137: Action: Backup performed on 8 of 8 objects (231.28KB) within 2 directories. Downloaded from Inbox, Contacts

graph_connector_test.go:137: Action: Backup performed on 23 of 23 objects (309.36KB) within 3 directories. Downloaded from Inbox, Contacts, Calendar
```
2022-10-03 23:25:26 +00:00
Keepers
fe148af455
normalize bucket naming during config lookup (#1025)
## Description

Ensure s3 bucket name inputs are normalized
in all uses.  Currently, the normalized format is
applied when using storage, but not when passing
around --bucket flag values elsewhere.

## Type of change

- [x] 🐛 Bugfix

## Issue(s)

* #975

## Test Plan

- [x] 💪 Manual
- [x]  Unit test
2022-10-03 21:37:20 +00:00
Danny
d60a771024
GC: TWO Message attachments (#1014)
## Description
Introduces mock testing to ensure that objects with two or more attachments are restored properly.

Minor change to restore pipeline for failure. attachments are restored separately in the event that one of the attachments is corrupted. 

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [x] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* closes #1007<issue>

## Test Plan

- [x]  Unit test
2022-10-03 20:59:03 +00:00
Georgi Matev
81a259cb32
Clean up CLI flags, add extended usage, enhance docs pages (#1022)
## Description

* Clean up pass on all CLI output
* Added basic extended usage
* Mark required params in generated docs

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [x] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* fixes #527  

## Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2022-10-03 19:35:57 +00:00
Keepers
11be4e4590
up the frequency of load tests (#1024)
## Description

Temporarily up the frequency of the load test kickoff for observation.

## Type of change

- [x] 🤖 Test

## Issue(s)

* #902

## Test Plan

- [x] 💚 E2E
2022-10-03 18:59:36 +00:00
ashmrtn
56287c42fa
Skip kopia upload when no collections given (#846)
## Description

<!-- Insert PR description-->
Currently does not throw an error. Open to other suggestions on return values.

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* closes #844 
* #1000 

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-10-03 17:49:20 +00:00
Keepers
7fa86824e9
Issue 1020 (#1021)
## Description

CI tests are showing a consistent panic when adding attendees
to events with zero content.

## Type of change

- [x] 🐛 Bugfix

## Issue(s)

* #1020

## Test Plan

- [x]  Unit test
- [x] 💚 E2E
2022-10-03 17:17:32 +00:00
Vaibhav Kamra
03bb63f52d
Use selectors in OneDrive CLI (#996)
## Description

Adds the following selectors to OneDrive details/restore :
- `file-name`, `folder`, `file-created-after`, `file-created-before`, `file-modified-after`, `file-modified-before`

Also includes a change where we remove the `drive/<driveID>/root:` prefix from parent path entries in details. This
is to improve readability. We will add drive back as a separate item in details if needed later.

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

* #627 

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-10-03 07:23:30 +00:00
ashmrtn
88af7f9b7c
restore/backup tests for events (#995)
## Description

Test restoring a single calendar and multiple calendars of events

Manually compare Location models because they have some fields that are marked as for internal use and appear to be overwritten/controlled by M365

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

* #913 

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-09-30 23:39:33 +00:00
ashmrtn
526cc8fa9b
Add code to disable default scheduling policy (#1010)
## Description

All backups with corso are manual and use data sourced outside of kopia's control. Disable kopia's scheduling policy so we don't accidentally confuse it

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

* closes #1009

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-09-30 23:10:10 +00:00
ashmrtn
b612718723
Update time parser to handle OneDrive format (#1013)
## Description

OneDrive format uses '-' in place of ':'. Purge command needs to be able to clear folders with this format as well.

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

* closes #1012

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-09-30 22:38:43 +00:00
ashmrtn
dd7b84871f
CLI and selectors tests for folder prefix match (#1011)
## Description

Specifying a folder on the CLI should cause the system to do a prefix match on that folder when doing a restore, backup details, or selector reduce operation.

Recently updated behavior, adjusting tests to match new expectations.

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

* #913 

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-09-30 20:52:19 +00:00
Georgi Matev
5e8b67a606
Explicit ordering of flags based on importance and use frequency (#1002)
## Description

Disable automatic flag sorting and explicitly order based on something more sensible. Fixes the order in both inline help output and docs. 

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [x] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* Fixes # 783

## Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2022-09-30 19:01:39 +00:00
Danny
a31a17ce1f
GC: Extend mockable to include attachments for mail (#990)
## Description
Mockable `exchange.Mail.Messages` extended to include direct attachments and oneDrive attachments. 

NOTE: There are changes to support as it was recognized that the lack of these variables during cloning caused Issue #978 for direct attachments. However, it does not solve it for OneDrive. 

<!-- Insert PR description-->

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [x] 🐛 Bugfix


## Issue(s)


**closes** #988<issue>

## Test Plan

- [x]  Unit test
2022-09-30 14:08:17 +00:00
Georgi Matev
fe2516b9ba
Fix all Vale [error] level lint issues and all markdownlint issues (#997)
## Description

* fixes all `markdownlint` issues, includeing autogenerated docs
* fixes [error] level Vale problems 
*
## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [x] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [x] 🐹 Trivial/Minor

## Issue(s)

## Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2022-09-30 07:31:58 +00:00
ashmrtn
5da8410ebf
Test backing up multiple top-level folders in mail (#993)
## Description

Use multiple restores to populate the account with known data and then run a backup over the restores to pull the data.

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

* #913 

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-09-30 01:00:10 +00:00
ashmrtn
0cba7a402c
Restore tests for contacts (#989)
## Description

Test restoring a single contacts folder and multiple contacts folders. This tests that items from different folders can be restored, but not that items from multiple folders can be backed up (because the restore colocates into a single folder)

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

* #913 

merge after
* #986 

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-09-30 00:32:28 +00:00
ashmrtn
1a188ce657
Extra checks for kopia retention policy (#964)
## Description

Add a regression test component to the retention test. This ensures that setting all values to 0 causes it to keep MaxInt latest snapshots.

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

* closes #906

pending merge of
* kopia/kopia#2454

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-09-29 23:55:35 +00:00
ashmrtn
4e86cd2f84
Fix bug fetching contact folders with filters (#986)
## Description

Stop fetching all folders for every new folder seen, erasing previous results. Regression test to come in a later patch as it requires some extra infrastructure

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

* closes #982 

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-09-29 23:20:50 +00:00
Danny
114fec6059
GC: Restore: Simplified Event Restore (#981)
## Description
Implementation of simplified Restore based on the following [spec](https://www.notion.so/alcion/Event-restore-semantics-061aee5288244629b1c53337e4dea306#6e974540c8804c4fa832218675534e1c)

<!-- Insert PR description-->

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature


## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
*closes  #954<issue>

## Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
2022-09-29 22:01:12 +00:00
ashmrtn
23e1db13df
Minor revisions to comparisons in restore/backup tests (#987)
## Description

* treat empty values and nil pointers as equal when comparing Graph models
* make sure all items are always iterated through so the test doesn't hang
* explicitly check number of items retrieved
* remove some unneeded casts and assignments

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

* #913 

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-09-29 17:26:13 +00:00
Keepers
d4390ac5ea
add diagnostics to load_test (#983)
## Description

Adds some go runtime diagnostics tracking to
load testing, including some trace regioning.
Unfortunately, I couldn't find any third party trace library that didn't depend on a sidecar server
to sample against the application on.  Therefore,
just starting with something basic.

## Type of change

- [x] 🤖 Test

## Issue(s)

* #902

## Test Plan

- [x] 💪 Manual
- [x] 💚 E2E
2022-09-29 16:47:09 +00:00
Vaibhav Kamra
b7f5ed73c3
OneDrive Folder and Item selectors (#985)
## Description

Adds support for folder and item selection

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

* #627 

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-09-29 02:44:08 +00:00
Keepers
011e24583f
add profiling, tracing to load testing (#971)
## Description

add diagnostic records to load test runs

## Type of change

- [x] 🤖 Test
- [x] 💻 CI/Deployment

## Issue(s)

* #902

## Test Plan

- [x] 💚 E2E
2022-09-29 02:01:15 +00:00
Vaibhav Kamra
e59d596152
Fix selector path matching (#979)
## Description

Fixes an issue with the path matching logic in selectors where if the path was a substring of what was
specified as the scope in the filter, the item would still be matched.

e.g. 
Given 2 items - `/fold/contact1` and `/folderA/folderB/contact2` and a 
selector `er.Include(er.ContactFolders("AnyUser", []string{"folderA/folderB"}))`, 
the selector would match both items because `fold` is contained in the selector scope `folderA/folderB`

The fix is to invert the comparison - we check if the selector scope is contained in the item path instead.
In the example above, the selector string `folderA/folderB` will then only match the second items path.

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor


## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-09-29 00:06:26 +00:00
ashmrtn
f0ff7ee982
Basic checks for bad/empty values for restore (#973)
## Description

Test bad services or empty collections don't cause an error

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

* #913 

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-09-28 22:41:50 +00:00
ashmrtn
3c0179986b
Restore and backup tests for GraphConnector (#980)
## Description

Add framework and tests for some exchange mail restore and backup situations. Framework can be used to test other situations in the future, this is just the starting point.

Some logic in the test can be further generalized/factored out once we know more about how paths will be transformed during restore

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

* #913 

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-09-28 20:23:54 +00:00
Danny
652bb9cd4a
GC: Restore: Contact: Restore to single Folder (#972)
## Description

The feature ensures that this Flat style directory only creates a single folder and restores all the contacts to the same folder during the restore process. 

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
*closes #880<issue>

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2022-09-28 14:16:08 +00:00
ashmrtn
507eab088d
Implement Size() in mock connector (#974)
## Description

Adds the ability to use mockconnector for onedrive and exchange restores, reducing potential code duplication overall.

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

* #913 

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-09-28 01:38:10 +00:00
ashmrtn
0a629e0807
Wire up RestoreDestination (#949)
## Description

* No new functionality exposed to CLI users
* generate restore folder names in CLI and pass down the stack
* update tests for new parameters
* centralize generation of restore container names

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

* #897 
* #913 

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2022-09-27 16:06:15 +00:00
Keepers
ed2dd9a026
scope integration tests to specific folders (#806)
## Description

Exchange app e2e tests need to scope their
backup dataset to only the primary folder, else
we risk exponential duplication of data when
the next backup picks up all the corso-copied
folders.

## Type of change

- [x] 🌻 Feature
- [x] 🐛 Bugfix

## Issue(s)

* #501
### Awaiting completion 
- #810 
- #811
## Test Plan

- [ ] 💪 Manual
- [ ]  Unit test
- [x] 💚 E2E
2022-09-27 14:09:10 +00:00
Danny
b57829cc05
MockData: Mail Updated w/ Current Escape Policy (#959)
Mockdata exchanged for a newer version Kiota serialized version.

## Description

<!-- Insert PR description-->

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🐹 Trivial/Minor

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* closes #956<issue>

## Test Plan

- [x]  Unit test
2022-09-27 01:20:05 +00:00
Danny
1cc0cf7d1f
Versioning updated for serialization. (#960)
## Description
Abstraction and Seriailiaztion libraries updated
<!-- Insert PR description-->

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature


## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
*Related to #956 <issue>

## Test Plan


- [x]  Unit test
2022-09-27 00:50:17 +00:00
Keepers
e061ce7c73
implement load tests (#937)
## Description

Adds the load test func calls and asserts for
both exchange and onedrive.

## Type of change

- [x] 🤖 Test

## Issue(s)

* #902

## Test Plan

- [x] 💚 E2E
2022-09-27 00:21:30 +00:00
Vaibhav Kamra
7d1cf2ce5b
Use path based addressing to get OneDrive folder (#951)
## Description

Addresses a follow-up from the folder lookup implementation where we were not iterating through
all children (no pagination was implemented) when searching for a folder.

Instead of implementing pagination, this uses OneDrive path based addressing to do a direct lookup
for the folder.

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

* #875 

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2022-09-26 23:53:22 +00:00
Keepers
984df8fcfb
add bup/restore ids for events (#946)
## Description

Start-end event pairs want ids as a means to
match each pair.  Restore operations now gen
a transient id for this purpose.  Backup ops pre-
generate the base ID instead of waiting for the
model store to generate the ID at the end of
the operation.

## Type of change

- [x] 🌻 Feature

## Issue(s)

* #894

## Test Plan

- [x]  Unit test
- [x] 💚 E2E
2022-09-26 23:22:49 +00:00
Vaibhav Kamra
1df48997ae
Add additional OneDrive metadata to backup details (#952)
## Description

Adds file size and timestamps to enable selectors using this metadata

## Type of change

- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

* #594 

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2022-09-26 22:54:42 +00:00
Vaibhav Kamra
ea73873ffb
Fix OneDrive restore regressions (#953)
## Description

Fixes a couple of regressions that got introduced on the restore path via code review changes and
kopia wrapper refactor.

Found while writing the OneDrive restore test. Would have been prevented if we had said test.

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-09-26 22:20:54 +00:00
ashmrtn
7b5c8d43f1
Create POD struct with restore destination info (#947)
## Description

This PR has just the struct definition to allow conversation about the design if needed. Future PR(s) will create and populate the struct during restore (in the CLI-layer since this struct is meant to be used to communicate that information) and pass it down the stack to the graph connector

The roadmap has plans for allowing users to

1. change the "root" folder the restore occurs in
1. change the resource owner of the restored items (e.x. restoring items from one user to a different user)

Both options can be enabled at the same time (e.x. restore "Inbox" from user A to "UserAInboxRestore" in user B). When the folder is not given it should default to "Corso_Restore_\<date time\>" and when the resource owner is not given the item should be restored under the original resource owner. As of now, we don't expect to have to have fine-grained control over restoring multiple users to different folders on a per-user basis. However, we may need to restore to specify a folder that is created for multiple users and restored to.

Examples:
**Not expected:** Restore UserA Inbox to folder1 in their hierarchy and UserB to folder2 in their hierarchy

**Expected:** Restore UserA Inbox to Corso_Restore_2022-09-23
POD struct:
```go
dest := selectors.RestoreDestination{
  ContainerName: "Corso_Restore_2022-09-23",
}
```

**Expected:** Restore UserA Inbox to UserB Corso_Restore_2022-09-23
POD struct:
```go
dest := selectors.RestoreDestination{
  ResourceOwnerOverride: "UserB",
  ContainerName: "Corso_Restore_2022-09-23",
}
```

**Maybe expected:** For UserA and UserB, restore Inbox to folder1 in their separate hierarchies
```go
dest := selectors.RestoreDestination{
  ContainerName: "Corso_Restore_2022-09-23",
}
```

Given the limited nature of the destination requirement, a POD struct seems like the best option for the foreseeable future. If we need more complex functionality down the line we can expand from this

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [x] 🐹 Trivial/Minor

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #897 

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2022-09-26 20:57:04 +00:00