314 Commits

Author SHA1 Message Date
Keepers
64219c3ae7
track item size in onedrive restores (#1159)
## Description

Hacks in the bytes written as the iteminfo size during onedrive item restores.  Would use the drive itemable, but the size property isn't instantiated locally during creation, and cannot be mutated afterwards.

## Type of change

- [x] 🐛 Bugfix

## Issue(s)

* #1113

## Test Plan

- [x] 💪 Manual
2022-10-13 01:13:35 +00:00
ashmrtn
41319c117f
Allow selective mail backup and change how mail is fetched from Graph (#1123)
* Move interfaces to common location

Upcoming PRs are using these interfaces across packages. Move them to a
common location so multiple packages can use them without import cycles
etc.

* Allow adding newly created folders to the cache (#1131)

* New function to add folders to cache

Allow adding new folders to the cache. Automatically cache the paths for
the new folders. Also add the new function to the interface.

* Reuse the AddToCache function during population

* Wire up ability to back up a single subfolder of mail (#1132)

* Expand cache to return items in it

Required to allow matching an item's path to a selector as the selector
will not provide which paths it matches on easily.

* Function to get collections from cached folders

Returned collections match any matchers given for the folders

* Thread resolver through iterator functions

Allow the folder resolver to be used in all iterator functions. The
resolver will be tied to the current category and user.

* Choose between using resolver and making queries

Allow either using the resolver to get folders with matching names or
using queries to get them.

* Wire up resolver at entry point

Create a resolver instance for each user/category of data being
backedup.

* Preparation for changing how mail enumeration is done (#1157)

* Step towards redoing mail fetching

Pull out old way to get data into a new function and setup some helper
functions etc.

* Switch to pulling mail items folder by folder (#1158)

* Function to pull mail items given collections

Given a set of collections and IDs for those collections pull the mail
items for each collection.

* Create helper function to fetch mail

New helper function to fetch mail items. This goes through each folder
and gets the items for them individually.

* Wire up new way to fetch mail

Leaves fetch logic for other data types undisturbed.

* Tests for new mail fetching logic

Remove tests that were previously in iterators_test.go and move them to
graph_connector_test.go. These tests only had to do with mail logic.
Tests that handled all data types in iterators_test.go have been updated
to skip mail now.
2022-10-12 16:44:24 -07:00
ashmrtn
f4befaf849
Allow purge command to delete OneDrive folders (#1094)
* Add functions for OneDrive purge

function is setup to return and take purgable instead of just userID,
itemID for deleter.

* Adjust other functions to take purgable

Adjust interface for deleter so that it allows more information to the
delete function. The called function can cast the purgable it was passed
back to the type that was returned by the getter function.

* Move struct definition

Per reviewer comment move struct definition to OneDrive package instead
of leaving in purge command.

* Fix lint error

Likely slipped through while GitHub actions was having issues.
2022-10-10 18:16:46 -07:00
Keepers
2cc9b7bc1d
add progress display handler and api (#1116)
## Description

Adds a new package- Observe- for owning user-
oriented displays like progress bars.  This PR adds
an initial progress bar to onedrive backups as a
proof-of-concept.  The API is more important than
the specific progress bar package at this time.
Future changes may opt for a different pkg.

Display format currently looks like:
```
59% [=============>           ] (6.9/12 kB, 14 MB/s)  |  Item_Name.txt
```

Known Issues:
* the `progressbar` package does not support multiline output, and [the author is not planning to add support](https://github.com/schollz/progressbar/issues/6).  This causes concurrent items to overwrite each other.  We will either need to fork the library, or change to a different one.  

## Type of change

- [x] 🌻 Feature

## Issue(s)

* #1112

## Test Plan

- [x] 💪 Manual
- [x]  Unit test
2022-10-10 22:53:48 +00:00
Keepers
0b65b20200
attach logger to test ctx, use flusher (#1083)
## Description

Replace all cases of context.Background() and
context.TODO() with the testing ctx, which seeds
a logging instance into the context.  The seed is
less important than the flush action, since errant log.Ctx(ctx) calls will automatically use the log
singleton.  But in order for the logs to show up
within the test, they need to get flushed.

See my comments for focus on non-chaff changes.

## Type of change

- [x] 🤖 Test

## Issue(s)

* #902

## Test Plan

- [x] 💪 Manual
- [x]  Unit test
2022-10-10 21:14:10 +00:00
Danny
211c248f35
GC: Iterative Function: Bug fix (#1110)
## Description
Contact Iterative sequence tests changed to reflect `exchange.Backup` functions used within `SetupExchangeCollectionVars()`. The previous bug was caused by misalignment from the old test configuration and the current implementation.
<!-- Insert PR description-->

## Type of change

- [x] 🐛 Bugfix


## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* closes  #1107<issue>

## Test Plan

- [x]  Unit test
2022-10-10 18:22:31 +00:00
ashmrtn
dd7710ca5b
OneDrive get and delete folder functions (#1093)
## Description

Useful helper functions for the purge script

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

* #1090 

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-10-08 01:34:15 +00:00
Danny
a5140d20b2
GC: Backup: CI: Stabilize Iterator test (#1079)
## Description
CI test timing during times of high activity. Reduced test suite to cover Inbox and adjusted outputs accordingly. 
<!-- Insert PR description-->

## Type of change

- [x] 🐛 Bugfix

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
*closes  #1078<issue>

## Test Plan

- [x]  Unit test
2022-10-07 18:55:17 +00:00
Danny
0724db938c
GC: CI: Streamline Tests and Reduction (#1082)
* Reduction of duplicated tests.  Removal of tests that are checked in other functions.
2022-10-07 13:01:37 -04:00
Keepers
89668ed164
describe time value standards, validate time flags (#1066)
## Description

Adds validation of cli inputs for time-based flags.
All time values that can be parsed by the time
handling in common count as supported, even
though official support includes a smaller set.

Also adds some clean-up to time.go and adds a
design document describing standard time value
formats, and the maintenance thereof, in corso.

## Type of change

- [x] 🌻 Feature
- [x] 🗺️ Documentation
- [x] 🤖 Test

## Issue(s)

* #943
* #924

## Test Plan

- [x]  Unit test
- [x] 💚 E2E
2022-10-06 17:46:21 +00:00
Keepers
30f35709be
purge restored load test folders (#1059)
## Description

purge restored folders for all users in the tenant following a load test to ensure we don't over-
produce data that causes future tests to time out.

## Type of change

- [x] 🤖 Test

## Issue(s)

* #1048

## Test Plan

- [x] 💪 Manual
- [x] 💚 E2E
2022-10-06 01:20:17 +00:00
Keepers
78632f56e7
remove --all flag support, minor cleanup (#1047)
## Description

Removes the -all flag from the exchange cli.  The
same functionality can be gained with the flag
--user *.

Additionally does some minor tidying of other
comments and outputs throughout the cli.

## Type of change

- [x] 🐹 Trivial/Minor

## Issue(s)

* #1032

## Test Plan

- [x]  Unit test
2022-10-06 00:10:56 +00:00
ashmrtn
4ba8f50c53
Restore/backup test for email attachments (#1057)
## Description

Test to check that attachments are properly populated and fetched during restore and backup

Some notes:
* make slice element comparison function generic so it can compare locations and attachments
* very hacky comparison for attachment content using reflection as it's not easily accessible otherwise. Changes to the underlying struct will cause runtime failures in the test case if/when they occur

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

* #913 

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-10-05 23:11:54 +00:00
Danny
7843787cd3
GC: Restore: Mail: Create Subfolder Feature (#1055)
## Description
The feature allows for the subFolders to be created. Test Suite Expanded. 
<!-- Insert PR description-->

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
*Closes Issue #1054<issue>

## Test Plan

- [x]  Unit test
2022-10-05 22:33:41 +00:00
Keepers
f4556288a5
add iteminfo to restore details (#1041)
## Description

Extends the restore details with itemInfo from restored items.

## Type of change

- [x] 🌻 Feature

## Issue(s)

* #977

## Test Plan

- [x] 💪 Manual
- [x] 💚 E2E
2022-10-05 21:19:21 +00:00
Danny
5bcdaef769
GC: Interface: Cache Refactor (#1043)
## Description
Code changed to support caching on the default folder of `exchange.Mail` as well as an independent node within the Inbox.
<!-- Insert PR description-->

## Type of change

- [x] 🌻 Feature

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* related to Issue #1004<issue>

## Test Plan
- [x]  Unit test
2022-10-05 20:04:50 +00:00
ashmrtn
3f0951dea2
Higher resolution on test restore folder names (#1040)
## Description

\~Only affects folders created by tests\~
Create and use helper function that sets the restore destination folders to have microsecond granularity. This is to avoid naming collisions with second granularity (of which I've observed at least once)

Switches all tests for all services/categories to use OneDrive DateTime format+microseconds

Manually verified that a manually created folder with the given timestamp format was removed by the CI purge script

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

* closes #1036

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-10-04 23:52:13 +00:00
ashmrtn
116d6e9df6
Multi folder backup of contacts and events (#1016)
## Description

Test ability to backup multiple disjoint folders in contacts and events

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

* #913

merge after
* #995 

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-10-04 21:18:04 +00:00
Keepers
d0560500d2
add deets to gc restore ops (#1029)
## Description

In order for a restore operation to accurately print
out the list of items that were restored, it needs to
track item details within the GC restoration like is
done within kopia in backup.   These details will
not be stored in a modelStore.  But they will get
printed out on the CLI for end users.

Part 1 of multiple.

## Type of change

- [x] 🌻 Feature

## Issue(s)

* #977

## Test Plan

- [x] 💪 Manual
- [x] 💚 E2E
2022-10-04 17:56:34 +00:00
Vaibhav Kamra
3df3d68a3e
OneDrive details UX improvements (#1034)
## Description

- Rename `LastModified` to `Modified` in OneDrive backup details
- Humanize file size units

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [x] 🐹 Trivial/Minor

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #627 

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-10-04 15:44:47 +00:00
Danny
c6a9d2feb6
GC Status Progress Detail Extension (#1001)
## Description
Feature to show the number of bytes and additional details for backup / restore progress from GraphConncector
Full description [here](https://www.notion.so/alcion/Corso-Testing-Notes-b9867ac719d8459d8b46dbc7b07b33e0#da3869278b434b0398e7d245554b609b)
<!-- Insert PR description-->

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* closes #559<issue>

## Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual

Feature Output changed to:

```go
graph_connector_test.go:137: Action: Backup performed on 5 of 5 objects (227.63KB) within 1 directories. Downloaded from Inbox
        
graph_connector_test.go:137: Action: Backup performed on 8 of 8 objects (231.28KB) within 2 directories. Downloaded from Inbox, Contacts

graph_connector_test.go:137: Action: Backup performed on 23 of 23 objects (309.36KB) within 3 directories. Downloaded from Inbox, Contacts, Calendar
```
2022-10-03 23:25:26 +00:00
Danny
d60a771024
GC: TWO Message attachments (#1014)
## Description
Introduces mock testing to ensure that objects with two or more attachments are restored properly.

Minor change to restore pipeline for failure. attachments are restored separately in the event that one of the attachments is corrupted. 

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [x] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* closes #1007<issue>

## Test Plan

- [x]  Unit test
2022-10-03 20:59:03 +00:00
Keepers
7fa86824e9
Issue 1020 (#1021)
## Description

CI tests are showing a consistent panic when adding attendees
to events with zero content.

## Type of change

- [x] 🐛 Bugfix

## Issue(s)

* #1020

## Test Plan

- [x]  Unit test
- [x] 💚 E2E
2022-10-03 17:17:32 +00:00
Vaibhav Kamra
03bb63f52d
Use selectors in OneDrive CLI (#996)
## Description

Adds the following selectors to OneDrive details/restore :
- `file-name`, `folder`, `file-created-after`, `file-created-before`, `file-modified-after`, `file-modified-before`

Also includes a change where we remove the `drive/<driveID>/root:` prefix from parent path entries in details. This
is to improve readability. We will add drive back as a separate item in details if needed later.

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

* #627 

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-10-03 07:23:30 +00:00
ashmrtn
88af7f9b7c
restore/backup tests for events (#995)
## Description

Test restoring a single calendar and multiple calendars of events

Manually compare Location models because they have some fields that are marked as for internal use and appear to be overwritten/controlled by M365

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

* #913 

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-09-30 23:39:33 +00:00
Danny
a31a17ce1f
GC: Extend mockable to include attachments for mail (#990)
## Description
Mockable `exchange.Mail.Messages` extended to include direct attachments and oneDrive attachments. 

NOTE: There are changes to support as it was recognized that the lack of these variables during cloning caused Issue #978 for direct attachments. However, it does not solve it for OneDrive. 

<!-- Insert PR description-->

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [x] 🐛 Bugfix


## Issue(s)


**closes** #988<issue>

## Test Plan

- [x]  Unit test
2022-09-30 14:08:17 +00:00
ashmrtn
5da8410ebf
Test backing up multiple top-level folders in mail (#993)
## Description

Use multiple restores to populate the account with known data and then run a backup over the restores to pull the data.

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

* #913 

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-09-30 01:00:10 +00:00
ashmrtn
0cba7a402c
Restore tests for contacts (#989)
## Description

Test restoring a single contacts folder and multiple contacts folders. This tests that items from different folders can be restored, but not that items from multiple folders can be backed up (because the restore colocates into a single folder)

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

* #913 

merge after
* #986 

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-09-30 00:32:28 +00:00
ashmrtn
4e86cd2f84
Fix bug fetching contact folders with filters (#986)
## Description

Stop fetching all folders for every new folder seen, erasing previous results. Regression test to come in a later patch as it requires some extra infrastructure

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

* closes #982 

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-09-29 23:20:50 +00:00
Danny
114fec6059
GC: Restore: Simplified Event Restore (#981)
## Description
Implementation of simplified Restore based on the following [spec](https://www.notion.so/alcion/Event-restore-semantics-061aee5288244629b1c53337e4dea306#6e974540c8804c4fa832218675534e1c)

<!-- Insert PR description-->

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature


## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
*closes  #954<issue>

## Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
2022-09-29 22:01:12 +00:00
ashmrtn
23e1db13df
Minor revisions to comparisons in restore/backup tests (#987)
## Description

* treat empty values and nil pointers as equal when comparing Graph models
* make sure all items are always iterated through so the test doesn't hang
* explicitly check number of items retrieved
* remove some unneeded casts and assignments

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

* #913 

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-09-29 17:26:13 +00:00
Keepers
d4390ac5ea
add diagnostics to load_test (#983)
## Description

Adds some go runtime diagnostics tracking to
load testing, including some trace regioning.
Unfortunately, I couldn't find any third party trace library that didn't depend on a sidecar server
to sample against the application on.  Therefore,
just starting with something basic.

## Type of change

- [x] 🤖 Test

## Issue(s)

* #902

## Test Plan

- [x] 💪 Manual
- [x] 💚 E2E
2022-09-29 16:47:09 +00:00
ashmrtn
f0ff7ee982
Basic checks for bad/empty values for restore (#973)
## Description

Test bad services or empty collections don't cause an error

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

* #913 

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-09-28 22:41:50 +00:00
ashmrtn
3c0179986b
Restore and backup tests for GraphConnector (#980)
## Description

Add framework and tests for some exchange mail restore and backup situations. Framework can be used to test other situations in the future, this is just the starting point.

Some logic in the test can be further generalized/factored out once we know more about how paths will be transformed during restore

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

* #913 

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-09-28 20:23:54 +00:00
Danny
652bb9cd4a
GC: Restore: Contact: Restore to single Folder (#972)
## Description

The feature ensures that this Flat style directory only creates a single folder and restores all the contacts to the same folder during the restore process. 

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
*closes #880<issue>

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2022-09-28 14:16:08 +00:00
ashmrtn
507eab088d
Implement Size() in mock connector (#974)
## Description

Adds the ability to use mockconnector for onedrive and exchange restores, reducing potential code duplication overall.

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

* #913 

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-09-28 01:38:10 +00:00
ashmrtn
0a629e0807
Wire up RestoreDestination (#949)
## Description

* No new functionality exposed to CLI users
* generate restore folder names in CLI and pass down the stack
* update tests for new parameters
* centralize generation of restore container names

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

* #897 
* #913 

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2022-09-27 16:06:15 +00:00
Keepers
ed2dd9a026
scope integration tests to specific folders (#806)
## Description

Exchange app e2e tests need to scope their
backup dataset to only the primary folder, else
we risk exponential duplication of data when
the next backup picks up all the corso-copied
folders.

## Type of change

- [x] 🌻 Feature
- [x] 🐛 Bugfix

## Issue(s)

* #501
### Awaiting completion 
- #810 
- #811
## Test Plan

- [ ] 💪 Manual
- [ ]  Unit test
- [x] 💚 E2E
2022-09-27 14:09:10 +00:00
Danny
b57829cc05
MockData: Mail Updated w/ Current Escape Policy (#959)
Mockdata exchanged for a newer version Kiota serialized version.

## Description

<!-- Insert PR description-->

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🐹 Trivial/Minor

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* closes #956<issue>

## Test Plan

- [x]  Unit test
2022-09-27 01:20:05 +00:00
Vaibhav Kamra
7d1cf2ce5b
Use path based addressing to get OneDrive folder (#951)
## Description

Addresses a follow-up from the folder lookup implementation where we were not iterating through
all children (no pagination was implemented) when searching for a folder.

Instead of implementing pagination, this uses OneDrive path based addressing to do a direct lookup
for the folder.

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

* #875 

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2022-09-26 23:53:22 +00:00
Vaibhav Kamra
1df48997ae
Add additional OneDrive metadata to backup details (#952)
## Description

Adds file size and timestamps to enable selectors using this metadata

## Type of change

- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

* #594 

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2022-09-26 22:54:42 +00:00
Vaibhav Kamra
ea73873ffb
Fix OneDrive restore regressions (#953)
## Description

Fixes a couple of regressions that got introduced on the restore path via code review changes and
kopia wrapper refactor.

Found while writing the OneDrive restore test. Would have been prevented if we had said test.

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-09-26 22:20:54 +00:00
ashmrtn
ff2db0c553
Export path package to other codebases (#912)
## Description

Moves the `path` package to the `pkg` package so other code outside of Corso can use it if they need it

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [x] 🐹 Trivial/Minor

## Issue(s)

* closes #908 

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2022-09-26 18:34:19 +00:00
Danny
06d2a389d9
GC: Backup: Events: Filter Calendars with Scope (#900)
## Description
Feature branch enabling filtering based upon `selectors.ExchangeScope`
<!-- Insert PR description-->

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
*closes #810<issue>

## Test Plan
- [x]  Unit test
2022-09-26 17:33:33 +00:00
Danny
f39a14eda3
GC: const variables extended to include default container names (#886)
## Description
Moves constants to a file and adds more source definitions and backgrounds to constants.
<!-- Insert PR description-->

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🐹 Trivial/Minor

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
*closes #823<issue>

## Test Plan
Const variables to be used in future PRs. No tests are available. 
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2022-09-23 14:17:59 +00:00
Danny
3f3c8217a3
GC: Component: Version 40.0 (#917)
## Description
Upgrades Graph API to the newest version. Breaking change from the previous version, all functionality needs to be inspected to ensure operation.  
<!-- Insert PR description-->

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature


## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
*closes #828<issue>

## Test Plan

- [x]  Unit test
- [x] 💚 E2E
2022-09-22 18:59:51 +00:00
ashmrtn
7850272a5e
Apply wsl linter to onedrive package (#911)
## Description

Finish enabling wsl linter on Corso repo by removing the exception for onedrive package

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [x] 🐹 Trivial/Minor

## Issue(s)

* closes #481

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2022-09-20 20:02:35 +00:00
ashmrtn
acf1837e77
909 path folders (#910)
## Description

Adds function call to the path package that returns the folder elements present in a path after the path prefix. This is safer than using `strings.Split()` because some paths may have `/` characters in elements.

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

* closes #909 

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-09-20 17:15:46 +00:00
Vaibhav Kamra
921d77cc05
Implement OneDrive DataCollection restore (#877)
## Description

Implements `data.Collection` restore for OneDrive and wires it up to the restore operation

Leverages previously added `onedrive` helpers that have integration tests.

An op/CLI level integration test will be added in a follow-up PR.

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

* #668 

## Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E

```
$ ./corso restore onedrive --backup 7de4d68f-67c2-4e81-95a2-81cf0d1bd11d
Restored OneDrive in S3 for user
```
2022-09-19 21:46:30 +00:00
Danny
f250665dd6
GC: Backup: Contacts: Ability to filter on folder [FEATURE] (#891)
## Description
PR enables Filtering for `exchange.Contact` items wherein if an `exchange.ContactFolder` is requested specifically, only the requested folders are backed up. 
<!-- Insert PR description-->

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature


## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
*closes  #811<issue>

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-09-19 17:42:12 +00:00