75 Commits

Author SHA1 Message Date
Danny
87014a132b
GC: Backup: Contacts: Retrieve IDs from all Contact Folders (#830)
## Description

Adds the ability to retrieve M365 IDs of contacts not in the default folder. 

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature


## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* closes #804 
* closes #767<issue>
* closes #669

## Test Plan
- [x]  Unit test
2022-09-15 21:38:52 +00:00
Keepers
e92cb1b62b
extend logging support for testing (#850)
## Description

Updating logging to help display/control within testing envs.
Restore error tracking within gc iterators.

## Type of change

- [x] 🐛 Bugfix
- [x] 🤖 Test

## Issue(s)

* #824

## Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2022-09-15 13:18:05 +00:00
Danny
f8a10b4de6
GC: Restore: Split restore data into separate file. (#857)
## Description
Places `exchange.Restore` logic into single file
<!-- Insert PR description-->

## Type of change


- [x] 🐹 Trivial/Minor

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-09-14 23:22:52 +00:00
ashmrtn
5ae8259fd5
Change return value of data.Collection.FullPath to path.Path (#841)
* Make data.Collection.FullPath return path.Path

* Fixup graph connector code for path struct

* Add Elements call to Path interface

Still should not be used in place of the named functions to get specific
path elements (e.x. ResourceOwner())

* Fixup kopia wrapper path handling

Mostly removing shim code and minor fixup for building the directory
tree.

* All the test fixes
2022-09-14 10:01:11 -07:00
ashmrtn
573f55686f
Make ExchangeDataCollection use path.Path internally (#820)
* Have exchange data collection store path.Path

Still complies with the old FullPath() string interface until we update
that.

* Pass path.Path to NewCollection for exchange

Basically fixes up errors introduced by previous commit.

* Fixup exchange recovery path indices

All exchange paths now use the path struct, meaning the service,
category, and user elements are in the standard positions.

* use path package in selector reduction (#822)

Currently, during a reduction process, scopes
compare their values to the raw split on repoRef.
This causes some brittle indexing to retrieve
values from the rr, carrying assumptions that are
difficult to track across changes.  This PR trades
the string split for the paths package to better
integrate identification of the path values.

Adds some mocks and amends some error
behaviors in order to fit paths into the current
testing schema.

Co-authored-by: Keepers <ryanfkeepers@gmail.com>
2022-09-13 13:58:20 -07:00
Danny
61f769cc15
812 gc retrieval function tests (#817)
Issue 812: Test Expansion
Retrieval functions added to the test exchange package.

Issue #831:
Retry handler added to message restore pipeline.
2022-09-13 11:33:08 -04:00
Keepers
fa489782a3
purge both mail and calendars (#813)
expands the `purge` command to accept args for
purging mail folders, calendars, or both.  This
allows the test cleanup to ensure we aren't over-
populating either mail folders or calendars, thus
blocking CI actions
2022-09-12 17:35:16 -06:00
Keepers
110e874e5b
rename password to passphrase (#796)
General user friendliness renaming.  Will cause a
breaking change in any local environment for
developers and Ark.
2022-09-12 10:09:59 -06:00
Danny
432f984e61
GC: Backup:Event Feature to incorporate event ids for all calendars (#808)
Feature enhancement to include all Calendars in the event `restore-all` feature.
2022-09-12 08:17:04 -04:00
Keepers
cb467a3e46
correct the transformer used for contact folders (#814) 2022-09-09 15:34:28 -06:00
Danny
2e3ee15fd4
Sent / Receive DateTime Stamp Incorrect (#792)
## Description
Single Legacy Policies added to allow the values for sent / receive times to correspond to the original content for `exchange.Mail` objects

## Type of change
- [x] 🐛 Bugfix


## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->

*closes  #645

## Test Plan

<!-- How will this be tested prior to merging.-->

- [x] 💪 Manual
Values can be inspected using e2e restore pipelines.
2022-09-09 03:39:52 +00:00
Danny
ca6331b66e
[Fix] Issue #801: Error during Paging (#802)
## Description
Pagination fix for nill pointer

## Type of change

- [x] 🐛 Bugfix## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* closes #801

## Test Plan

- [x]  Unit test
2022-09-09 02:27:47 +00:00
Danny
c02817ac86
CI: Bug Fix Iterator Tests (#800)
## Description

Fixes mapping alignment during iterator tests

## Type of change


- [x] 🐛 Bugfix
- [x] 💻 CI/Deployment


## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* closes #797
## Test Plan

- [x]  Unit test
2022-09-09 01:59:46 +00:00
Keepers
6cdc691c6f
correct the corso module (#749)
swaps the corso go module from github.com/
alcionai/corso to github.com/alcionai/corso/src
to align with the location of the go.mod and
go.sum files inside the repo.

All other changes in the repository update the
package imports to the new module path.
2022-09-07 15:50:54 -06:00
Danny
6640d42aac
GC Restore: Event pipeline not working [FIX] (#780)
Bug Fix to ensure E2E testing. Test suite expanded.
2022-09-07 17:14:40 -04:00
ashmrtn
d707c22205
Resolve mail exchange folders to their real path in the hierarchy (#752)
## Description

Creates and uses a `ContainerResolver` interface to fetch container paths for items of different categories (when other resolvers are implemented). If the resolver is not available or fails to resolve a folder, defaults to the old implementation of using the folder's ID as its path.

## Type of change

Please check the type of change your PR introduces:
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 🐹 Trivial/Minor

## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->

* #456 

## Test Plan

<!-- How will this be tested prior to merging.-->

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-09-02 23:18:49 +00:00
ashmrtn
12f61dd71c
Switch to using path categories in graph connector (#734)
Will help make later PRs easier as the category will already be known.

## Description

<!-- Insert PR description-->

## Type of change

Please check the type of change your PR introduces:
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [x] 🐹 Trivial/Minor

## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
on path to:
* #456 

## Test Plan

<!-- How will this be tested prior to merging.-->

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-09-02 20:12:22 +00:00
Vaibhav Kamra
ed1a4bebce
Use a WaitGroup for AwaitStatus (#716)
## Description

This builds on the `MergeStatus` proposal and `WaitGroup` discussion proposed in #494 
Required for OneDrive where we are operating on multiple collections

## Type of change

Please check the type of change your PR introduces:
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 🐹 Trivial/Minor

## Issue(s)
#494 

## Test Plan

<!-- How will this be tested prior to merging.-->

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-09-02 17:17:17 +00:00
ashmrtn
61ce920972
Cleanup parameters to functions to appease linters (#730)
## Description

Group like parameters to graph functions so SonarCloud is happier

## Type of change

Please check the type of change your PR introduces:
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [x] 🐹 Trivial/Minor

## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
on the path to:
* #456 

## Test Plan

<!-- How will this be tested prior to merging.-->

- [ ] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2022-09-02 00:26:31 +00:00
ashmrtn
ae28971be3
Thread contexts through some GC functions (#728)
## Description

Will be used later and just generally useful if we ever add tracing or
such to these functions.

## Type of change

Please check the type of change your PR introduces:
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [x] 🐹 Trivial/Minor

## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->

#456 

## Test Plan

<!-- How will this be tested prior to merging.-->

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-09-01 23:31:43 +00:00
ashmrtn
4a96f2571d
Second attempt at directory cache/resolver for a user (#724)
## Description

Directory cache capable of converting a folder ID to a complete path to the folder

## Type of change

Please check the type of change your PR introduces:
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 🐹 Trivial/Minor

## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->

#456 

## Test Plan

<!-- How will this be tested prior to merging.-->

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-09-01 23:03:35 +00:00
Danny
7a25a7b70e
GC: Restore Event to M365 user account feature (#711)
## Description
logic and orchestration to take byte representation of M365 event, create a Corso calendar, and place the event and calendar into the M365 user's account. 
## Type of change

Please check the type of change your PR introduces:
- [x] 🌻 Feature

## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
closes #599 
closes #710
Requires PR #708 to ensure test folder is where events are created. 
## Test Plan

<!-- How will this be tested prior to merging.-->

- [x]  Unit test
2022-09-01 00:02:36 +00:00
Danny
8ead744e9f
GC: Restore: Create / Delete Calendar Feature (#708)
Feature creation: Create/Delete Calendar for aM365 user added to main branch
2022-08-31 22:03:54 +00:00
Keepers
5c59522fc2
print separate tables per entry type (#709)
## Description

Since a backup can encapsulate multiple data
types (ex: mail, contacts, and events), it doesn't
make sense to print one table with all disjoint
values.  This change splits up the print output
so that each data type in the details gets its
own table.

## Type of change

Please check the type of change your PR introduces:
- [x] 🌻 Feature

## Issue(s)

#501 

## Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-08-31 21:10:10 +00:00
Danny
305de77188
Iterate-Test Suite Move (#701)
Test suite move of iterator functions from `exchange-service-test.go` to `iterators_test.go`
2022-08-31 17:55:36 +00:00
Danny
b5a5378113
GC: Restore: Retrieve CalendarID Feature (#687)
Feature added to be able to retrieve containerID for `exchange.Event` types.
2022-08-31 17:28:57 +00:00
Danny
068f0560e4
Issue #501: Event support (#704)
Added EventRecurs as option for backup details.
2022-08-31 10:04:13 -06:00
Danny
67bc038c55
GC: Add WSL Linting Formatting src/internal/connector (#694)
Lint remaining of `internal/connector` package for wsl
2022-08-30 22:12:38 +00:00
Keepers
6f04321a60
replace scope values with filters (#674)
The filters package allows callers to specify both a target
to match on, and behavior of the comparison.  While data-
type scopes always equate to "equals", the control over
different comparison behavior is useful for info-type
filters.  This change integrates filters into scopes for
built-in control of those comparisons.
2022-08-30 21:29:24 +00:00
ashmrtn
e3abc281d6
Enable wsl for most of internal/connector package (#680)
* Lint part of internal/connector package with wsl

* Cleanup some wsl lint errors in connector package
2022-08-30 13:13:00 -07:00
Danny
0b3f345727
GC: Restore Contact Interface (#642)
Functionality added to restore `exchange.Contact` to M365 back store. Unit tests added to support proper testing of new feature.
2022-08-30 17:46:47 +00:00
Danny
b2d3e62536
Contact Restore: Create / Delete Folders
Contact folder creation and deletion enabled by the PR. Test suites expanded
2022-08-29 21:23:34 +00:00
Danny
d55ee98d7d
GC: Iterate "Select All Messages/Contacts" Consolidation with new Interface (#663)
`Descendable` interface used to move "IterateSelectAll" functions into a singular function.
Test suites are modified to support changes.
2022-08-29 16:29:15 +00:00
Danny
d3bced28f9
Bug fix: (#662)
Fixes bug introduced by merge.
2022-08-26 16:28:13 -06:00
Danny
ab92358ea6
GC: Backup all users flag support for exchange (#631)
All-users flag support enabled for Graph-Connector
2022-08-26 15:05:14 -06:00
Danny
2ceafd8d62
Contact / Mail folder interfaces (2) (#659)
Added 2 interfaces for future abstractions.
2022-08-26 16:15:55 -04:00
Danny
02c2b0b4d6
GC: Iterate-Functions moved to separate file (#657)
* Issue #654: Prepwork: `service_iterate.go` contains iterate functions for the package.
2022-08-26 11:47:49 -06:00
Danny
a2a5260d3f
Gc option extract (#653)
Content Separation `query_options.go` created for `exchange package.
2022-08-26 12:47:45 -04:00
Danny
6e7335cc4d
Gc restore multi framework (#639)
* Abstraction extended to restore. Application logic moved to service_functions.go
2022-08-23 20:30:46 +00:00
Danny
a1f79862f1
GC: Event collections to be referenced by calendarID instead of folder (#636)
`data.Collection` of type event  `fullPath()` contain the calendar id
2022-08-23 17:38:30 +00:00
Danny
d7abed1406
GC: setTenants not checking *users.GetMail() (#629)
User query updated from `*user.GetMail()` -> `*user.GetPrincipalName()`. Guard checks added and added GetUsers as one of the GraphQueryFunctions
2022-08-22 14:07:57 -07:00
Danny
4f5bb2b995
Exchange event info (#569)
Exchange Info feature added and verified with `mockdata` to ensure correctness
2022-08-19 13:43:26 -06:00
Danny
9d18d50cf8
Populate function Refactor to single function (#585)
Populate functions streamlined into one function with a few additional abstractions
2022-08-19 08:19:42 -07:00
Vaibhav Kamra
f8bfe93261
Scope restore CLI test (#592) 2022-08-18 14:19:08 +00:00
Danny
90be4200ca
GC: Events Support added. Not connected within ExchangeDataCollection (#516)
m365 events added for querying and iteration. Test suite added. Mock data is in a different PR.
2022-08-17 16:29:37 +00:00
Danny
ad2c17876f
ContactInfo added to exchange package (#574)
ContactInfo created for models.Contactable. ContactInfo test suite created.
2022-08-17 13:47:27 +00:00
Danny
be7b778769
GC Contacts requires additional tests placed in Exchange Services (#475)
Test coverage extended to GraphQuery and GraphIterateFuncs for exchange.Mail and exchange.Contact use cases.
2022-08-15 17:50:06 +00:00
ashmrtn
e76860fd80
Set line length to 120 characters (#506)
* Enable line width linter

Set to 120 which should be long enough to not be annoying but keep
things from getting "too long." Adding to get rid of the subjectiveness
of what is "too long." Tabs count as a single character.
2022-08-12 16:05:46 +00:00
Vaibhav Kamra
88bcd053e5
Simplify test utilities (#532) 2022-08-12 02:49:22 +00:00
ashmrtn
da66cc4c2f
Enable more rigorous version of gofmt linting (#488)
* Enable a stricter linter

* Fix new lint errors
2022-08-05 13:33:20 -07:00