## Description
Adds foundational cli integration tests for backup/restore
of events and calendars
## Type of change
- [x] 🤖 Test
## Issue(s)
#501
## Test Plan
- [ ] 💪 Manual
- [ ] ⚡ Unit test
- [x] 💚 E2E
## Description
Single Legacy Policies added to allow the values for sent / receive times to correspond to the original content for `exchange.Mail` objects
## Type of change
- [x] 🐛 Bugfix
## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
*closes #645
## Test Plan
<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
Values can be inspected using e2e restore pipelines.
## Description
Introduces a Corso specific (more desirable) formatting for the auto generated documentation.
More or less an adaptation or https://github.com/spf13/cobra/blob/main/doc/md_docs.go. Unfortunately the original
package does not have sufficient hooks to allow for plugging in a new renderer.
Other approaches considered:
* [Override the UsageTemplate ](https://github.com/spf13/cobra/blob/main/user_guide.md#defining-your-own-help) - unfortunately does not apply to Markdown generation
* Use a golang template for the docs output - unfortunately the `pflags` package does not have a suitable way of iterating over the flags that can be easily fed into a template.
Follow on work:
* ~~Styling of flags tables as part of docs rendering - Done~~
* Opens up possibility for flag ordering - need to set `pflags.SortFlags = false` before parsing the flags - https://github.com/alcionai/corso/issues/783
* Can simplify sidebar command maintenance by naming the rendered docs files line similar to `<Num>_corso_command1_command2` where `<Num>` can be set as annotation on the commands. Order in the navbar can be set as part of command authoring - https://github.com/alcionai/corso/issues/784
## Type of change
Please check the type of change your PR introduces:
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [x] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 🐹 Trivial/Minor
## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* https://github.com/alcionai/corso/issues/536
* https://github.com/alcionai/corso/issues/528
## Test Plan
<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ] ⚡ Unit test
- [ ] 💚 E2E
## Description
Pagination fix for nill pointer
## Type of change
- [x] 🐛 Bugfix## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* closes#801
## Test Plan
- [x] ⚡ Unit test
## Description
Fixes mapping alignment during iterator tests
## Type of change
- [x] 🐛 Bugfix
- [x] 💻 CI/Deployment
## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* closes#797
## Test Plan
- [x] ⚡ Unit test
## Description
Enables a logging middleware that will log the Graph http calls if the `LOG_GRAPH_REQUESTS` environment variable
is set
## Type of change
Please check the type of change your PR introduces:
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [x] 🐹 Trivial/Minor
## Test Plan
<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ] ⚡ Unit test
- [ ] 💚 E2E
```$ LOG_GRAPH_REQUESTS=true CORSO_CI_TESTS=true go test -v .
...
=== RUN TestGraphConnectorIntegrationSuite
2022-09-08T13:39:25.556-0700 INFO graph/service_helper.go:61 REQUEST: GET /v1.0/users?%24select=id%2CuserPrincipalName HTTP/1.1
Host: graph.microsoft.com
Accept: application/json
Authorization: Bearer <REDACTED>
integration_runners.go:58: TestGraphConnectorIntegrationSuite run at 2022-09-08T20:39:25.967824Z
=== RUN TestGraphConnectorIntegrationSuite/TestAccessOfInboxAllUsers
2022-09-08T13:39:26.398-0700 INFO graph/service_helper.go:61 REQUEST: GET /v1.0/users?%24select=id%2CuserPrincipalName HTTP/1.1
Host: graph.microsoft.com
Accept: application/json
Authorization: Bearer <REDACTED>
```
## Description
Wires up the OneDrive collection logic to `operation.Backup`
Includes an integration test that runs against the test domain
Two bug fixes:
- Skip the "root" item that is returned by the delta query
- Fix incorrect usage of the `filepath.SplitList` function which does
not split a path into components. Instead use `strings.Split`. This
is ok because the paths returned here are not OS specific.
Regardless - this logic will be refactored when we use the `path` pkg.
## Type of change
Please check the type of change your PR introduces:
- [x] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 🐹 Trivial/Minor
## Issue(s)
#548
## Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [x] 💚 E2E
swaps the corso go module from github.com/
alcionai/corso to github.com/alcionai/corso/src
to align with the location of the go.mod and
go.sum files inside the repo.
All other changes in the repository update the
package imports to the new module path.
The path package changed the standard format
of fullPath and repoRef design. This should have
failed tests before being pushed to main, but was
able to slip in falsely while github actions were
configured to pass all tests until failed.
## Description
Creates and uses a `ContainerResolver` interface to fetch container paths for items of different categories (when other resolvers are implemented). If the resolver is not available or fails to resolve a folder, defaults to the old implementation of using the folder's ID as its path.
## Type of change
Please check the type of change your PR introduces:
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 🐹 Trivial/Minor
## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #456
## Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
## Description
Will be useful for parts of graph connector code that are abstracted to
take a category and return collections. Does not work for exchange
events or contacts yet as those paths haven't been defined.
## Type of change
Please check the type of change your PR introduces:
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 🐹 Trivial/Minor
## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
part of:
* #671
* #456
## Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
Will help make later PRs easier as the category will already be known.
## Description
<!-- Insert PR description-->
## Type of change
Please check the type of change your PR introduces:
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [x] 🐹 Trivial/Minor
## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
on path to:
* #456
## Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
## Description
Instead of initializing static AWS credentials, we rely on the
credential provider chain in [Kopia](https://github.com/kopia/kopia/pull/2213) to discover and initialize credentials.
This currently supports the following in this order:
- Static credentials
- Environment variables (what Corso used to implement)
- IAM
Going forward, this will also allow us to support shared credentials (cred file) once that is
added to the credential provider chain.
## Type of change
Please check the type of change your PR introduces:
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 🐹 Trivial/Minor
## Issue(s)
- #384
## Test Plan
<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual (see test output below)
- [ ] ⚡ Unit test
- [x] 💚 E2E
## Description
This builds on the `MergeStatus` proposal and `WaitGroup` discussion proposed in #494
Required for OneDrive where we are operating on multiple collections
## Type of change
Please check the type of change your PR introduces:
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 🐹 Trivial/Minor
## Issue(s)
#494
## Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
## Description
Turn into single constructor with bool instead of separate item and
folder constructors.
## Type of change
Please check the type of change your PR introduces:
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [x] 🐹 Trivial/Minor
## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
#671
## Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
## Description
Adds the new event filter selectors to the CLI for
backup detail and restore commands.
## Type of change
Please check the type of change your PR introduces:
- [x] 🌻 Feature
## Issue(s)
#501
## Test Plan
- [x] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
## Description
Add events and contacts. Still cannot create paths of these types
though.
## Type of change
Please check the type of change your PR introduces:
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [x] 🐹 Trivial/Minor
## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
#671
## Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
## Description
Group like parameters to graph functions so SonarCloud is happier
## Type of change
Please check the type of change your PR introduces:
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [x] 🐹 Trivial/Minor
## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
on the path to:
* #456
## Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [x] 💚 E2E
## Description
Will be used later and just generally useful if we ever add tracing or
such to these functions.
## Type of change
Please check the type of change your PR introduces:
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [x] 🐹 Trivial/Minor
## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
#456
## Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
## Description
Directory cache capable of converting a folder ID to a complete path to the folder
## Type of change
Please check the type of change your PR introduces:
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 🐹 Trivial/Minor
## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
#456
## Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
## Description
Prevents output printing of detail entries from splitting into
multiple json arrays when the entries contains multiple item
type.s
## Type of change
Please check the type of change your PR introduces:
- [x] 🐛 Bugfix
## Issue(s)
#501
## Test Plan
- [x] 💪 Manual
- [ ] ⚡ Unit test
- [ ] 💚 E2E
## Description
Adds the eventOrganizer exchange filter, wraps
up the addition of eventRecurs filtering, and fixes
a couple improperly designed filter constructors.
## Type of change
Please check the type of change your PR introduces:
- [x] 🌻 Feature
## Issue(s)
#501
## Test Plan
- [x] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
## Description
When a mock event is created the event is sent to personnel. Requires that personnel have valid email addresses.
Addresses created and updated in the mockconnector package.
## Type of change
Please check the type of change your PR introduces:
- [x] 🐛 Bugfix
## Test Plan
- [x] 💪 Manual
No emails populate when `TestRestoreEvents` completes
## Description
Exchange events were amended to use calendars
as a folder structure. This updates the selector
to treat events as having folders similar to mail and
contacts.
## Type of change
Please check the type of change your PR introduces:
- [x] 🌻 Feature
## Issue(s)
#501
## Test Plan
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
## Description
logic and orchestration to take byte representation of M365 event, create a Corso calendar, and place the event and calendar into the M365 user's account.
## Type of change
Please check the type of change your PR introduces:
- [x] 🌻 Feature
## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
closes#599closes#710
Requires PR #708 to ensure test folder is where events are created.
## Test Plan
<!-- How will this be tested prior to merging.-->
- [x] ⚡ Unit test
## Description
Adds the event-calendar flag to the cli to prepare
for selectors including the calendar as the events'
"folder" structure.
## Type of change
Please check the type of change your PR introduces:
- [x] 🌻 Feature
## Issue(s)
#501
## Test Plan
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
## Description
Since a backup can encapsulate multiple data
types (ex: mail, contacts, and events), it doesn't
make sense to print one table with all disjoint
values. This change splits up the print output
so that each data type in the details gets its
own table.
## Type of change
Please check the type of change your PR introduces:
- [x] 🌻 Feature
## Issue(s)
#501
## Test Plan
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
## Description
Adds filtering support for exchnage events in
the exchange selector.
## Type of change
Please check the type of change your PR introduces:
- [x] 🌻 Feature
## Issue(s)
#501
## Test Plan
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
## Description
* logic to split escaped path strings into elements
* wire up validity checks/struct construction
* tests for both invalid and valid path strings
## Type of change
Please check the type of change your PR introduces:
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 🐹 Trivial/Minor
## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
part of #671
merge after:
* #648
* #689
* #690
## Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
## Description
This adds the logic that materializes collections for a specified user
The collection paths are currently derived from OneDrive metadata but a future
PR will introduce `paths` pkg support to create these.
## Type of change
Please check the type of change your PR introduces:
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 🐹 Trivial/Minor
## Issue(s)
#388
## Test Plan
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
* Logic to validate service/category pairs for paths
* convert string to service or category const
* check that a given service/category pair is a valid resource type
* Add basic tests for validateServiceAndCategory
* Re-add logic for getting Exchange mail paths
Paths can either be for an item (email) or a folder. Returned struct
provides safe access to underlying information in the path once it is
created.
* Refine tests, add struct comment
Add test to make sure Item() and Folder() work properly for a resource
that has no parent folder, just the prefix stuff.
* Rework resource-specific paths based on comments
* use a single type to represent resource specific paths
* use service/category enums to denote the resource the path represents
* update tests to check for service/category of the path
* rename exchange_path*.go -> resource_path*.go
The filters package allows callers to specify both a target
to match on, and behavior of the comparison. While data-
type scopes always equate to "equals", the control over
different comparison behavior is useful for info-type
filters. This change integrates filters into scopes for
built-in control of those comparisons.
Multiple unescaped trailing path separators should be trimmed from input elements so there are no empty elements or trailing unescaped separators at the end of the path or between elements
Turn on now to keep from cycle of fixing most code, waiting for some PRs
in other packages, and then needing to do more fixups. Will slowly
enable on various files in connector package until we get full coverage.
