## Description
Three primary changes:
1. move the list of load test users into a secret in the github test domain.
2. purge all data that was generated for users in the load test
3. fix all `m356` typos.
## Type of change
- [x] 🤖 Test
- [x] 💻 CI/Deployment
## Issue(s)
* #1266
## Test Plan
- [x] 💪 Manual
- [x] 💚 E2E
## Description
Store and retrieve owner info for onedrive files.
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor
## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* https://github.com/alcionai/corso/issues/1366
## Test Plan
<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [x] ⚡ Unit test
- [x] 💚 E2E
## Description
Adds two specialized filters: path prefix and path
contains. These filters differ from prior additions
in two ways: 1/ they accept a slice of inputs and
perform an implicit any-match when compared,
2/ they have particular rules about matching
completely within delimited elements.
In the next PR, these filters will replace the
standard prefix and contains comparisons that
are currently used in folder selection.
## Type of change
- [x] 🌻 Feature
## Issue(s)
* #1224
## Test Plan
- [x] 💚 E2E
## Description
Introduces config options (`--disable-tls` and `--disable-tls-verification`) to turn off TLS and TLS verification
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor
## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #1415
## Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
## Description
As the exchange container resolvers are pretty close implementation-wise, consolidate the generic code. Also move some of the tests around to better fit what file has the code being tested
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor
## Issue(s)
* closes#1362
## Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
## Description
With this in place, we just need client details to hook up to xray in
order to receive open telemetry info from our load test suite.
## Type of change
- [x] 🤖 Test
## Issue(s)
* #902
## Test Plan
- [x] 💚 E2E
## Description
Add tags to each kopia snapshot that include all service/category pairs in the snapshot and all resource owners in the snapshot. This allows future snapshots to lookup existing snapshots by those tags so they can be fed into the snapshot function. Feeding previous snapshots into the snapshot function enables kopia to detect previously uploaded files and skip uploading the data again
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor
## Issue(s)
* #1404
## Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
## Description
Swap from GraphAPI-based data cleanup (which doesn't allow
us to access hidden folders like Audit retention or delete-restoration)
to powershell script-based cleanup (which allows us to make
SOAP requests against legacy exchange apis).
## Type of change
- [x] 🐛 Bugfix
- [x] 🤖 Test
## Issue(s)
* #1266
## Test Plan
- [x] 💪 Manual
- [x] 💚 E2E
## Description
Grouping of event and contact folder caches to the same file. MailFolderCache left in a different folder until it can be decided where the tests before
<!-- Insert PR description-->
## Type of change
- [x] 🤖 Test
## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>
## Test Plan
- [x] ⚡ Unit test
- [ ] 💚 E2E
## Description
The panic in development mode is interfering with test scrutiny and other general behavior. Errors will be reported naturally without the need for this additional panic.
## Type of change
- [x] 🤖 Test
- [x] 🐹 Trivial/Minor
## Issue(s)
* #902
## Test Plan
- [x] 💚 E2E
## Description
By default goreleaser trims it from the generated filename.
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [x] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor
## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* fixes https://github.com/alcionai/corso/issues/1326
## Test Plan
<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ] ⚡ Unit test
- [ ] 💚 E2E
## Description
The secondary scripts in corso (factory, purge, etc) are not currently flushing their logs if an error occurrs since os.Exit(1) kills the current process without alloting time for deferred funcs to run.
## Type of change
- [x] 🐛 Bugfix
## Issue(s)
* #902
## Test Plan
- [x] 💚 E2E
## Description
Adds feature to be able to create mock events with attachments
<!-- Insert PR description-->
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* closes #1005 <issue>
## Test Plan
Testing:
```
internal/connector/exchange$ go test -v . --run TestExchangeServiceSuite/TestRestoreExchangeObject/Test_Event_with_Attachment
```
- [x] ⚡ Unit test
## Description
Add aws xray observability tooling, plus ways to quickly and
easily run the tools/load tests. Next steps are 1/ running the
daemon container alongside the load tests within github, and
2/ creating client credentials in aws's xray service so that
gathered trace info is propagated from the daemon proxy
to the xray service.
## Type of change
- [x] 🤖 Test
- [x] 💻 CI/Deployment
## Issue(s)
* #902
## Test Plan
- [x] 💪 Manual
- [x] 💚 E2E
## Description
Use the corso logger in analytics, to allow control
over log outputs from that package.
## Type of change
- [x] 🐹 Trivial/Minor
## Issue(s)
* #1324
## Test Plan
- [x] 💪 Manual
## Description
Overrides default goreleaser behavior and skips adding any files to the archive other than the binary
See https://goreleaser.com/customization/archive/#packaging-only-the-binaries
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [x] 🐹 Trivial/Minor
## Description
Adds production of 10 exchange items of each type, only for the load_test user, every time the load tests succeed. Also migrates that user ID out to a secret value stored in github.
## Type of change
- [x] 🤖 Test
## Issue(s)
* #902
## Test Plan
- [x] 💚 E2E
## Description
Builds a scope.Category() -> path.CategoryType trans- former func into scopes to standardize the relationship between the two properties.
## Type of change
- [x] 🌻 Feature
## Issue(s)
* #1133
## Test Plan
- [x] ⚡ Unit test
## Description
Intercept the lower-level RepoAlreadyExists error, and field a repository package version of the same error for sdk consumers to identify when initializing corso repos.
## Type of change
- [x] 🌻 Feature
## Issue(s)
* #1136
## Test Plan
- [x] 💪 Manual
- [x] 💚 E2E
## Description
Backup load tests are working successfully, we can
un-skip the restore portion of load tests now.
## Type of change
- [x] 🤖 Test
## Issue(s)
* #902
## Test Plan
- [x] 💪 Manual
- [x] 💚 E2E
## Description
Command line utility that allows for the printing of mockData based on the m365ID. Supports:
- `exchange.Mail`
- `exchange.Contacts`
- `exchange.Events`
<!-- Insert PR description-->
## Type of change
- [x] 🌻 Feature
## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
Related to:
* #1255<issue>
* #999
* #705
## Description
Load tests fail on the post-test cleanup using the purge script,
but are not currently logging the cause. Local repro has not
been successful. This change adds additional error logging
output for review in case of failure.
## Type of change
- [x] 🤖 Test
## Issue(s)
* #902
## Test Plan
- [x] 💪 Manual
## Description
Show messages for specific user interactions which previously had none.
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [x] 🐹 Trivial/Minor
## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* fixes https://github.com/alcionai/corso/issues/1177
## Test Plan
<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ] ⚡ Unit test
- [x] 💚 E2E
## Description
This adds the following functionality to the CLI progress output:
- Add a message e.g. to describe a completed stage
- Add a message describing a stage in progress (using a spinner) and trigger completion
- Add a progress tracker (as a counter) for a specified number of items (when the # items is known up front
- Improves `ItemProgress` by aligning the columns a bit better and also removing the progress bar and replacing
with a byte counter
Finally - the above are used in the `backup` and `backup create onedrive` flows. Follow up PRs will wire these up for
exchange and the restore flows also.
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor
## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #1278
## Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
## Description
CI associates config information with individual command instances. This causes problems when multiple test suites are running at the same time. Work around this by always using a new instance of a command when the command tree is built.
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor
## Issue(s)
* closes#1306
unblocks:
* #1192
## Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
## Description
Test flakes because CI cleanup job goes through and deletes some folders the folder cache ends up looking for. Solve this by only selecting specific mail folders instead of all of them.
Will need to rely on other tests to check if fetching all mail folders works. As only some folders are checked for though, overall impact of removing this test should be minor as the more targeted test is still present
Mistakenly re-added by #1134
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor
## Issue(s)
* closes#1222
## Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
## Description
Basic smoke tests to ensure multi-user restore and backup work properly. Will need updated to add OneDrive tests later on once the OneDrive test pattern is merged
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor
## Issue(s)
* #913
merge after
* #1186
## Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
## Description
Adds a flag that allows users to toggle off the progress
bar spinners. Primarily useful when running in a lower
log level such as debug.
## Type of change
- [x] 🌻 Feature
## Issue(s)
* #1112
## Test Plan
- [x] 💪 Manual
## Description
Small typo pending from review on https://github.com/alcionai/corso/pull/1247#discussion_r1000802903
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [x] 🐹 Trivial/Minor
## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>
## Test Plan
<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [x] ⚡ Unit test
- [x] 💚 E2E
## Description
Unify plural/singular for flags
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor
## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
## Test Plan
<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ] ⚡ Unit test
- [ ] 💚 E2E
## Description
Use zip as extension for windows binary release artifacts.
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [x] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor
## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* fixes https://github.com/alcionai/corso/issues/1234
## Test Plan
<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ] ⚡ Unit test
- [ ] 💚 E2E
## Description
Remove old "restored..." confirmation line from the CLI. Renamed the "Reference" column header to "ID" in the tabular output, which represents the shortRef aka item id.
## Type of change
- [x] 🐹 Trivial/Minor
## Issue(s)
* #1269
## Test Plan
- [x] 💪 Manual
- [x] ⚡ Unit test
## Description
verifyBackupInputs() is a newly created function that ensures that a selector given to GraphConnector is valid to continue.
Test src file: `graph_connector_test.go` expanded.
<!-- Insert PR description-->
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [x] 🐛 Bugfix
## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
*closes #1258<issue>
## Test Plan
- [x] ⚡ Unit test
## Description
<!-- Insert PR description-->
## Type of change
- [x] 🗺️ Documentation
## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
*Related to #1225<issue>
## Test Plan
<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
## Description
Switches the test accounts used in CI since the previously used account was hitting slowness due to
items in Deleted Folders.
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor
## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #1191
## Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [ ] ⚡ Unit test
- [x] 💚 E2E
## Description
Adds a tag describing the service within the backup model and the backup details model. Adds a filter builder pattern to the store wrapper for filtering results according to certain tags. Finally, adds a filter builder to specify the service type when listing backups.
## Type of change
- [x] 🐛 Bugfix
## Issue(s)
* #1226
## Test Plan
- [x] 💪 Manual
- [x] ⚡ Unit test
## Description
`path` is not cross platform as it hardcodes / as the separator. Also add a lint rule for this.
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor
## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* fixes https://github.com/alcionai/corso/issues/1236
## Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [ ] ⚡ Unit test
- [ ] 💚 E2E
## Description
Code removal from GraphConnector
`AddItemsToCollection` removed from graph_connector.go
## Type of change
- [x] 🐹 Trivial/Minor
## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>
## Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
## Description
`graph.ContainerResolver` has the capacity to keep the directory structure of the m365 objects that are helpful within a user's account. Leveraging this abstraction allows for a better flow of data from M365 into storage.
## Type of change
- [x] ⚡ : Optimization
## Issue(s)
* closes #1125<issue>
* closes#1122
## Test Plan
- [x] ⚡ Unit test
## Description
Adds a progress spinner tracking the collection-level work on an incremental basis. Useful for tracking collection progress when individual files may be too quickly resolved to visibly show progress bars.
In the future, we could combine the two so that the current item in progress always displays below the collection tracker.
## Type of change
- [x] 🌻 Feature
## Issue(s)
* #1112
## Test Plan
- [x] 💪 Manual
- [x] ⚡ Unit test
## Description
Call the OneDrive flag validation function to ensure flags that take time strings are formatted properly. Also add tests to ensure there's no future regressions for this
First commit has important code changes. Remainder of commits have test data/setup
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor
## Issue(s)
* closes#1231
* #913
## Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
