## Description
Now that GC is tracking the details entries during
a restore procedure, it can display the results in
the cli. Due to an absence of itemInfo, only the
item shortRef is displayed. But we can expand
on that from here.
## Type of change
- [x] 🌻 Feature
## Issue(s)
* #977
## Test Plan
- [x] 💪 Manual
- [x] 💚 E2E
## Description
In order for a restore operation to accurately print
out the list of items that were restored, it needs to
track item details within the GC restoration like is
done within kopia in backup. These details will
not be stored in a modelStore. But they will get
printed out on the CLI for end users.
Part 1 of multiple.
## Type of change
- [x] 🌻 Feature
## Issue(s)
* #977
## Test Plan
- [x] 💪 Manual
- [x] 💚 E2E
## Description
CI tests are showing a consistent panic when adding attendees
to events with zero content.
## Type of change
- [x] 🐛 Bugfix
## Issue(s)
* #1020
## Test Plan
- [x] ⚡ Unit test
- [x] 💚 E2E
## Description
Adds the following selectors to OneDrive details/restore :
- `file-name`, `folder`, `file-created-after`, `file-created-before`, `file-modified-after`, `file-modified-before`
Also includes a change where we remove the `drive/<driveID>/root:` prefix from parent path entries in details. This
is to improve readability. We will add drive back as a separate item in details if needed later.
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor
## Issue(s)
* #627
## Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
## Description
Adds some go runtime diagnostics tracking to
load testing, including some trace regioning.
Unfortunately, I couldn't find any third party trace library that didn't depend on a sidecar server
to sample against the application on. Therefore,
just starting with something basic.
## Type of change
- [x] 🤖 Test
## Issue(s)
* #902
## Test Plan
- [x] 💪 Manual
- [x] 💚 E2E
## Description
* No new functionality exposed to CLI users
* generate restore folder names in CLI and pass down the stack
* update tests for new parameters
* centralize generation of restore container names
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor
## Issue(s)
* #897
* #913
## Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [x] 💚 E2E
## Description
Exchange app e2e tests need to scope their
backup dataset to only the primary folder, else
we risk exponential duplication of data when
the next backup picks up all the corso-copied
folders.
## Type of change
- [x] 🌻 Feature
- [x] 🐛 Bugfix
## Issue(s)
* #501
### Awaiting completion
- #810
- #811
## Test Plan
- [ ] 💪 Manual
- [ ] ⚡ Unit test
- [x] 💚 E2E
## Description
Start-end event pairs want ids as a means to
match each pair. Restore operations now gen
a transient id for this purpose. Backup ops pre-
generate the base ID instead of waiting for the
model store to generate the ID at the end of
the operation.
## Type of change
- [x] 🌻 Feature
## Issue(s)
* #894
## Test Plan
- [x] ⚡ Unit test
- [x] 💚 E2E
## Description
Track the count of bytes read and written in
kopia. For backups, this means the count of
bytes fed into kopia (hashed bytes), and the
amount written after compression and dedupe
(total file bytes). For restore, this is the count of bytes in all files read.
## Type of change
- [x] 🌻 Feature
## Issue(s)
* #894
## Test Plan
- [x] ⚡ Unit test
- [x] 💚 E2E
## Description
Moves the `path` package to the `pkg` package so other code outside of Corso can use it if they need it
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [x] 🐹 Trivial/Minor
## Issue(s)
* closes#908
## Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [x] 💚 E2E
## Description
Upgrades Graph API to the newest version. Breaking change from the previous version, all functionality needs to be inspected to ensure operation.
<!-- Insert PR description-->
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
*closes #828<issue>
## Test Plan
- [x] ⚡ Unit test
- [x] 💚 E2E
## Description
shorten and consolidate the error message when
the provided selectors match zero entries in the
backup entity list.
## Type of change
- [x] 🐹 Trivial/Minor
## Issue(s)
* #564
## Test Plan
- [x] 💪 Manual
- [ ] ⚡ Unit test
- [ ] 💚 E2E
## Description
Extend the kopia metrics to include the snapshot's TotalFileSize parameter. This value also gets used as the metric for BytesWritten at the end of a
backup.
## Type of change
- [x] 🌻 Feature
## Issue(s)
* #894
## Test Plan
- [x] ⚡ Unit test
- [x] 💚 E2E
## Description
Backup and Restore metrics need to know the
count of resource owners in the operation. This
processor extracts that data from the collections
that get passed from producer to consumer.
## Type of change
- [x] 🌻 Feature
## Issue(s)
* #894
## Test Plan
- [x] ⚡ Unit test
- [x] 💚 E2E
## Description
Implements `data.Collection` restore for OneDrive and wires it up to the restore operation
Leverages previously added `onedrive` helpers that have integration tests.
An op/CLI level integration test will be added in a follow-up PR.
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor
## Issue(s)
* #668
## Test Plan
<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
```
$ ./corso restore onedrive --backup 7de4d68f-67c2-4e81-95a2-81cf0d1bd11d
Restored OneDrive in S3 for user
```
## Description
1/ introduce an ID() method int the account
package which provides the tenantID of the
configured account provider.
2/ introduce a event bus mock for testing.
## Type of change
- [x] 🌻 Feature
## Issue(s)
* #741
## Test Plan
- [x] ⚡ Unit test
## Description
Wires up the OneDrive Backup(create,list,details,delete) and Restore commands
Unit tests added but integration tests will be added after the underlying operation
PRs are merged.
## Type of change
Please check the type of change your PR introduces:
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 🐹 Trivial/Minor
## Issue(s)
#658#668
## Test Plan
<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
## Description
Adds events for repo init, backup start and end,
and restore start and end. Not all expected
values are available. TODOs have been marked
for values that need future implementation.
## Type of change
- [x] 🌻 Feature
## Issue(s)
* #741
## Test Plan
- [x] ⚡ Unit test
## Description
Small refactor of the data collection restore helper so that we can plug-in the OneDrive helpers in a
follow-up PR
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [x] 🐹 Trivial/Minor
## Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [ ] ⚡ Unit test
- [x] 💚 E2E
* Have exchange data collection store path.Path
Still complies with the old FullPath() string interface until we update
that.
* Pass path.Path to NewCollection for exchange
Basically fixes up errors introduced by previous commit.
* Fixup exchange recovery path indices
All exchange paths now use the path struct, meaning the service,
category, and user elements are in the standard positions.
* use path package in selector reduction (#822)
Currently, during a reduction process, scopes
compare their values to the raw split on repoRef.
This causes some brittle indexing to retrieve
values from the rr, carrying assumptions that are
difficult to track across changes. This PR trades
the string split for the paths package to better
integrate identification of the path values.
Adds some mocks and amends some error
behaviors in order to fit paths into the current
testing schema.
Co-authored-by: Keepers <ryanfkeepers@gmail.com>
## Description
Wires up the OneDrive collection logic to `operation.Backup`
Includes an integration test that runs against the test domain
Two bug fixes:
- Skip the "root" item that is returned by the delta query
- Fix incorrect usage of the `filepath.SplitList` function which does
not split a path into components. Instead use `strings.Split`. This
is ok because the paths returned here are not OS specific.
Regardless - this logic will be refactored when we use the `path` pkg.
## Type of change
Please check the type of change your PR introduces:
- [x] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 🐹 Trivial/Minor
## Issue(s)
#548
## Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [x] 💚 E2E
swaps the corso go module from github.com/
alcionai/corso to github.com/alcionai/corso/src
to align with the location of the go.mod and
go.sum files inside the repo.
All other changes in the repository update the
package imports to the new module path.
## Description
Exchange events were amended to use calendars
as a folder structure. This updates the selector
to treat events as having folders similar to mail and
contacts.
## Type of change
Please check the type of change your PR introduces:
- [x] 🌻 Feature
## Issue(s)
#501
## Test Plan
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
* Fix wsl lint errors in pkg package
* Fix wsl lint errors in most of internal package
Leave some sub-packages out that have higher churn at the moment.
Adds the basic cli-based backup integration test.
Due to discovering some corner cases about panic
conditions and other error handling in bad runtime
state, updates many other packages for safety.
* Enable line width linter
Set to 120 which should be long enough to not be annoying but keep
things from getting "too long." Adding to get rid of the subjectiveness
of what is "too long." Tabs count as a single character.
* Turn on revive linter, ignore only a few things
* Fix lint errors
Ignore shadowing of 'suite' in tests for now. Also move some constants
that had the same value to tester.
In order to both maintain control of test env data, as
well as ensure open-source users are able to run the
integration tests on personal account data, the test
user must be derived from the test configuration, rather
than hardcoded within the tests.
The alias of testing to ctesting prevented auto-importing
of the testing library. This change (arbitrarily) renames
the package so that it doesn't collide with the core pkg
for "testing".
In order for sdk users to build an operation using a repo,
without importing /inernal packages, the operation
options needs to be available via a /pkg package.
* return errors if selectors reduce all entries
During a `backup details` or `restore` command, if the
selectors produce 0 results, return an error rather than
running a no-op process.
embedding a baseModel means that objects like backup
have a backup.StableID, but no backup.ID nor backup.
backupID. This could be confusing for users. This change
swaps the value and type names, so that baseModels
include an ID value of type StableID.
* store backup operation results in the backup manifest
Adds backup operation metadata like the outcome
statistics and selector definitions to the backup manifest
entry. These additional details will appear when users
call `corso backup list`.
* add must-pass filters to selectors
Extends the selectors scope set to include Filters. This
allows users to define all-pass matchers (filters),
separate from any-pass matchers (includes), and global
exclusions.
* filter backup details by flags
`backup details` should have its output filtered by the flags provided by
the user. In addition, the selector's FilterDetails should maintain
information (esp service info) about the entries, rather than slicing them
down to only the path reference.
* refactor selector scopes to accept slices
Cli flag implementation was showcasing a toil issue: building
selectors required a lot of repetitious code for combining
inputs into sets of scopes. Since all of these productions
were effectively identical (eg: for each user, then each folder,
create a scope with the ids), the cleaner solution is to pack
that behavior into the scope constructors themselves.
