## Description
Start-end event pairs want ids as a means to
match each pair. Restore operations now gen
a transient id for this purpose. Backup ops pre-
generate the base ID instead of waiting for the
model store to generate the ID at the end of
the operation.
## Type of change
- [x] 🌻 Feature
## Issue(s)
* #894
## Test Plan
- [x] ⚡ Unit test
- [x] 💚 E2E
## Description
Adds file size and timestamps to enable selectors using this metadata
## Type of change
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor
## Issue(s)
* #594
## Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [x] 💚 E2E
## Description
Fixes a couple of regressions that got introduced on the restore path via code review changes and
kopia wrapper refactor.
Found while writing the OneDrive restore test. Would have been prevented if we had said test.
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor
## Test Plan
<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
## Description
This PR has just the struct definition to allow conversation about the design if needed. Future PR(s) will create and populate the struct during restore (in the CLI-layer since this struct is meant to be used to communicate that information) and pass it down the stack to the graph connector
The roadmap has plans for allowing users to
1. change the "root" folder the restore occurs in
1. change the resource owner of the restored items (e.x. restoring items from one user to a different user)
Both options can be enabled at the same time (e.x. restore "Inbox" from user A to "UserAInboxRestore" in user B). When the folder is not given it should default to "Corso_Restore_\<date time\>" and when the resource owner is not given the item should be restored under the original resource owner. As of now, we don't expect to have to have fine-grained control over restoring multiple users to different folders on a per-user basis. However, we may need to restore to specify a folder that is created for multiple users and restored to.
Examples:
**Not expected:** Restore UserA Inbox to folder1 in their hierarchy and UserB to folder2 in their hierarchy
**Expected:** Restore UserA Inbox to Corso_Restore_2022-09-23
POD struct:
```go
dest := selectors.RestoreDestination{
ContainerName: "Corso_Restore_2022-09-23",
}
```
**Expected:** Restore UserA Inbox to UserB Corso_Restore_2022-09-23
POD struct:
```go
dest := selectors.RestoreDestination{
ResourceOwnerOverride: "UserB",
ContainerName: "Corso_Restore_2022-09-23",
}
```
**Maybe expected:** For UserA and UserB, restore Inbox to folder1 in their separate hierarchies
```go
dest := selectors.RestoreDestination{
ContainerName: "Corso_Restore_2022-09-23",
}
```
Given the limited nature of the destination requirement, a POD struct seems like the best option for the foreseeable future. If we need more complex functionality down the line we can expand from this
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [x] 🐹 Trivial/Minor
## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #897
## Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [ ] ⚡ Unit test
- [ ] 💚 E2E
## Description
Track the count of bytes read and written in
kopia. For backups, this means the count of
bytes fed into kopia (hashed bytes), and the
amount written after compression and dedupe
(total file bytes). For restore, this is the count of bytes in all files read.
## Type of change
- [x] 🌻 Feature
## Issue(s)
* #894
## Test Plan
- [x] ⚡ Unit test
- [x] 💚 E2E
## Description
Moves the `path` package to the `pkg` package so other code outside of Corso can use it if they need it
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [x] 🐹 Trivial/Minor
## Issue(s)
* closes#908
## Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [x] 💚 E2E
## Description
Feature branch enabling filtering based upon `selectors.ExchangeScope`
<!-- Insert PR description-->
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
*closes #810<issue>
## Test Plan
- [x] ⚡ Unit test
## Description
Add tests for the `backup list exchange` subcommand. Tests mostly center around indirectly testing how selectors are created and used in this subcommand by checking the output of running the `Reduce` call on a known set of `details.DetailsEntry`s. Also check various error cases
Tests for invalid formats of flag values are disabled as that code does not currently exist. These tests can be enabled when #943 is resolved
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor
## Issue(s)
* #913
## Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
## Description
Add Google Analytics to docs and website
This can only be tested in production for docs but it looks like the HTML does the right thing.
## Type of change
- [x] 🌻 Feature
## Test Plan
- [x] 💪 Manual
## Description
Change the website preview ports to 5050/5051
This allows one to run the docs and website previews at the same time.
## Type of change
- [x] 🌻 Feature
## Test Plan
- [x] 💪 Manual
## Description
Moves constants to a file and adds more source definitions and backgrounds to constants.
<!-- Insert PR description-->
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [x] 🐹 Trivial/Minor
## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
*closes #823<issue>
## Test Plan
Const variables to be used in future PRs. No tests are available.
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [ ] ⚡ Unit test
- [ ] 💚 E2E
## Description
Fix broken docs links
This was caught by `make build` and should ultimately be a part of the build process.
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [x] 🐛 Bugfix
## Test Plan
<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
## Description
Enable dark/light mode controls.
This also reads the system preferences.
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [x] 🗺️ Documentation
## Test Plan
<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
## Description
Improve webpage and docs meta information
## Type of change
- [x] 🐹 Trivial/Minor
## Test Plan
<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
## Description
Correctly wire the tutorial up
This also adds the tutorial to the sidebar so that the user doesn't get lost.
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [x] 🗺️ Documentation
## Test Plan
<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
## Description
Consolidate function parameters by passing them as a single struct instead of a bunch of parameters. Helps keep function signatures shorter and also makes it easier to modify in the future (just change the struct rather than all call sites).
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [x] 🐹 Trivial/Minor
## Issue(s)
* #913
## Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
## Description
Aid in testing. Break the interface in two for the moment, one of which is embedded in the other. The smaller interface focuses just on getting information about backups from the ModelStore.
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [x] 🐹 Trivial/Minor
## Issue(s)
* #913
## Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
## Description
Upgrades Graph API to the newest version. Breaking change from the previous version, all functionality needs to be inspected to ensure operation.
<!-- Insert PR description-->
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
*closes #828<issue>
## Test Plan
- [x] ⚡ Unit test
- [x] 💚 E2E
## Description
- Add Corso logo to the docs footer
- Update Corso logo used in the header
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [x] 🐹 Trivial/Minor
## Test Plan
<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
## Description
shorten and consolidate the error message when
the provided selectors match zero entries in the
backup entity list.
## Type of change
- [x] 🐹 Trivial/Minor
## Issue(s)
* #564
## Test Plan
- [x] 💪 Manual
- [ ] ⚡ Unit test
- [ ] 💚 E2E
## Description
Adds the footprint file for running nightly load
tests out of the repository package. The tests
don't do anything at this stage, but that's okay.
Also introduces a workflow to kick off the tests
every day at 2am UTC.
## Type of change
- [X] 🤖 Test
- [X] 💻 CI/Deployment
## Issue(s)
* #902
## Test Plan
- [X] 💚 E2E
## Description
Unify colors between docs and the website.
This PR also brings some more color to the homepage
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [x] 🗺️ Documentation
## Test Plan
- [x] 💪 Manual
## Description
Upgrade Docusaurus
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [x] 🗺️ Documentation
## Test Plan
- [x] 💪 Manual
## Description
Add test data and more complex tests for the selectors package. This helps codify the interface behavior of selectors which can help avoid surprises in the future
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor
## Issue(s)
* #913
## Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
## Description
Flushes out more information in the why everyone loves Corso section
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
## Test Plan
<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
## Description
Add links to Corso Discord.
This is not a one-way door decision until we actually launch
but, based on multiple conversations with DevRel candidates,
Discord will be superior for many reasons.
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
## Test Plan
<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
## Description
Extend the kopia metrics to include the snapshot's TotalFileSize parameter. This value also gets used as the metric for BytesWritten at the end of a
backup.
## Type of change
- [x] 🌻 Feature
## Issue(s)
* #894
## Test Plan
- [x] ⚡ Unit test
- [x] 💚 E2E
## Description
Largest changes around
* Adding tests for some init functions for conn
* Checking modelstore get metadata with tags functionality
* wrapper is best-effort for restore when there's errors
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor
## Issue(s)
* #913
## Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
## Description
Fade in key features, ordered by column
Without this change, the rest of the page was visually loading way before the key features section was done
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [x] 🐛 Bugfix
## Test Plan
<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
## Description
Backup and Restore metrics need to know the
count of resource owners in the operation. This
processor extracts that data from the collections
that get passed from producer to consumer.
## Type of change
- [x] 🌻 Feature
## Issue(s)
* #894
## Test Plan
- [x] ⚡ Unit test
- [x] 💚 E2E
## Description
Finish enabling wsl linter on Corso repo by removing the exception for onedrive package
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [x] 🐹 Trivial/Minor
## Issue(s)
* closes#481
## Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [ ] ⚡ Unit test
- [ ] 💚 E2E
Make sure dependencies are properly installed before building
<!-- Insert PR description-->
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor
## Issue(s)
## Test Plan
<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ] ⚡ Unit test
- [ ] 💚 E2E
## Description
Adds function call to the path package that returns the folder elements present in a path after the path prefix. This is safer than using `strings.Split()` because some paths may have `/` characters in elements.
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor
## Issue(s)
* closes#909
## Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
## Description
Disable the kopia retention policy that it uses by default. All snapshot deletions in corso must be done manually.
There should really be a follow up change to this after making a method in kopia public. Right now this PR is going off [some code](7bda16ab33/snapshot/policy/retention_policy.go (L117)) that shows the effective number of "latest" snapshots will default to intMAX if all of the other fields are set. We really should have a regression test or something for this to ensure that's actually the case. I've manually tested this patch, but that's not a scalable solution
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor
## Issue(s)
* closes#889
## Test Plan
<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
## Description
Fix responsive ordering of key features
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor
## Issue(s)
## Test Plan
<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ] ⚡ Unit test
- [ ] 💚 E2E
## Description
Over-restrictive scope correlation caused the
reduce processor to only include filters which
matched the data type of the scope. Ironically,
this allowed a superset of information to match,
by evading the _all-match_ expectations of
filter scopes.
Also replaces the data category consts inside /details
with the path category types, since those are acting as
better canonical owners of data type identification
throughout the app.
## Type of change
- [x] 🐛 Bugfix
## Issue(s)
* #890
## Test Plan
- [x] 💪 Manual
- [x] ⚡ Unit test
## Description
Add messaging pillar information on the website
Bullet points will be added next.
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
## Test Plan
- [x] 💪 Manual
## Description
Implements `data.Collection` restore for OneDrive and wires it up to the restore operation
Leverages previously added `onedrive` helpers that have integration tests.
An op/CLI level integration test will be added in a follow-up PR.
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor
## Issue(s)
* #668
## Test Plan
<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
```
$ ./corso restore onedrive --backup 7de4d68f-67c2-4e81-95a2-81cf0d1bd11d
Restored OneDrive in S3 for user
```
## Description
In order for purge to gracefully handle the variety of timestring values that may appear within a
folder name, the common/time.go file has added
a func to extract a time substring from a string.
Purge is switched to that extractor instead of
a len-based suffix check.
## Type of change
- [x] 🐛 Bugfix
## Issue(s)
* #805
## Test Plan
- [x] 💪 Manual
## Description
1/ introduce an ID() method int the account
package which provides the tenantID of the
configured account provider.
2/ introduce a event bus mock for testing.
## Type of change
- [x] 🌻 Feature
## Issue(s)
* #741
## Test Plan
- [x] ⚡ Unit test
## Description
Allow users to provide bucket names prefixed
with the s3 scheme: s3://my-bucket
## Type of change
- [x] 🐹 Trivial/Minor
## Issue(s)
* #560
## Test Plan
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
## Description
PR enables Filtering for `exchange.Contact` items wherein if an `exchange.ContactFolder` is requested specifically, only the requested folders are backed up.
<!-- Insert PR description-->
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
*closes #811<issue>
## Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
## Description
Encode all paths given to kopia with base64URL to ensure no special characters that kopia can't handle end up in there. The encoded paths are not stored in backup details nor are they ever surfaced to the user. This also works around the previous limitation where Corso was unable to properly backup or restore exchange email layouts that had folders containing `/` characters
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor
## Issue(s)
* closes#865
## Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [x] 💚 E2E
