## Description
Test restoring a single contacts folder and multiple contacts folders. This tests that items from different folders can be restored, but not that items from multiple folders can be backed up (because the restore colocates into a single folder)
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor
## Issue(s)
* #913
merge after
* #986
## Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
## Description
Add a regression test component to the retention test. This ensures that setting all values to 0 causes it to keep MaxInt latest snapshots.
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor
## Issue(s)
* closes#906
pending merge of
* kopia/kopia#2454
## Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
## Description
Stop fetching all folders for every new folder seen, erasing previous results. Regression test to come in a later patch as it requires some extra infrastructure
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor
## Issue(s)
* closes#982
## Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
## Description
Apparently `go mod tidy` was not the right way to go to ensure dependencies were there when building `mdgen`
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor
## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
## Test Plan
<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ] ⚡ Unit test
- [ ] 💚 E2E
## Description
Implementation of simplified Restore based on the following [spec](https://www.notion.so/alcion/Event-restore-semantics-061aee5288244629b1c53337e4dea306#6e974540c8804c4fa832218675534e1c)
<!-- Insert PR description-->
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
*closes #954<issue>
## Test Plan
<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
## Description
* treat empty values and nil pointers as equal when comparing Graph models
* make sure all items are always iterated through so the test doesn't hang
* explicitly check number of items retrieved
* remove some unneeded casts and assignments
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor
## Issue(s)
* #913
## Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
## Description
Adds some go runtime diagnostics tracking to
load testing, including some trace regioning.
Unfortunately, I couldn't find any third party trace library that didn't depend on a sidecar server
to sample against the application on. Therefore,
just starting with something basic.
## Type of change
- [x] 🤖 Test
## Issue(s)
* #902
## Test Plan
- [x] 💪 Manual
- [x] 💚 E2E
## Description
Adds support for folder and item selection
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor
## Issue(s)
* #627
## Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
## Description
add diagnostic records to load test runs
## Type of change
- [x] 🤖 Test
- [x] 💻 CI/Deployment
## Issue(s)
* #902
## Test Plan
- [x] 💚 E2E
## Description
Github cache imposes a 10gb limit, after which cache entries
will automatically get evictions (LRU policy). Various recent
queries have pinned our current cache size at 14-19 gb. The
golangci-lint cache alone accounts for 5-9 gb of that usage.
Meanwhile, it takes a shorter time for that package to download
missing modules than it does to run our tests. This change shuts
down the golangci cache in hopes that it improves our branch
caching retention.
## Type of change
- [x] 💻 CI/Deployment
## Issue(s)
* #790
## Test Plan
- [x] 💚 E2E
## Description
Fixes an issue with the path matching logic in selectors where if the path was a substring of what was
specified as the scope in the filter, the item would still be matched.
e.g.
Given 2 items - `/fold/contact1` and `/folderA/folderB/contact2` and a
selector `er.Include(er.ContactFolders("AnyUser", []string{"folderA/folderB"}))`,
the selector would match both items because `fold` is contained in the selector scope `folderA/folderB`
The fix is to invert the comparison - we check if the selector scope is contained in the item path instead.
In the example above, the selector string `folderA/folderB` will then only match the second items path.
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor
## Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
## Description
Test bad services or empty collections don't cause an error
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor
## Issue(s)
* #913
## Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
## Description
Add framework and tests for some exchange mail restore and backup situations. Framework can be used to test other situations in the future, this is just the starting point.
Some logic in the test can be further generalized/factored out once we know more about how paths will be transformed during restore
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor
## Issue(s)
* #913
## Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
## Description
The feature ensures that this Flat style directory only creates a single folder and restores all the contacts to the same folder during the restore process.
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor
## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
*closes #880<issue>
## Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [ ] ⚡ Unit test
- [ ] 💚 E2E
## Description
Adds the ability to use mockconnector for onedrive and exchange restores, reducing potential code duplication overall.
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor
## Issue(s)
* #913
## Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
## Description
Addresses some of the latest issues with the Dockerfile
* Go back to Alpine since it is smaller image. `ash` is available as shell so it is ok
* Add `corso` as UID and GUID 1001 for a predictable default user
* Add `/app/corso` as home directory for the `corso` which should address some of the issues around predictably mapping an external folder.
* Add Kopia config that mirrors the Kopia container from https://github.com/kopia/kopia/blob/master/tools/docker/Dockerfile
<!-- Insert PR description-->
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor
## Issue(s)
* fixes#968
## Test Plan
<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ] ⚡ Unit test
- [ ] 💚 E2E
## Description
* No new functionality exposed to CLI users
* generate restore folder names in CLI and pass down the stack
* update tests for new parameters
* centralize generation of restore container names
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor
## Issue(s)
* #897
* #913
## Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [x] 💚 E2E
## Description
Exchange app e2e tests need to scope their
backup dataset to only the primary folder, else
we risk exponential duplication of data when
the next backup picks up all the corso-copied
folders.
## Type of change
- [x] 🌻 Feature
- [x] 🐛 Bugfix
## Issue(s)
* #501
### Awaiting completion
- #810
- #811
## Test Plan
- [ ] 💪 Manual
- [ ] ⚡ Unit test
- [x] 💚 E2E
## Description
<!-- Insert PR description-->
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [x] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor
## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>
## Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [ ] ⚡ Unit test
- [ ] 💚 E2E
Mockdata exchanged for a newer version Kiota serialized version.
## Description
<!-- Insert PR description-->
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [x] 🐹 Trivial/Minor
## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* closes #956<issue>
## Test Plan
- [x] ⚡ Unit test
## Description
Abstraction and Seriailiaztion libraries updated
<!-- Insert PR description-->
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
*Related to #956 <issue>
## Test Plan
- [x] ⚡ Unit test
## Description
Adds the load test func calls and asserts for
both exchange and onedrive.
## Type of change
- [x] 🤖 Test
## Issue(s)
* #902
## Test Plan
- [x] 💚 E2E
## Description
Addresses a follow-up from the folder lookup implementation where we were not iterating through
all children (no pagination was implemented) when searching for a folder.
Instead of implementing pagination, this uses OneDrive path based addressing to do a direct lookup
for the folder.
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor
## Issue(s)
* #875
## Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [x] 💚 E2E
## Description
Start-end event pairs want ids as a means to
match each pair. Restore operations now gen
a transient id for this purpose. Backup ops pre-
generate the base ID instead of waiting for the
model store to generate the ID at the end of
the operation.
## Type of change
- [x] 🌻 Feature
## Issue(s)
* #894
## Test Plan
- [x] ⚡ Unit test
- [x] 💚 E2E
## Description
Adds file size and timestamps to enable selectors using this metadata
## Type of change
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor
## Issue(s)
* #594
## Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [x] 💚 E2E
## Description
Fixes a couple of regressions that got introduced on the restore path via code review changes and
kopia wrapper refactor.
Found while writing the OneDrive restore test. Would have been prevented if we had said test.
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor
## Test Plan
<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
## Description
This PR has just the struct definition to allow conversation about the design if needed. Future PR(s) will create and populate the struct during restore (in the CLI-layer since this struct is meant to be used to communicate that information) and pass it down the stack to the graph connector
The roadmap has plans for allowing users to
1. change the "root" folder the restore occurs in
1. change the resource owner of the restored items (e.x. restoring items from one user to a different user)
Both options can be enabled at the same time (e.x. restore "Inbox" from user A to "UserAInboxRestore" in user B). When the folder is not given it should default to "Corso_Restore_\<date time\>" and when the resource owner is not given the item should be restored under the original resource owner. As of now, we don't expect to have to have fine-grained control over restoring multiple users to different folders on a per-user basis. However, we may need to restore to specify a folder that is created for multiple users and restored to.
Examples:
**Not expected:** Restore UserA Inbox to folder1 in their hierarchy and UserB to folder2 in their hierarchy
**Expected:** Restore UserA Inbox to Corso_Restore_2022-09-23
POD struct:
```go
dest := selectors.RestoreDestination{
ContainerName: "Corso_Restore_2022-09-23",
}
```
**Expected:** Restore UserA Inbox to UserB Corso_Restore_2022-09-23
POD struct:
```go
dest := selectors.RestoreDestination{
ResourceOwnerOverride: "UserB",
ContainerName: "Corso_Restore_2022-09-23",
}
```
**Maybe expected:** For UserA and UserB, restore Inbox to folder1 in their separate hierarchies
```go
dest := selectors.RestoreDestination{
ContainerName: "Corso_Restore_2022-09-23",
}
```
Given the limited nature of the destination requirement, a POD struct seems like the best option for the foreseeable future. If we need more complex functionality down the line we can expand from this
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [x] 🐹 Trivial/Minor
## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #897
## Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [ ] ⚡ Unit test
- [ ] 💚 E2E
## Description
Track the count of bytes read and written in
kopia. For backups, this means the count of
bytes fed into kopia (hashed bytes), and the
amount written after compression and dedupe
(total file bytes). For restore, this is the count of bytes in all files read.
## Type of change
- [x] 🌻 Feature
## Issue(s)
* #894
## Test Plan
- [x] ⚡ Unit test
- [x] 💚 E2E
## Description
Moves the `path` package to the `pkg` package so other code outside of Corso can use it if they need it
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [x] 🐹 Trivial/Minor
## Issue(s)
* closes#908
## Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [x] 💚 E2E
## Description
Feature branch enabling filtering based upon `selectors.ExchangeScope`
<!-- Insert PR description-->
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
*closes #810<issue>
## Test Plan
- [x] ⚡ Unit test
## Description
Add tests for the `backup list exchange` subcommand. Tests mostly center around indirectly testing how selectors are created and used in this subcommand by checking the output of running the `Reduce` call on a known set of `details.DetailsEntry`s. Also check various error cases
Tests for invalid formats of flag values are disabled as that code does not currently exist. These tests can be enabled when #943 is resolved
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor
## Issue(s)
* #913
## Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
## Description
Add Google Analytics to docs and website
This can only be tested in production for docs but it looks like the HTML does the right thing.
## Type of change
- [x] 🌻 Feature
## Test Plan
- [x] 💪 Manual
## Description
Change the website preview ports to 5050/5051
This allows one to run the docs and website previews at the same time.
## Type of change
- [x] 🌻 Feature
## Test Plan
- [x] 💪 Manual
## Description
Moves constants to a file and adds more source definitions and backgrounds to constants.
<!-- Insert PR description-->
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [x] 🐹 Trivial/Minor
## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
*closes #823<issue>
## Test Plan
Const variables to be used in future PRs. No tests are available.
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [ ] ⚡ Unit test
- [ ] 💚 E2E
## Description
Fix broken docs links
This was caught by `make build` and should ultimately be a part of the build process.
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [x] 🐛 Bugfix
## Test Plan
<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
## Description
Enable dark/light mode controls.
This also reads the system preferences.
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [x] 🗺️ Documentation
## Test Plan
<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
## Description
Improve webpage and docs meta information
## Type of change
- [x] 🐹 Trivial/Minor
## Test Plan
<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
## Description
Correctly wire the tutorial up
This also adds the tutorial to the sidebar so that the user doesn't get lost.
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [x] 🗺️ Documentation
## Test Plan
<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
## Description
Consolidate function parameters by passing them as a single struct instead of a bunch of parameters. Helps keep function signatures shorter and also makes it easier to modify in the future (just change the struct rather than all call sites).
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [x] 🐹 Trivial/Minor
## Issue(s)
* #913
## Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
## Description
Aid in testing. Break the interface in two for the moment, one of which is embedded in the other. The smaller interface focuses just on getting information about backups from the ModelStore.
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [x] 🐹 Trivial/Minor
## Issue(s)
* #913
## Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
## Description
Upgrades Graph API to the newest version. Breaking change from the previous version, all functionality needs to be inspected to ensure operation.
<!-- Insert PR description-->
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
*closes #828<issue>
## Test Plan
- [x] ⚡ Unit test
- [x] 💚 E2E
## Description
- Add Corso logo to the docs footer
- Update Corso logo used in the header
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [x] 🐹 Trivial/Minor
## Test Plan
<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
## Description
shorten and consolidate the error message when
the provided selectors match zero entries in the
backup entity list.
## Type of change
- [x] 🐹 Trivial/Minor
## Issue(s)
* #564
## Test Plan
- [x] 💪 Manual
- [ ] ⚡ Unit test
- [ ] 💚 E2E
## Description
Adds the footprint file for running nightly load
tests out of the repository package. The tests
don't do anything at this stage, but that's okay.
Also introduces a workflow to kick off the tests
every day at 2am UTC.
## Type of change
- [X] 🤖 Test
- [X] 💻 CI/Deployment
## Issue(s)
* #902
## Test Plan
- [X] 💚 E2E
## Description
Unify colors between docs and the website.
This PR also brings some more color to the homepage
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [x] 🗺️ Documentation
## Test Plan
- [x] 💪 Manual
## Description
Upgrade Docusaurus
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [x] 🗺️ Documentation
## Test Plan
- [x] 💪 Manual
