65 Commits

Author SHA1 Message Date
ashmrtn
bb73d142c5
Remove duplicate string->path.CategoryType helper function (#2260)
## Description

Tackles a few things:
* normalize case when going from string->path.Service or path.Category
* removes duplicate helper function for string->path.Category in favor of path package one

## Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No 

## Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

## Issue(s)

* closes #2259

## Test Plan

- [x] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-01-25 20:13:50 +00:00
Danny
53c5828caa
Path: Add category --> Pages. (#2109)
## Description
Adds path.Category `Pages` for SharePoint pages data type
<!-- Insert PR description-->

## Does this PR need a docs update or release note?
- [x]  No 

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature


## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* closes #2108 #<issue>

## Test Plan

- [z]  Unit test
2023-01-11 18:38:22 +00:00
ashmrtn
df086bd6f4
Move OneDrive path helper func to path package (#1880)
## Description

Later code in details package will need access to this function so it can update item details. However, importing the onedrive package in the details package leads to an import cycle. Moving it to the path package breaks the cycle while still allowing the code to be accessed.

## Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No 

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [x] 🐹 Trivial/Minor

## Issue(s)

* #1800 

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-12-21 23:32:47 +00:00
ashmrtn
8c15c3ce16
Select subtrees of data from base snapshots based on reason snapshot was picked (#1828)
## Description

Pick what data to source from a base snapshot by examining the reason the snapshot was selected as a base. This helps avoid two issues:
* pulling in unwanted data when the base has a superset of what is being backed up. Example: pulling in contacts from the base when only email is being backed up
* clobbering already selected data from a different base with data in the base currently being examined. Example: two snapshots, one with contacts and emails and the other with just emails. Second snapshot is newer than the first. The email items in the first snapshot should not clobber those in the first when building the hierarchy

This PR also has the effect of, under some conditions, reducing the amount of data that is pulled from the remote kopia repo when building the hierarchy. This occurs because only the subtrees that will be used in the new backup are traversed instead of traversing the entire snapshot

## Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No 

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

* #1740 

## Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-12-16 22:31:13 +00:00
Vaibhav Kamra
93a7ff27e3
Store backup details in a separate snapshot (#1755)
## Description

This PR uses a separate snapshot to store backup details instead of a Corso model (i.e. a kopia manifest).

Introduces a `StreamStore` that can be leveraged to store larger metadata objects. We can also leverage this
for incrementals or restartable backups going forward.

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #1735 

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2022-12-10 00:47:49 +00:00
ashmrtn
5c7d2168fc
Create path categories for metadata folders (#1718)
## Description

Corso needs to store some transient information between backups so that it can perform more efficient incremental backups. This transient information will live at a path `tenant-id/service/user-id/categoryMetadata`. This patch adds an initializer, category enums, and tests for these new path locations

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

* #1685

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-12-07 02:32:49 +00:00
Danny
94145ecbf6
Path: Category expanded to include SharePoint Lists (#1610)
## Description
Backup of SharePoint requires the extension of `path.CategoryType` to be able to support the creation of `data.Collections` for `SharePoint.Lists`. 
## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature


## Test Plan

- [x]  Unit test
2022-11-28 20:12:57 +00:00
Keepers
5b0549fb32
migrate sharepoint cat from file to library (#1555)
## Description

Adds a LibraryCategory in paths, and changes
the current sharepoint code to use that cat
instead of the onedrive files cat.  This is mostly to keep sharepoint library drives separate from
onedrive and sharepoint site list items.

## Type of change

- [x] 🌻 Feature

## Issue(s)

* #1506

## Test Plan

- [x]  Unit test
2022-11-22 19:01:05 +00:00
Keepers
10acf0ccf6
generic drive retrieval for sharepoint (#1536)
## Description

Adapts the graph onedrive library to handle
access to drive data across both onedrive and
sharepoint services.

## Type of change

- [x] 🌻 Feature

## Issue(s)

* #1506

## Test Plan

- [x] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2022-11-22 18:05:47 +00:00
Rajat Kateja
80bbbdb87d
Make split function path package public (#1485)
## Description

Want to use the `split` function through the path package to split arbitrary escaped strings. 

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [X] 🐹 Trivial/Minor

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [X]  Unit test
- [ ] 💚 E2E

Existing unit tests pass.
2022-11-16 23:55:33 +00:00
Keepers
11371f6e94
add sharepoint to path (#1465)
Adds the sharepoint service to /pkg/path.  Currently uses the
"Files" category for its category type, which is just a placeholder
for kicking off development.

Additionally, uncomments selector tests that were dependent
upon the path service declaration.
2022-11-14 15:35:57 -07:00
Keepers
302c3f39dd
add path prefix/contains filters (#1425)
## Description

Adds two specialized filters: path prefix and path
contains.  These filters differ from prior additions
in two ways: 1/ they accept a slice of inputs and
perform an implicit any-match when compared,
2/ they have particular rules about matching
completely within delimited elements.

In the next PR, these filters will replace the
standard prefix and contains comparisons that
are currently used in folder selection.

## Type of change

- [x] 🌻 Feature

## Issue(s)

* #1224

## Test Plan

- [x] 💚 E2E
2022-11-03 22:37:25 +00:00
ashmrtn
72a3c7ab3b
Trim trailing '/' from CLI input for restore/details (#1175)
## Description

Use paths package to trim unescaped trailing '/' characters from input for Exchange mail and OneDrive folder names. Add tests for Exchange showing that the trimming also works properly if the folder name ends with '/'.

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

* closes #1147 

## Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-10-17 17:54:36 +00:00
ashmrtn
ec916bc1b3
Test ensuring shortrefs are unique for edgecase (#1105)
## Description

Some paths can generate identical unescaped strings even though they have different elements. Add a test ensuring they generate unique ShortRefs even under these conditions.

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

* closes #1092 

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-10-10 22:47:23 +00:00
ashmrtn
ff2db0c553
Export path package to other codebases (#912)
## Description

Moves the `path` package to the `pkg` package so other code outside of Corso can use it if they need it

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [x] 🐹 Trivial/Minor

## Issue(s)

* closes #908 

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2022-09-26 18:34:19 +00:00