65 Commits

Author SHA1 Message Date
Hitesh Pattanayak
dfb4a73f56
modifies list restore handler and handles lists advanced restore (#5024)
- includes GetList in list restore handler
- includes GetListsByCollisionKey in list restore handler
- updates mock list restore handler


#### Does this PR need a docs update or release note?
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature

#### Issue(s)
#4754 

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2024-01-23 16:18:47 +05:30
Hitesh Pattanayak
66664b69e5
skips backup of non-recoverable list templates (#4940)
Skips backup of non-recoverable list templates.

#### Does this PR need a docs update or release note?
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)
#4754 

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2023-12-27 09:59:10 +00:00
Hitesh Pattanayak
50fd28aab2
adds ListInfo details to SharepointInfo details (#4916)
adds ListInfo details to SharepointInfo details
Changes previously approved in PR: https://github.com/alcionai/corso/pull/4851

#### Does this PR need a docs update or release note?
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature

#### Issue(s)
#4754 

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2023-12-22 06:50:34 +00:00
Keepers
c58cd9302f
move /cli/config to /pkg (#4858)
Part 1 of two changes I've been carting around since the hackathon.
This PR moves the /cli/config package into /pkg/config.  This allows
sdk consumers to access corso configuration handling, intead of
requiring the CLI package to access it.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🤖 Supportability/Tests


#### Test Plan

- [x] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2023-12-18 18:24:33 +00:00
Abin Simon
68256cf59f
Change printing of backup stats to single line (#4526)
Previously we were printing a table for a single item which, for one
was unnecessary, but this also make the output of a backup more
compact.

Before:

```
Logging to file: /home/meain/.cache/corso/logs/2023-10-20T10-21-23Z.log
Connecting to M365                               done
Connecting to repository                      1s done

Backing Up ∙ Teams Testing (meain)
Discovering items to backup                  27s done
    Libraries (TeamsTestingmeain)                  done (found 0 items)
    Libraries (TeamsTestingmeain-Shared0)          done (found 0 items)
    Libraries (TeamsTestingmeain-Private2)         done (found 0 items)
    Messages                                    7s done (found 6 channels)
Backing up data                               7s done
Backup complete ∙ 7b40dd40-f808-4d57-8e39-b4553e48dc5d
    ID                                    Bytes Uploaded  Items Uploaded  Items Skipped  Errors
    7b40dd40-f808-4d57-8e39-b4553e48dc5d  0 B             0               0              0

Completed Backups:
    ID                                    Started At            Duration       Status     Resource Owner
    7b40dd40-f808-4d57-8e39-b4553e48dc5d  2023-10-20T10:21:32Z  36.632747912s  Completed  Teams Testing (meain)
```

After:
```
Connecting to M365                               done
Connecting to repository                      1s done

Backing Up ∙ Teams Testing (meain)
Discovering items to backup                  31s done
    Libraries (TeamsTestingmeain)                  done (found 0 items)
    Libraries (TeamsTestingmeain-Shared0)          done (found 0 items)
    Libraries (TeamsTestingmeain-Private2)         done (found 0 items)
    Messages                                    9s done (found 6 channels)
Backing up data                               7s done
Backup complete ∙ ffb2f619-1cb7-4a11-b3e2-7300aa513c6a
Bytes Uploaded: 0 B | Items Uploaded: 0 | Items Skipped: 0 | Errors: 0

Completed Backups:
    ID                                    ID                    Started At     Duration   Status                 Resource Owner
    ffb2f619-1cb7-4a11-b3e2-7300aa513c6a  2023-10-20T10:23:35Z  40.096203016s  Completed  Teams Testing (meain)
```


---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-11-16 12:43:27 +00:00
Abin Simon
eb8f3131f9
Misc fixes for handling groups (#4195)
A bunch of places where we missed handling groups
<!-- PR description-->

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-09-07 06:36:18 +00:00
ashmrtn
7972ff6e8f
Shuffle around logic for details merging (#4087)
Shuffle around some logic for details merging so that
we always attempt to extract a LocationRef from the
backup base entry that's currently being examined.

A LocationRef should always be available from either
the LocationRef field in the details entry (newer
backups) or by extracting it from the RepoRef (older
backups)

Manually tested incremental backups for exchange in
the following scenarios:
1. v0.3.0 backup (calendars use IDs in RepoRef) ->
   incremental with this patch -> incremental with
   this patch
1. v0.2.0 backup (exchange uses folder names in
   RepoRef) -> incremental with this patch ->
   incremental with this patch

The above tests should cover the cases where:
* base backup details don't have LocationRef for
  exchange items
* base backup details have LocationRef for exchange
  items

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* closes #3716

#### Test Plan

- [x] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-08-23 19:16:42 +00:00
ashmrtn
75c3e5cd33
Don't munge drive-based ShortRefs (#4009)
Munging code originally existed so that SDK
users would get unique ShortRefs if an item
moved or was renamed. However, we no longer
need to support that so this removes that
munging code

Manually tested making a new backup with this
PR where the merge base was a backup made prior
to this PR. A folder with a subfolder and items
was moved between the two backups. Backup
details for the new backup contained all the
expected entries

Restore selector logic should be unaffected
as that assumes the user has passed in a
ShortRef that was previously printed to the
CLI by `corso backup details`. Input is
compared against the ShortRef stored in the
entry, `ShortRef()` is not called on any `Path`

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* closes #4012

#### Test Plan

- [x] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-08-16 17:37:30 +00:00
ashmrtn
eb95b43010
Remove updated field from details (#4004)
No longer required by SDK users and not exposed directly to CLI users unless they're looking at the JSON output from details. This field is not documented anywhere though so there's no guarantees that we technically need to uphold with it.

Manually tested:
* restoring from a backup that had this field
* making a fresh backup without this field when the merge base did have this field

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* closes #3986

#### Test Plan

- [x] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2023-08-11 16:32:21 +00:00
Abin Simon
8c939c0f0d
Split services and collections for OneDrive & SharePoint (#4002)
Only code movement, no code changes.

Moved services to `/internal/m365/services/{onedrive,sharepoint,exchange}`
Moved collections to `/internal/m365/collection/{drive,site}`

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-08-10 06:08:34 +00:00
Keepers
2f6d731993
rename connector -> m365 (#3600)
renames /internal/connector to /internal/m365.  No logic changes in this PR.  Only the dir rename, import renames, and one linter shadowing rename.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #1996

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-06-13 18:35:39 +00:00
Abin Simon
6b7502bcb8
Ensure calling Details() twice does not duplicate folder names (#3556)
Related to https://github.com/alcionai/corso/pull/3140 . We were editing the deets in place and returning a pointer. This is fine as long as we were only calling it once, but https://github.com/alcionai/corso/pull/3480 created an additional call for this which causes the `konwnFolders` to be appended twice.

Old backups will still have incorrect data, but an incremental backup on top of it should fix up the data.

---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* Fixes https://github.com/alcionai/corso/issues/3555

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-06-01 18:48:37 +00:00
Keepers
6c410c298c
consts for drives, root: (#3385)
#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-05-11 22:46:51 +00:00
ashmrtn
67d5c53420
Fix NPE in SharePoint incremental backup (#3307)
Original code was switching based on the ItemType, but SharePoint historically used the OneDriveItem ItemType, making the system think it should be updating a OneDriveItemInfo struct instead of a SharePointItemInfo struct.

Also add a regression test for older backup formats.

---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

* closes #3306

#### Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-05-04 17:35:12 +00:00
Keepers
c152ce8b25
move and rename common/time.go (#3215)
moves common/time.go to common/dttm/dttm.go, and renames many of the consts and funcs with names that better explain their design and intent of usage.

Only code movement, no logical changes.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-05-02 19:39:29 +00:00
Keepers
3b9d2841d4
rename DetailsEntry to Entry to avoid stuttering (#3265)
#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-05-01 19:29:06 +00:00
Keepers
306db14339
add itemref to details entry (#3160)
Adds an new details entry field: itemRef.
This holds a stable, semi-unique identifier to the item represented by that entry.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🌻 Feature

#### Issue(s)

* #3027

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-04-19 23:25:11 +00:00
ashmrtn
f01e25ad83
Redo folder generation in backup details (#3140)
Augment the folder backup details entries with
LocationRef and some information about what
data type generated the entry. Also add top-level
container information like drive name/ID if
applicable

Refactor code to do folder generation to make it
more contained and require less parameters to add
entries to backup details

Important changes are in details.go. All other
changes are just to keep up with the slightly
modified function API or update tests

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

* closes #3120
* closes #2138

#### Test Plan

- [x] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-04-19 20:27:55 +00:00
ashmrtn
1826b66e8d
Populate and persist OneDrive LocationRef (#3111)
Leverage structs from previous PRs to
populate and persist the LocationRef
for OneDrive

Right now the source of the
LocationRef is still based off the
RepoRef path, but this is in the
OneDrive code only

Manually tested
* updating a subfolder path when parent folder was moved
* populating LocationRef when the base backup didn't have LocationRefs when nothing was changed in OneDrive
* populating LocationRef when the base backup didn't have LocationRefs when something was changed in OneDrive

---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

* closes #2486

#### Test Plan

- [x] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-04-14 21:31:16 +00:00
ashmrtn
9e692c7e2e
Create abstraction to represent a unique location (#3100)
UniqueLocation allows representing a folder path that has
both a location that will be stored in backup details and
a location that can be used as a key in maps. The key for
maps is guaranteed to be unique for all data types within
a service

Add a function to extract a unique location from a backup
details entry and tests for that

UniqueLocation will eventually be used in GraphConnector
and KopiaWrapper so it needs to be in a package that both
of them can import

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #2486

#### Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-04-12 18:08:46 +00:00
ashmrtn
dae79f4acf
Folder display name fix (#3062)
Fixes folder display names in backup details.
They got off when locations were switched to
returning path builders instead of full paths

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue
- closes #3061

#### Test Plan

- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-04-07 05:34:28 +00:00
ashmrtn
59e17757bf
Switch LocationPath return type (#3038)
LocationRefs don't need prefix directories, so using a path.Path is
overkill. This will also fit better with changes to come where we need
to parse and update the LocationRef of merged backup details entries.

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #2486

#### Test Plan

- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-04-05 21:10:42 +00:00
Keepers
9d73d0c8e2
replace errors.* with clues.* (#2924)
Mostly find/replace on errors.N and errors.W. Also turns all wrapf into wrap, and removes as many errorf calls as possible.

Might follow up with a linter to enforce this change.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #1970

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-03-28 04:01:54 +00:00
ashmrtn
dfceb79e0a
Update Exchange backup detail ParentPath when merging details (#2826)
Exchange now holds a reference to the parent folder so be sure to update
that if needed when merging backup details

Calendar is the only thing that populates location ref right now so this
alone doesn't completely solve the issue

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [x] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #2819

#### Test Plan

- [x] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-03-27 20:44:41 +00:00
Abin Simon
c3610d1832
Revert "force omission of meta files (#2662)" (#2706)
This reverts commit 1945db6b65827e6dd6d57de8ac42fe4a119db685.

We do add the `isMeta` flags for missing items at f60207ee32/src/pkg/repository/repository.go (L363)

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-03-24 07:48:28 +00:00
Keepers
285f8f01ef
update sharepoint cli examples (#2822)
Updates the sharepoint cli examples to better
reflect the current support and utilities.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🌻 Feature

#### Issue(s)

* #2786

#### Test Plan

- [x] 💪 Manual
- [x] 💚 E2E
2023-03-21 17:33:24 +00:00
Keepers
76b3fe3b86
append ToCore to all errors tests (#2793)
In order to retrieve all clues structured error data in tests, we need to extract it from the error using the clues library.

This change appends `clues.ToCore(err)` to all
variations of `assert.NoError(t, err)`.  The only
other changes are those necessary to preserve
linting, or to produce an error variable for the
ToCore call.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🤖 Test
- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #1970

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-03-15 19:02:47 +00:00
ashmrtn
8a81624b98
Change to storing files by ID instead of display name (#2496)
## Description

Change to storing OneDrive files by ID instead of their OneDrive display name. This allows delta token-based incrementals to use an exclude list to remove previously backed up items from a backup during hierarchy merging

Also updates the following:
* selectors to match on file display name instead of kopia file name
* ShortRefs for OneDrive files to update when the file display name is updated

## Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No 

## Type of change

- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

## Issue(s)

* closes #1535

## Test Plan

- [x] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2023-03-14 22:32:06 +00:00
Keepers
635e4f6bc4
rename libraries -> libraryFolder (#2774)
The selector moniker "libraries" is incorrect.  It should refer to the library folder, since it only
matches on directory structures within a given
drive.  The 'library' is analogous to the drive
itself, and will need a separate selector of its
own.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🐛 Bugfix

#### Issue(s)

* #2757

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-03-14 01:38:54 +00:00
Keepers
dc21f4ce07
add a QoL path.Build func (#2763)
Adds a path.Build() func that can arbitrarily build a path according to expected standards without first initializing a path.Builder{}.
---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-03-13 16:56:31 +00:00
Danny
0766ee7afd
GC: SharePoint: ItemType Decouple (#2720)
<!-- Insert PR description-->
ItemTypes for SharePoint were `SharePointItem` only. This did not allow for data to be separated by category. 
Newly created categories:
- SharePointLibrary
- SharePointList
- SharePointPage

Breaks history for `SharePointItems`. All previous SharePoint items are available during `Corso List` command. However, when using the details command with the backup ID for `SharePointItems` restored prior to this PR , the return will be:
```
no items match the specified selectors
```
---

#### Does this PR need a docs update or release note?


- [x] 📝 : Will need documentation

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature


#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* related to #2709<issue>

#### Test Plan

- [x]  Unit test
2023-03-09 15:51:24 +00:00
Keepers
0b2d0067ee
refactor streamstore for generic use (#2723)
Refactors the stream store so that it can comply with multiple different writers and readers beyond just the details.Details struct.  This will allow us to use the stream store to ingest large lists of errors fault items without burdening the backup manifest.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #2708

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-03-08 17:16:09 +00:00
Danny
030147ddcb
Corso: Exchange: Mail: Backup Details to include recipients --JSON ONLY (#2689)
<!-- Insert PR description-->
#### Adds  Email recipients to the backup details for Exchange Mail. If the recipients are of length greater than one, the list of individuals is included and separated by a `,`

---

#### Does this PR need a docs update or release note?
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature


#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* closes  #2687<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x]  Unit test
2023-03-07 02:32:38 +00:00
Danny
36d7d44bb4
Corso: Exchange: Details: add parentpath (#2690)
<!-- Insert PR description-->
Parent Path field added to  `Exchange` Backup Details 
---
- Only Exchange mail will display the path for  backup details. 
#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* closes #2352<issue>

#### Test Plan


- [x]  Unit test
2023-03-06 18:01:26 +00:00
Keepers
1945db6b65
force omission of meta files (#2662)
While the `isMeta` flag will solve this in the
future, the current state requires us to filter
out all meta and dirmeta files, even historically.

---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included

#### Type of change

- [x] 🐛 Bugfix

#### Issue(s)

* #1970

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-02-27 22:46:55 +00:00
Danny
ad75540b03
GC: SharePoint: Library: Details: Change to Displayed data (#2632)
<!-- Insert PR description-->
Updates for `./corso backup details sharepoint --backup <bID>` 
- Ensures that Drive Name is part of the backup
- Adds field DisplayName to backup details

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature


#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
*closes  #2601 <issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
2023-02-27 15:07:42 +00:00
ashmrtn
40cb810861
Use test wrapper in pkg package (#2531)
## Description

Leave out account and storage subpackages as those are used by tester and create an import cycle.

## Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No 

## Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Test
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

## Issue(s)

* #2373

## Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-02-25 04:50:00 +00:00
Keepers
cb12329d0c
don't handle meta files in details (#2606)
## Description

Remove the display and interaction with .meta
files from details entries.

## Does this PR need a docs update or release note?

- [x]  Yes, it's included

## Type of change

- [x] 🐛 Bugfix

## Test Plan

- [x]  Unit test
2023-02-23 04:14:17 +00:00
Keepers
52455356e5
add locationRef to details (#2427)
## Description

Adds a new reference to the details ent: location-
ref.  The location holds the human-readable
version of the item's location in whatever m365
service sourced the item.  Hookup is incomplete,
following PRs will fill out functionality.

Also adds a LocationPather interface to data_
collections to pass this data back and forth
between producers and consumers.

Should be safe to merge into main.

## Does this PR need a docs update or release note?

- [x] 🕐 Yes, but in a later PR

## Type of change

- [x] 🌻 Feature
- [x] 🐛 Bugfix

## Issue(s)

* #2423

## Test Plan

- [x]  Unit test
2023-02-13 20:19:04 +00:00
Danny
9f05e83d43
CLI: Adds to the displayable headers for OneDrive and SharePoint. (#2090)
## Description
The update adds `DriveName` to SharePoint details. Values are currently the M365ID

## Does this PR need a docs update or release note?

- [x]  No 

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature


## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* related to #1938<issue>
* related to #2064 


## Test Plan
- [x]  Unit test
2023-01-11 12:49:20 +00:00
Vaibhav Kamra
ad691148fe
Mark folder updated when an updated item is added (#1987)
## Description

If an item is updated, update the folder entries it belongs to. 

Also contains a minor refactor - unexport `FolderEntry` to `folderEntry`

## Does this PR need a docs update or release note?

- [x]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No 

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #1812 

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-12-29 23:41:04 +00:00
Vaibhav Kamra
3141b181b6
Address review feedback from #1896 (#1913)
## Description

- Adds accessor methods for size/modified
- Do not initialize folder in test with a modified time

## Does this PR need a docs update or release note?

- [x]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No 

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [x] 🐹 Trivial/Minor

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-12-22 06:13:30 +00:00
ashmrtn
d3573d12dc
Allow updating portions of ItemInfo for some types (#1883)
## Description

Add helper functions that allows updating ItemInfo given a new path.Path. This is required because some items embed portions of the path in their details.

## Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No 

## Type of change

- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

* #1800 

## Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-12-22 00:09:40 +00:00
Vaibhav Kamra
a45aeda4a2
Update folder size and modified time in details (#1881)
## Description

Caches folder info added during details construction in the details builder and keeps the size/modified time
updated as newer items are added.

As part of this, this PR refactors the details package to separate out building `details.Details` from the in-memory representation and model.

## Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [x] 🕐 Yes, but in a later PR
- [ ]  No 

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #1850 

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-12-21 19:17:24 +00:00
Keepers
a2330bc314
add sharepoint output columns, fix flag bugs (#1749)
## Description

Extends the columns printed on sharePoint details
to include the expected data.  Fixes some bugs with
flag processing for details and restore commands.

## Type of change

- [x] 🌻 Feature
- [x] 🐛 Bugfix

## Issue(s)

* #1616

## Test Plan

- [x] 💪 Manual
- [x]  Unit test
2022-12-13 23:23:36 +00:00
Abin Simon
825cd6f21b
Add created and modified time to all (#1452)
## Description

Adds Created and Modified time to all objects. Need to go after
https://github.com/alcionai/corso/pull/1446 as it is built on top of
that. Keeping in draft until then.

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic
words" - "closes, fixes" to auto-close the Github issue. -->
* https://github.com/alcionai/corso/issues/1366

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E

Co-authored-by: aviator-app[bot] <48659329+aviator-app[bot]@users.noreply.github.com>
2022-11-15 10:24:06 +05:30
Keepers
11371f6e94
add sharepoint to path (#1465)
Adds the sharepoint service to /pkg/path.  Currently uses the
"Files" category for its category type, which is just a placeholder
for kicking off development.

Additionally, uncomments selector tests that were dependent
upon the path service declaration.
2022-11-14 15:35:57 -07:00
Abin Simon
5089f9d949
Add EndTime to Exchange events (#1446)
## Description

Add EndTime to Exchange events. Also fixes a testcase around start time.

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* https://github.com/alcionai/corso/issues/1366

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-11-07 09:28:27 +00:00
Abin Simon
935fe5cbec
Store/Retrieve owner info for OneDrive files (#1436)
## Description

Store and retrieve owner info for onedrive files.

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* https://github.com/alcionai/corso/issues/1366

## Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2022-11-04 07:01:45 +00:00
Keepers
f335fd8b5b
clean up cli outputs (#1271)
## Description

Remove old "restored..." confirmation line from the CLI. Renamed the "Reference" column header to "ID"  in the tabular output, which represents the shortRef aka item id.

## Type of change

- [x] 🐹 Trivial/Minor

## Issue(s)

* #1269

## Test Plan

- [x] 💪 Manual
- [x]  Unit test
2022-10-21 23:02:05 +00:00