420 Commits

Author SHA1 Message Date
Keepers
cbbc8d2f6c
drive-focused api refactoring prep (#3471)
setting up the drive api calls into files/spaces that will cascade naturally to the addition of an api client for users and sites.  contains some partial implementation of these clients, which will get completed in the next pr.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #1996

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-06-02 20:16:47 +00:00
Abin Simon
5c4d57b416
Parallelize restores within a collection for OneDrive (#3492)
This should massively speed up when restoring a collection with many
items. Will not impact much if we have a lot of collections with few
items each.

Numbers 🔢 :
- Restoring ~7000 files, mostly small, totaling 1.5GB
  - Sequential: ~70m
  - Parallel: ~50m
- Restoring 1200 50mb files
  - Sequential: 4h 45m
  - Parallel: <40m

---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* https://github.com/alcionai/corso/issues/3011
* closes https://github.com/alcionai/corso/issues/3536

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2023-06-02 04:56:11 +00:00
Keepers
cdf26b7988
refactor exchange restore to use interfaces (#3456)
refactors exchange restore from near-duplicate per-category functions and switch-based process trees with interfaces.

At the top of restoring all collections, each category creates a categoryRestoreHandler to supply the necessary restore behavior.  The appropriate handler gets passed in to the collection restore, and all restore code after that takes a single path using a common restore interface to switch between categorical behavior.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #1996

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-06-02 00:58:09 +00:00
ashmrtn
6fd453a699
Remove old code for finding bases (#3526)
Now that we're finding bases with a different component
remove the code that used to find bases since it's no
longer in use

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #3202

#### Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2023-06-01 15:41:16 +00:00
Keepers
2da554f00e
minor logging updates (#3546)
Co-authored-by: aviator-app[bot] <48659329+aviator-app[bot]@users.noreply.github.com>
2023-06-01 07:01:08 +00:00
ashmrtn
7cb01b1e27
Wrap new base finder and wire it up (#3524)
Take the new base finder code, wrap it in
a thin wrapper and use it in place of the
previous way of finding base snapshots

This solution is not focused on efficiency
as it does result in looking up backup
models multiple times. It's more to get the
new way of finding bases out and then we
can go back and smooth things over when
we have the chance

---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #3202

#### Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2023-05-31 18:27:15 +00:00
neha_gupta
06b04c6eff
update files count and total size on CLI for backup (#3480)
<!-- PR description-->
Currently the backup output show,
- total backup size - size of files + size of metafiles
- no of files - no of files + no of metafile
With this change the output of backup command shows size of only files and count include only file count.

NOTE: all current three services results will be impacted here.

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🐛 Bugfix

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* https://github.com/alcionai/corso/issues/3304

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ]  Unit test
- [x] 💚 E2E
2023-05-31 18:02:38 +00:00
neha_gupta
bb16dea4c2
don't throw error on backup with service disabled (#3426)
<!-- PR description-->

If a user does not have a service(like exchange) enabled- don't throw an error but just skip that user and move forward with other backups.

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🐛 Bugfix

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* https://github.com/alcionai/corso/issues/3392

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ]  Unit test
- [x] 💚 E2E
2023-05-29 06:24:15 +00:00
ashmrtn
9675c54c46
Add new code for finding bases for incremental backups (#3502)
Add a new set of functions to find
bases for incremental backups. The
new set of functions return
* set of backup models and the reasons for selecting each
* set of snapshots to merge and the reasons for selecting each
* set of snapshots for kopia-assisted incrementals and reasons for selecting each

New code is not wired into existing
system so no behavioral changes are
expected

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #3202

#### Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-05-27 02:56:06 +00:00
ashmrtn
264c1968ab
Move some interfaces to kopia package (#3500)
Searching for new bases is mostly kopia-specific but is going to require the store interface. With the way packages and interfaces are currently laid out it makes it impossible to easily get the interface added to the inject package without a cycle.

This moves functions that the kopia package provides into a subpackage of the kopia package that other things can then import

This change only moves code. No logic is updated

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #3202

#### Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-05-26 22:22:49 +00:00
Keepers
874e3c3a50
cleanup exchange api code, move out of support (#3455)
normalizes exchange api naming and client usage.
Also moves serialization and transformation to and from graph client objects out of graph/support and into the api. Some parts of graph/support have been moved into conn/ exchange instead, since they're more oriented around certain forms of object consumption rather than broad object serialization.

Except for different uses of api client graph servicers, all changes are movement/renaming.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #1996

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-05-25 23:53:11 +00:00
Keepers
7181e2ef90
move ad-hoc drive api calls into m365/api (#3451)
ensure that all drive-baased graph client calls exist in the m365/api package, not defined ad-hoc throughout the codebase.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #1996

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-05-24 21:45:07 +00:00
Keepers
1fb8a426dc
add test name to test context (#3484)
populate a "test_name" clues field in the
tester context, so that logs can be associated
with the calling test.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🤖 Supportability/Tests

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-05-24 17:48:36 +00:00
Keepers
e64f93a7d3
migrate exchange api to pkg/services/m365 (#3416)
No logic changes, just code movement.

Begins the process ofcentralizing the graph package in pkg/services/m365, instead of dividing up the funcs between different connector subpackages.  While the ownership of apis in each connector subpackage does make sense, this movement will reduce duplication, centralize import arrangements, and make it easy
to export usage of the api to sdk consumers.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #1996

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-05-17 00:21:20 +00:00
ashmrtn
089a96d437
Create CLI command for maintenance (#3226)
Creates hidden CLI command to run
maintenance.

Flag names and descriptions can be
updated if something else would fit
better

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #3077

#### Test Plan

- [x] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-05-16 21:03:06 +00:00
Abin Simon
67e055360d
Bump msgraph deps to 1.0.0 (#3345)
Bumps ms graph and all dependent libs to 1.0.0

*Still can't drop betasdk as `models.SitePageable` is still not available.*

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* Closes https://github.com/alcionai/corso/pull/3327
* Closes https://github.com/alcionai/corso/pull/3326
* Closes https://github.com/alcionai/corso/pull/3324
* Closes https://github.com/alcionai/corso/pull/3323
* Closes https://github.com/alcionai/corso/pull/3322
* Closes https://github.com/alcionai/corso/pull/3321
* Closes https://github.com/alcionai/corso/pull/3320
* Closes https://github.com/alcionai/corso/issues/3354

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2023-05-16 17:44:16 +00:00
Keepers
97ed97c1c3
logic change for sharepoint permission support (#3333)
adds logical changes required for sharepoint permission handling, both backup and restore.

---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included

#### Type of change

- [x] 🌻 Feature

#### Issue(s)

* #3135

#### Test Plan

- [x] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2023-05-15 21:03:00 +00:00
Keepers
60f6d4a035
renaming/cleanup for sharepoint perms pt.1 (#3330)
No logic changes, just renaming and minor cleanup.

PRs to follow:
1. collect various maps in onedrive collections into a single cache.
2. logic changes and tests to produce sharepoint permissions backup/restore e2e.
3. extend permission identity-type retention.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #3135

#### Test Plan

- [x]  Unit test
2023-05-12 21:23:52 +00:00
Keepers
695f8060da
insert restoreEnd event into deferred func (#3400)
move the end-of-op notification event to a defer within backup/restore.Do(). This ensures we'll get the end event data even in case of failure.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🐛 Bugfix
- [x] 🤖 Supportability/Tests

#### Issue(s)

* #3388

#### Test Plan
2023-05-12 19:18:53 +00:00
Abin Simon
f8aa37b822
Add non delta pagers to exchange (#3212)
When the user's mailbox is full, we cannot make use of delta apis. This adds initial changes needed to create separate delta and non delta pagers for all of exchange.

*I would suggest looking commit wise when reviewing the PR.*

---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-05-12 15:56:49 +00:00
Keepers
1b417af5bb
minor updates to the path.Path interface (#3384)
Swaps path.Append from a single item method to accept a variadic list of string elements.  Since 95% of all calls to path.Append were items, also adds a shorthand
AppendItem func to the interface for easy clarity. Finally, adds a Last() method to elements for getting the last element in the slice.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-05-12 01:29:17 +00:00
ashmrtn
674d3eec91
Switch to folder IDs in Exchange (#3373)
Store all folders from Exchange by folder ID
in kopia. Remove the old duplicate folder
name stop-gap measure as well since it's no
longer required

Update tests to check LocationPath instead
of FullPath since LocationPath still has
display name info

---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

- [x] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

* closes #3197

#### Test Plan

- [x] 💪 Manual
- [ ]  Unit test
- [x] 💚 E2E
2023-05-11 23:07:38 +00:00
Keepers
6c410c298c
consts for drives, root: (#3385)
#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-05-11 22:46:51 +00:00
Keepers
caea3ab6da
make the linter happy (#3394)
#### Type of change

- [x] 🧹 Tech Debt/Cleanup
2023-05-11 20:38:33 +00:00
ashmrtn
2e4fc71310
Add restore path generation code (#3362)
In preparation for switching to folder IDs,
add logic to generate the restore path based
on prefix information from the RepoRef and
LocationRef of items

Contains fallback code (and tests) to handle
older details versions that may not have had
LocationRef

Manually tested restore from old backup that
didn't have any LocationRef information

Manually tested restore checking that
calendar names are shown instead of IDs in
progress bar

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #3197
* fixes #3218

#### Test Plan

- [x] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-05-11 02:49:32 +00:00
Keepers
c5b388a721
add indeets test helper, implement in exchange op (#3295)
Adds a helper for building expected details entries and checking
them after a backup.  Implements the helper in the exchange backup
tests in operations/backup integration.  Will follow with a onedrive
implementation.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🤖 Supportability/Tests

#### Issue(s)

* #3240

#### Test Plan

- [x] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2023-05-11 02:28:18 +00:00
Keepers
c0725b9cf9
some quick logging and error naming updates (#3348)
#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #3344

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-05-11 02:03:31 +00:00
Keepers
833216c8ae
normalize folders restored in testing (#3363)
Normalize all test folders to use a constant prefix: Corso_Test.  This cleans up and centralizes all per- test variations on the restore destination.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🤖 Supportability/Tests

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-05-11 01:02:30 +00:00
ashmrtn
b508ea3b72
Update kopia restore logic to take a (RepoRef, Collection Restore Path) pair (#3337)
Begin expanding the restore logic to take a pair of
paths, one denoting the precise location of the item
in kopia and the other denoting the "restore location"
or path the item should be placed at during restore

This PR is not expected to change system functionality
at all

This is the first of 2 PRs to setup all the logic for
this. This PR does not handle properly merging
together multiple collections that have the same
restore location but different RepoRefs due to recent
updates to the kopia wrapper restore logic

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #3197

#### Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-05-10 01:50:38 +00:00
Keepers
3a2d0876dd
consolidate aggregation of parent-item exclude map (#3258)
introduces a new type wrapping a nested map so that aggregation of globally excluded items in driveish services don't need to manage the map updates themselves.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #2340

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-05-04 19:32:47 +00:00
Keepers
d067a79483
move permissions into metadata (#3216)
Moves the UserPermissions struct into onedrive/metadata as
the Permission struct, for better shared support across onedrive,
sharepoint, and any other packages that reference it.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #3135

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-05-03 18:52:25 +00:00
Keepers
c152ce8b25
move and rename common/time.go (#3215)
moves common/time.go to common/dttm/dttm.go, and renames many of the consts and funcs with names that better explain their design and intent of usage.

Only code movement, no logical changes.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-05-02 19:39:29 +00:00
ashmrtn
8cca7f12df
Create basic maintenance operation (#3225)
Add a maintenance operation to run
kopia maintenance

Using this instead of calling kopia
directly will allow us to hook into
metrics reporting in a more
consistent manner if we want metrics

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #3077

#### Test Plan

- [ ] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-05-02 18:18:40 +00:00
ashmrtn
b6ef2d5266
Create wrapper around kopia maintenance (#3224)
Create a wrapper function around the kopia
maintenance operation. Will allow users to
run maintenance, though they still need to
be careful not to run maintenance
concurrently if they override the username
and hostname that kopia uses

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #3077

#### Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-05-02 00:52:15 +00:00
Keepers
3b9d2841d4
rename DetailsEntry to Entry to avoid stuttering (#3265)
#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-05-01 19:29:06 +00:00
ashmrtn
6ac8c9f331
Allow overriding the default username/hostname in kopia (#3223)
This allows setting the username/hostname
kopia will use for maintenance. This
information is recorded in the kopia config
file during connect and reused during open

If no values are given, kopia pulls the
values from the OS

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #3077

#### Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-04-27 22:47:35 +00:00
Abin Simon
534924f696
Don't try to fetch backup version if no compete backups available (#3230)
Regression from 62daf10213 (diff-f088d3a5f56348ddaf6fbf485c2aeaa6d8b40a860de014f7084db6daf0753829R267)


---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-04-26 19:15:56 +00:00
Keepers
edef23bfc4
errant cleanups before incremental support (#3201)
a handful of errant cleanups to help the PR for adding incremental support to sharepoint.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #3136

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-04-25 21:09:41 +00:00
Keepers
0ae1a04097
use nanosecond folders for testing (#3205)
we've been failing on folder name collision, particularly in this package.  Probably because it's using second-granularity folder names.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🐛 Bugfix
- [x] 🤖 Supportability/Tests

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-04-25 20:33:39 +00:00
Keepers
7b1ce22a64
enable user pn->id migrations for reporef prefix (#3200)
Enables migrations of all user-based repoRef prefixes away from the user's PrincipalName and onto the user's ID for a more stable reference.

---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included

#### Type of change

- [x] 🌻 Feature

#### Issue(s)

* #2825

#### Test Plan

- [x] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2023-04-24 22:37:11 +00:00
Keepers
62daf10213
migrate onedrive using prefix collection (#3122)
#### Does this PR need a docs update or release note?

- [x] 🕐 Yes, but in a later PR

#### Type of change

- [x] 🌻 Feature

#### Issue(s)

* #2825

#### Test Plan

- [x] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2023-04-24 17:36:10 +00:00
Keepers
6405c8246e
move parallelism logs/checks upstream (#3003)
The fetch paralellism checks and logs occur on
every item streamed from GC.  This is a bit chatty,
and has been moved upstream in the process for
a more centralized behavior.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Test Plan

- [x] 💪 Manual
- [x]  Unit test
2023-04-21 20:23:41 +00:00
Keepers
cfe8d576b8
cleaning up some gc/selector test construction (#3175)
Duplicate of https://github.com/alcionai/corso/pull/3049, fixing a git rebase screwup.
2023-04-21 19:50:31 +00:00
Keepers
897c0f8a07
fix the file/line output on panic recovery (#3190)
#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🐛 Bugfix

#### Test Plan

- [x] 💪 Manual
- [x]  Unit test
2023-04-21 15:53:10 +00:00
Keepers
306db14339
add itemref to details entry (#3160)
Adds an new details entry field: itemRef.
This holds a stable, semi-unique identifier to the item represented by that entry.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🌻 Feature

#### Issue(s)

* #3027

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-04-19 23:25:11 +00:00
ashmrtn
f01e25ad83
Redo folder generation in backup details (#3140)
Augment the folder backup details entries with
LocationRef and some information about what
data type generated the entry. Also add top-level
container information like drive name/ID if
applicable

Refactor code to do folder generation to make it
more contained and require less parameters to add
entries to backup details

Important changes are in details.go. All other
changes are just to keep up with the slightly
modified function API or update tests

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

* closes #3120
* closes #2138

#### Test Plan

- [x] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-04-19 20:27:55 +00:00
ashmrtn
ba724ac63d
Add immutable ID header to Get requests (#3157)
This allows using immutable IDs for Exchange objects. It does not add this header to put requests as we don't record the IDs of restored objects anywhere.

May slightly increase performance of events backups as
it now sets the delta query page size to 200 like with
other requests

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #2861

#### Test Plan

- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-04-19 16:16:49 +00:00
Keepers
d5eee4479d
move largeItem client constructor into new GC (#3130)
It's an unnecessary paramter.  Plus, this sets up the next pr to have a smaller footprint.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #3129

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-04-18 18:52:22 +00:00
Keepers
86f9d60bf7
relocate connector mocks (#3048)
It makes more sense for mock data types to be
owned by the service package, not by the connector package as a whole.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-04-17 17:54:19 +00:00
ashmrtn
1826b66e8d
Populate and persist OneDrive LocationRef (#3111)
Leverage structs from previous PRs to
populate and persist the LocationRef
for OneDrive

Right now the source of the
LocationRef is still based off the
RepoRef path, but this is in the
OneDrive code only

Manually tested
* updating a subfolder path when parent folder was moved
* populating LocationRef when the base backup didn't have LocationRefs when nothing was changed in OneDrive
* populating LocationRef when the base backup didn't have LocationRefs when something was changed in OneDrive

---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

* closes #2486

#### Test Plan

- [x] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-04-14 21:31:16 +00:00