15 Commits

Author SHA1 Message Date
ashmrtn
74cf0ab737
Create service specific handlers that know how to run an export (#4491)
First step in reducing the number of places we have to check the service type manually. Create a way to get a handle to a service specific handler and implement exports for those handlers

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #4254

#### Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2023-11-01 15:55:30 +00:00
Keepers
5e0014307c
remove ctx embedding for counter (#4497)
#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-10-31 16:15:05 +00:00
ashmrtn
09531de08c
Cleanup API for BackupBases (#4471)
Update the BackupBases API to return BackupBase
structs that contain both the snapshot and backup
model instead of having separate functions for
each. Minor logic updates to accommodate these
changes

This PR also updates tests and mock code for the
new API

Suggest viewing by commit

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* closes #3943

#### Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2023-10-26 18:38:14 +00:00
Abin Simon
040257f8be
Add stats to export operations (#4461)
Have a way to gather stats about the exported data.

Users can now call `ExportOperation.GetStats()` at the end of the run to get the stats for the operations. The data will be in the format `map[path.CategoryType]data.KindStats` whre `KindStats` is:

```go
type KindStats struct {
	BytesRead     int64
	ResourceCount int64
}
```
---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* https://github.com/alcionai/corso/issues/4311

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-10-11 05:54:17 +00:00
Keepers
c88b5764a9
hand resource down to drive controller (#4436)
hands the backup resource into the drive collection for the handler to use to record as the siteID

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🐛 Bugfix

#### Issue(s)

* #3988

#### Test Plan

- [x] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2023-10-05 16:38:17 +00:00
Abin Simon
6e1be1f08c
Incrementals for Groups SharePoint backup (#4205)
<!-- PR description-->

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* https://github.com/alcionai/corso/issues/3990

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-09-14 06:54:39 +00:00
Keepers
9a8c413b52
some code cleanup before groups export (#4202)
various tidbits of data cleanup before moving forward with adding export behavior to groups.
* move duplicate collections mocks into data/mock
* move the export collection struct into pkg/export (to prevent future duplicates in the next PR)
* rename export.Collection to Collectioner, because it's an interface.
* some other non-logic rearrangement

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #3991 

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-09-12 22:35:03 +00:00
Abhishek Pandey
59bf350603
Add service isolation (#4096)
<!-- PR description-->

1. Renamed `IsBackupRunnable` to `IsServiceEnabled` & extended to restore operation. Removed `checkServiceEnabled`. Reasons:
- The above 2 functions were doing the same thing. Removed `checkServiceEnabled` in favor of `IsBackupRunnable` since we want this check to be as soon as possible during backup/restore op initialization
- Renamed `IsBackupRunnable` to `IsServiceEnabled`, because a) we are only doing service enabled checks right now, b) common code that can be used for both restore & backup.

2. Wire corso code to use new helpers in internal/m365/common.go
3. Note: SDK wiring and related integ tests will be added in a follow up PR

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* https://github.com/alcionai/corso/issues/3844

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-09-06 17:15:40 +00:00
Keepers
fc6119064b
introduce backupProducerConfig (#3904)
Adds another inject container-of-things to hold
common properties used by backup collection producers.

No logic changes, just code movement, renames, and placing things into structs.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-08-02 00:38:19 +00:00
Keepers
a973f69d18
rename export interface (#3916)
to better match current standards

#### Type of change

- [x] 🧹 Tech Debt/Cleanup
2023-07-27 06:29:56 +00:00
Keepers
3613cb2aa0
look up restore resource if specified (#3853)
If the restore configuration specifies a protected resource as a restore target, use that as the destination for the restore.  First step is to ensure the provided target can be retrieved and identified.

---

#### Does this PR need a docs update or release note?

- [x] 🕐 Yes, but in a later PR

#### Type of change

- [x] 🌻 Feature

#### Issue(s)

* #3562

#### Test Plan

- [x]  Unit test
2023-07-26 17:24:53 +00:00
Abin Simon
f6d44b3c60
Export data from OneDrive (#3819)
Core logic for exporting data from OneDrive

Next: https://github.com/alcionai/corso/pull/3820

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [x] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* https://github.com/alcionai/corso/issues/3670
* https://github.com/alcionai/corso/pull/3797

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-07-26 12:30:39 +00:00
Keepers
9359679f99
add api funcs for creating documentLibs (#3793)
Adds api handlers for creating document libraries in sharepoint. This is the first step in allowing us to restore drives that were deleted between backup and restore.

---

#### Does this PR need a docs update or release note?

- [x] 🕐 Yes, but in a later PR

#### Type of change

- [x] 🌻 Feature

#### Issue(s)

* #3562

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-07-20 21:05:30 +00:00
Keepers
2a150cc610
add counter for skip-collision counting (#3722)
Introduces a counting bus, and threads it into restore operations so that we can count the number of
collision skips that occur.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🌻 Feature

#### Issue(s)

* #3562

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-07-06 19:22:02 +00:00
Keepers
2f6d731993
rename connector -> m365 (#3600)
renames /internal/connector to /internal/m365.  No logic changes in this PR.  Only the dir rename, import renames, and one linter shadowing rename.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #1996

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-06-13 18:35:39 +00:00