330 Commits

Author SHA1 Message Date
ryanfkeepers
e298881a6c add chat members to the backup
lazily fetch the chat members and add them to the chat
during backup item downloads.
2024-01-24 16:20:15 -07:00
ryanfkeepers
14817de592 get details working (and other cleanup)
details wasn't properly listing backed up items.  This fixes
the details display, and contains some code clean-up that occurred
along the way.
2024-01-24 15:17:14 -07:00
ryanfkeepers
4cba77343a lazily fetch messages in chat
once we're past kopia assits, the chat can
download all of its messages and store them in the body
uploded to kopia.
2024-01-24 12:48:13 -07:00
ryanfkeepers
94b02ed6f3 fix item filtering, remove prefetch coll 2024-01-24 12:47:38 -07:00
ryanfkeepers
7e3d5d2074 fixes for running backup chats from the cli
Some necessary changes to get a green backup creation from the cli.
2024-01-23 17:01:24 -07:00
ryanfkeepers
7d9d0e04cc add teamschats to m365 backup
tests will arrive in a later PR
2024-01-23 14:34:43 -07:00
ryanfkeepers
9831fb8f91 fix package name 2024-01-23 13:16:20 -07:00
ryanfkeepers
db14816ac8 add teamschat boilerplate to m365/service 2024-01-22 17:19:42 -07:00
ryanfkeepers
a17ed85220 boilerplate teamschat collection package
seems like a lot of code, but this is 95% boilerplate additions copied
from other similar collections packages.  Some noteworthy differences:
* teamsChats does not handle metadata, so all metadata, delta, and previous
path handling was removed
* teamsChats does not produce tombstones
* chats are never deleted, so no "removed" items are tracked
* all chats gets stored at the prefix root, so no "containers" are iterated, and therefore only one collection is ever produced.

This means that, overall, the boilerplate here is much reduced compared to
boilerplate in similar packages.
2024-01-22 16:46:53 -07:00
ashmrtn
b8b1299514
Cleanup immutable ID config passing (#5013)
This value was present in the control.Options in the struct and
passed as a parameter. This PR removes the parameter so it's clear
the control.Options one is used

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #5012

#### Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2024-01-17 00:51:11 +00:00
Keepers
6e50d5216e
remove builder.toServicePath from paths (#5028)
path.Build produces the same result with more standard and centralized behavior.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #4025

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2024-01-16 16:42:49 +00:00
Abin Simon
6beb774ba8
Enable events export (#5015)
<!-- PR description-->

---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* closes https://github.com/alcionai/corso/issues/3890

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2024-01-12 20:06:50 +00:00
Keepers
1694581e26
make a not found core error (#4975)
make a canonical 'not found' error for our core sentinels. This replaces the graph.DeletedInFlight error, which acted as an interpretation of "not found".

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #4685

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2024-01-12 01:29:39 +00:00
Keepers
279a6dab3c
normalize observe usage, ensure deferred closers (#5006)
some housecleaning before reviewing deadlock issues

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Test Plan

- [x]  Unit test
2024-01-11 22:42:10 +00:00
Hitesh Pattanayak
ad783172b7
refactors post list to accept deseriallized stored list (#5008)
refactors post list to accept deseriallized stored list

#### Does this PR need a docs update or release note?
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)
#4754 

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2024-01-11 13:52:48 +00:00
Keepers
b1af13b7b6
centralize core err interpretation (#4972)
now that we have a core error library, we can start swapping out the percolation of graph api errors with core error sentinels.  This PR begins that process by adding a centralized transformer to the graph service handlers that ensures all calls produce the same set of core errors according to some standard categorizations. This is just an initial step, and does not transfer all graph errors into core sentinels.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🤖 Supportability/Tests

#### Issue(s)

* #4685 

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2024-01-10 17:25:25 +00:00
ashmrtn
584fc21d36
Fixup a few calls for adding clues to errors (#4989)
Called functions add clues so there's no need to add them ourselves.

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

#### Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2024-01-08 23:04:35 +00:00
Keepers
af64b487ac
swap pkg/metrics to threadsafe map (#4899)
<!-- PR description-->

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🐛 Bugfix
- [x] 🧹 Tech Debt/Cleanup

#### Test Plan

- [x] 💚 E2E
2024-01-08 20:22:54 +00:00
ashmrtn
443aed639c
Extra logging and clues context for folder caches (#4988)
Add extra logging and clues to errors when populating container caches for Exchange. This could help track down some things related to time spent getting data.

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Test Plan

- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2024-01-08 19:37:02 +00:00
Abin Simon
ca50b1e908
Filter out non existent entities in permissions and link shares (#4955)
<!-- PR description-->

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2024-01-08 13:44:25 +00:00
Abin Simon
a52b085cea
Update handling of permissions restore failures (#4954)
- All permissions failures are now recoverable to the granularity of a single permission or link share
- Failure to delete child permissions because the permission was not restored on parent will now not produce any warnings

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2024-01-08 10:32:41 +00:00
Hitesh Pattanayak
87104ce404
exports sharepoint lists (#4959)
provision to export sharepoint lists

#### Does this PR need a docs update or release note?
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature

#### Issue(s)
#4752 

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2024-01-08 05:57:32 +00:00
Hitesh Pattanayak
d113fa3926
store lists previouspath (#4938)
saves previous path for lists in storage. 

#### Does this PR need a docs update or release note?
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature

#### Issue(s)
#4754 
#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2024-01-04 13:49:33 +00:00
Keepers
971d874462
introduce errs/core (#4897)
first step towards having a centralized set of error sentinels that can be passed around corso instead of re-using low level error sentinels like those found in the graph/errors.go file.

This PR works as a standalone, and only handles the lowest hanging fruit. SDK consumers will need to change their error enum references, but all api behavior should remain the same.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🤖 Supportability/Tests

#### Issue(s)

* #4685

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2024-01-03 18:13:41 +00:00
Abin Simon
b47fa8b14f
Ensure that we propagate errors if available (#4964)
<!-- PR description-->

Fixed all cases where we missed propagating errors. Also added a lint rule in CI to catch it.

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2024-01-03 15:38:46 +00:00
Abin Simon
b49c124512
Events export as ICS (#4958)
There are a few more pending items. I've added TODO entries for them. The PR was getting pretty big and so I thought I would address them in a followup PR.

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* https://github.com/alcionai/corso/issues/3890

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2024-01-03 10:44:28 +00:00
Hitesh Pattanayak
c87533a266
fix tests referencing top level of sharepointInfo for listInfo (#4936)
fix tests referencing top level of SharepointInfo for list details instead of nested ListInfo.
also disables a test that requires list backup to be enabled in CLI level.

#### Does this PR need a docs update or release note?
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🐛 Bugfix

#### Issue(s)
https://github.com/alcionai/corso/actions/runs/7312375792/job/19922878417#step:6:666

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2024-01-03 08:48:44 +00:00
Hitesh Pattanayak
7a8c96964c
ignores accessRequest list templates from backup and restore (#4957)
ignores accessRequest list templates from backup and restore

#### Does this PR need a docs update or release note?
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature

#### Issue(s)
#4754 

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2024-01-03 08:08:26 +00:00
Abin Simon
dcff3056ff
Contacts export for exchange (#4883)
<!-- PR description-->

---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* closes https://github.com/alcionai/corso/issues/3891

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2024-01-03 05:22:54 +00:00
Hitesh Pattanayak
13f549c6cb
lists prefetech (#4943)
refactors sharepoint lists collection by:
- renaming `Collection` interface as `preFetchCollection` interface
- making `finishPopulation` method as independent function 
- seralizing list using `serializeContent` function

#### Does this PR need a docs update or release note?
- [x]  No

#### Type of change
- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)
#4754 

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2023-12-30 15:08:29 +00:00
Hitesh Pattanayak
823fcdc559
skips lists with certain templates from being backed-up (#4931)
skips documentLibrary templated lists from:
- restoration
- details

but the lists with 'documentLibrary' are backed-up.
this is in addition to 'webTemplateExtensionsList' templated lists which were previously skipped from restore.

the `skipping` of lists with template 'documentLibrary' is temporary and to be enabled with correct handling.

**update:27/12/2023**
skips `documentLibrary`, `sharingLinks`, `webTemplateExtensionsList` templated lists from being backed-up altogether


site contents of `https://10rqc2.sharepoint.com/sites/CorsoCI`:
![Screenshot from 2023-12-23 20-20-45](https://github.com/alcionai/corso/assets/48874082/c31bf233-57c9-497d-bb59-2dab0ffc5dab)

backup details
```
./corso backup details sharepoint --backup 5e0a13b0-bff0-471b-8860-2aec869717b2
Logging to file: /home/hitesh/.cache/corso/logs/2023-12-23T14-52-37Z.log
Connecting to M365                               done 
Connecting to repository                     30s done 
  ID            List                                                     Items  Created               Modified            
  c7ffe2f0a275  Test List                                                0      2023-05-12T12:28:51Z  2023-05-12T12:28:51Z
  5345df87b565  Meh                                                      0      2023-05-12T12:59:50Z  2023-05-12T12:59:50Z
  cd2854591ab4  corso-restore-list_135b9067-0e19-4a5c-9475-ced51730c211  0      2023-12-11T10:25:27Z  2023-12-23T13:19:24Z
  0f56b7f450fa  corsorestorelist_newlist                                 0      2023-12-11T10:31:54Z  2023-12-11T10:31:54Z
  3a7be2fb9d50  new list                                                 0      2023-11-29T18:29:07Z  2023-11-29T18:29:07Z
  edab0c775e5e  New List Name                                            0      2023-05-12T12:39:06Z  2023-12-23T13:25:39Z
  4d7ad9b1df98  MockListing                                              0      2023-05-12T12:15:14Z  2023-12-14T06:22:36Z
  d2bdca596f75  integration-test-list                                    6      2023-11-30T06:57:01Z  2023-12-14T06:09:45Z
  c9eccc9771a3  Sharing Links                                            74     2023-06-26T07:21:31Z  2023-12-14T06:06:31Z
  5e79dcd93c92  New List Name 2                                          0      2023-05-12T12:40:15Z  2023-05-12T12:40:15Z
  6a6acbaa0d10  corsorestorelist_7f99d792-cae0-4a87-9e68-fd930f5b123e    0      2023-12-11T10:28:25Z  2023-12-23T13:19:57Z
```

restore details
```
./corso restore sharepoint --backup 5e0a13b0-bff0-471b-8860-2aec869717b2
Logging to file: /home/hitesh/.cache/corso/logs/2023-12-23T14-53-37Z.log
Connecting to M365                               done 
Connecting to repository                     26s done 
Restoring to folder Corso_Restore_23-Dec-2023_14-54-05
  
Restoring ∙ https://10rqc2.sharepoint.com/sites/CorsoCI              
Discovered 11 items in backup 5e0a13b0-bff0-471b-8860-2aec869717b2 to restore   
Enumerating items in repository             1m2s done 
Restoring data                               50s done 
Restore Complete
Restored 11 items
  ID            List                                                                                        Items  Created               Modified            
  edab0c775e5e  Corso_Restore_23-Dec-2023_14-54-05_New List Name                                            0      2023-12-23T14:54:23Z  2023-12-23T14:54:23Z
  cd2854591ab4  Corso_Restore_23-Dec-2023_14-54-05_corso-restore-list_135b9067-0e19-4a5c-9475-ced51730c211  0      2023-12-23T14:54:25Z  2023-12-23T14:54:25Z
  5345df87b565  Corso_Restore_23-Dec-2023_14-54-05_Meh                                                      0      2023-12-23T14:54:26Z  2023-12-23T14:54:26Z
  4d7ad9b1df98  Corso_Restore_23-Dec-2023_14-54-05_MockListing                                              0      2023-12-23T14:54:28Z  2023-12-23T14:54:28Z
  d2bdca596f75  Corso_Restore_23-Dec-2023_14-54-05_integration-test-list                                    6      2023-12-23T14:54:29Z  2023-12-23T14:54:31Z
  c7ffe2f0a275  Corso_Restore_23-Dec-2023_14-54-05_Test List                                                0      2023-12-23T14:54:34Z  2023-12-23T14:54:34Z
  c9eccc9771a3  Corso_Restore_23-Dec-2023_14-54-05_Sharing Links                                            74     2023-12-23T14:54:36Z  2023-12-23T14:54:36Z
  5e79dcd93c92  Corso_Restore_23-Dec-2023_14-54-05_New List Name 2                                          0      2023-12-23T14:55:09Z  2023-12-23T14:55:09Z
  6a6acbaa0d10  Corso_Restore_23-Dec-2023_14-54-05_corsorestorelist_7f99d792-cae0-4a87-9e68-fd930f5b123e    0      2023-12-23T14:55:10Z  2023-12-23T14:55:10Z
  0f56b7f450fa  Corso_Restore_23-Dec-2023_14-54-05_corsorestorelist_newlist                                 0      2023-12-23T14:55:11Z  2023-12-23T14:55:11Z
```


#### Does this PR need a docs update or release note?
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🐛 Bugfix

#### Issue(s)
#4754 

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2023-12-28 16:59:23 +00:00
Hitesh Pattanayak
66664b69e5
skips backup of non-recoverable list templates (#4940)
Skips backup of non-recoverable list templates.

#### Does this PR need a docs update or release note?
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)
#4754 

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2023-12-27 09:59:10 +00:00
Abhishek Pandey
c6b9615c7d
Integrate lazy item reader into conversations backup (#4921)
<!-- PR description-->




---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature


#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* Closes https://github.com/alcionai/corso/issues/4862

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [x] 💚 E2E
2023-12-27 07:09:12 +00:00
Abhishek Pandey
abc5ec1942
Skip group mailbox items that were deleted in flight (#4939)
<!-- PR description-->

Discovered an issue with how we were handling items deleted in flight with the lazy item reader. This needs to go in before integration PR. https://github.com/alcionai/corso/pull/4921 

Here are the details.

**Orig:**
* For such items, we were using the exchange approach to persist an empty data file (with serialized version) and not persist it in details file.
* Exchange discovers the deleted items during next backup during delta query and excludes them.

**Problem:**
* Group mailbox doesn't support delta queries. Also, there is no graph API to discover/fetch deleted items.
* So if we use the exchange lazy reader approach like above, the ongoing backup will succeed. However, the next incremental backup fails during details merge with ` running backup: merging details: incomplete migration of backup details`. 
* Failure is [here](c58cd9302f/src/internal/operations/backup.go (L837)). It's because the deleted item reporefs are still present in the merge base. Since there is no delta query, there is no `GetDeleted()` to exclude the item.

**Mod:**
* Using the approach drive lazy reader code uses to mark an item as skipped, so that kopia doesn't error out during upload.

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* https://github.com/alcionai/corso/issues/4862

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-12-27 05:30:57 +00:00
ashmrtn
aa876cabb9
Sanitize JSON input if it's invalid on the first attempt (#4928)
Add code and tests for sanitizing emails

Also adds code for sanitizing events and contacts but we can remove
that if we'd like to address it if/when needed. We haven't yet seen
or tried to create instances of those

---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-12-26 20:26:07 +00:00
ashmrtn
f63a6e9b4f
Some testing QoL functions (#4927)
Functions help check things like
  * are two items equal when taking into account nil, empty slices, and
    other zero-values?
  * is something fully populated?

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Test Plan

- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-12-26 18:52:50 +00:00
Hitesh Pattanayak
1c18131122
replaces deleteList with errors fault bus (#4933)
replaces list deletion, on creation of list items failure, with fault bus. Thereby restores list with list items that could be restored.

#### Does this PR need a docs update or release note?
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)
#4754 

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2023-12-26 12:29:40 +00:00
Abhishek Pandey
f5f65f0cfe
Add lazy item reader for groups (#4913)
<!-- PR description-->

Lazy item reader similar to exchange. Tests are also inspired from exchange.

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* https://github.com/alcionai/corso/issues/4862

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-12-23 02:31:05 +00:00
Abhishek Pandey
1940d3a1f1
Minor refactor of status update code (#4912)
<!-- PR description-->

I missed this change in https://github.com/alcionai/corso/pull/4906. Moving out the status updater code out of `prefetchCollection` scope so that `lazyFetchCollection` can also use it in upcoming PRs.


---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-12-23 01:15:48 +00:00
Abhishek Pandey
127570953a
Add prefetch collection for groups (#4906)
<!-- PR description-->

Minor refactor before introducing `lazyFetchCollection` for groups. We'll utilize `lazyFetchCollection` for group mailboxes and will continue to use `prefetchCollection` for channels.


---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* https://github.com/alcionai/corso/issues/4862

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-12-22 23:18:58 +00:00
Hitesh Pattanayak
a1590e0d2f
enables sharepoint to use lists restore handler for lists ops (#4923)
enables sharepoint to use lists restore handler for lists ops
Changes previously approved in: 
- https://github.com/alcionai/corso/pull/4854
- https://github.com/alcionai/corso/pull/4910

#### Does this PR need a docs update or release note?
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature

#### Issue(s)
#4754 

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2023-12-22 17:18:24 +00:00
Hitesh Pattanayak
dc47001cba
enables sharepoint to use lists backup handler for lists ops (#4920)
enables sharepoint to use lists backup handler for lists ops
Changes previously approved in:
-  https://github.com/alcionai/corso/pull/4786
- https://github.com/alcionai/corso/pull/4787
- https://github.com/alcionai/corso/pull/4909

#### Does this PR need a docs update or release note?
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature

#### Issue(s)
#4754 

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2023-12-22 14:26:23 +00:00
Hitesh Pattanayak
00662c4cd9
adds list backup and restore handlers (#4919)
adds list backup and restore handlers
Changes previously approved in:
- https://github.com/alcionai/corso/pull/4786
- https://github.com/alcionai/corso/pull/4854

#### Does this PR need a docs update or release note?
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature

#### Issue(s)
#4754 

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2023-12-22 09:59:19 +00:00
Hitesh Pattanayak
1270e64637
adds list client apis and pagers (#4917)
adds list client apis and pagers to fetch, create and delete lists and list relations.
Changes previously approved in PRs:
- https://github.com/alcionai/corso/pull/4785
- https://github.com/alcionai/corso/pull/4815
- https://github.com/alcionai/corso/pull/4852

#### Does this PR need a docs update or release note?
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature

#### Issue(s)
#4754 

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2023-12-22 08:02:32 +00:00
Hitesh Pattanayak
50fd28aab2
adds ListInfo details to SharepointInfo details (#4916)
adds ListInfo details to SharepointInfo details
Changes previously approved in PR: https://github.com/alcionai/corso/pull/4851

#### Does this PR need a docs update or release note?
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature

#### Issue(s)
#4754 

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2023-12-22 06:50:34 +00:00
Abin Simon
c3b15a00bd
Note about anonymous link shares (#4905)
<!-- PR description-->

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-12-22 06:13:28 +00:00
ashmrtn
c65527e5c4
Log and ignore sharing disabled errors when doing permissions restores (#4901)
We've encountered an error where the user previously had sharing for
one or more external users on an item and then did a backup. They
subsequently disabled sharing for the SharePoint site in either the
tenant-wide config or for the site in particular. Once sharing was
disabled, they attempted a restore which subsequently failed while
trying to restore sharing for the external user

This PR logs and ignores errors about sharing being disabled when
doing a restore

This has been tested manually

---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Test Plan

- [x] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-12-21 19:28:55 +00:00
Keepers
753ed1a075
use tree version in nightly tests (#4885)
#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🤖 Supportability/Tests

#### Issue(s)

* #4689

#### Test Plan

- [x] 💚 E2E
2023-12-21 18:44:09 +00:00
ashmrtn
1944c070cf
Beef up Exchange export error handling (#4898)
Add log statements when errors are encountered so we get full clues
output and make sure context clues are added to returned errors. The
additional logging is necessary because not all corso SDK consumers
will use clues which means they could miss out on valuable information
if they just log the returned errors normally

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Test Plan

- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-12-21 16:40:47 +00:00
ashmrtn
e46119a818
Remove options form service handler initializer (#4895)
Now that optiosn are being broken up a bit more passing a struct of a single type to the service handler initializer doesn't make much sense. Just remove the parameter since it was unused anyway.

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

#### Issues

- #4896

#### Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-12-20 18:49:28 +00:00