1579 Commits

Author SHA1 Message Date
Abhishek Pandey
29068c13ea One last adustment 2024-02-11 11:44:18 -08:00
Abhishek Pandey
afb8de0be2 Fix indentation a bit 2024-02-11 11:42:10 -08:00
Abhishek Pandey
790e95a26c Handle item attachments with missing name 2024-02-11 11:36:00 -08:00
Vaibhav Kamra
b3b52c0dfc
Do not backup shared calendars (#5207)
Skip backup of shared calendars. These will be backed up with the resource that owns the calendar.

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [x] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2024-02-11 18:58:09 +00:00
Abin Simon
8502e1fee6
Use recurrence timezone for ics exports (#5206)
<!-- PR description-->

---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2024-02-10 08:50:38 +00:00
Keepers
45886e2ad9
allow eml export when attachments have no name (#5199)
#### Does this PR need a docs update or release note?

- [x]  Yes, it's included

#### Type of change

- [x] 🐛 Bugfix

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2024-02-09 18:41:10 +00:00
Keepers
f00dd0f88a
introduce intg and unit test setup packages (#5140)
corso has a thousand bespoke approaches to setting the same info in all of its tests.  This is a first step towards minimizing and standardizing the lift around that work. Future PRs will distribute these packages through the repo.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🤖 Supportability/Tests
- [x] 🧹 Tech Debt/Cleanup

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2024-02-07 18:01:35 +00:00
Abin Simon
a2d40b4d38
Handle itemAttachments for emails (#5181)
<!-- PR description-->

---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* https://github.com/alcionai/corso/issues/5040

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2024-02-07 04:50:30 +00:00
Abhishek Pandey
c1ec1585a2
Skip exchange items which fail to download due to ErrorCorruptData error (#5191)
<!-- PR description-->

* We are seeing 500 errors from graph during exchange(email) backup. * Error is `:{"code":"ErrorCorruptData","message":"Data is corrupt., Invalid global object ID: some ID`. 
* Catch the error and add a skip since these items cannot be downloaded from graph. 

---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2024-02-07 02:40:14 +00:00
Abin Simon
9c8ac96aed
Skip emails with incorrect email when export ics file (#5190)
<!-- PR description-->

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2024-02-06 17:03:58 +00:00
Keepers
e86592f51e
replace graph Wrap and Stack with clues (#5018)
Now that graph errors are always transformed as part of the graph client wrapper or http_wrapper, we don't need to call graph.Stack or graph.Wrap outside of those inner helpers any longer. This PR swaps all those graph calls with equivalent clues funcs as a cleanup.

Should not contain any logical changes.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #4685 

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2024-02-06 01:08:26 +00:00
Abhishek Pandey
53a0525bfd
Handle previous paths for group mailbox (#5154)
<!-- PR description-->

* Tombstone collections weren't being added because we were not processing prev paths for group mailbox category.
* Hence deleted conversations were being carried forward even if `--disable-incrementals` was used.

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2024-02-05 23:10:09 +00:00
Abhishek Pandey
de22131b23
Split tombstone handling for groups services (#5170)
<!-- PR description-->

* Prerequisite to the currently failing PR https://github.com/alcionai/corso/pull/5154 
* Conversations tombstone IDs are currently 2 part ( `convID/threadID`).
* However, tombstones are looked up by `convID` part only, see [code](9a603d1f21/src/internal/m365/collection/groups/backup.go (L128)).
* Adding a change so that tombstone IDs are reduced to `convID`. Otherwise, we'd never delete tombstones while processing collections, and we'll run into `conflict: tombstone exists for a live collection` errors.
* This is safe to do as there is always a 1:1 relationship between `convID` and `threadID`. For e.g. attempting to create another thread inside a conversation creates a new conversation.

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2024-02-05 22:24:09 +00:00
Abhishek Pandey
7e2b9dab62
Add group mailbox export (#5153)
<!-- PR description-->

* Add EML exports for group mailbox.
* Tested E2E manually along with unit tests added in this PR.
* Will follow it up with a sanity test PR.
---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2024-01-31 05:14:59 +00:00
Abhishek Pandey
1537db59c4
Convert serialized posts to eml (#5152)
<!-- PR description-->

* Add the serialized `Postable` to EML converter code for group mailboxes and associated tests.
* We should converge this with `Messageable` code at some point with a small translator function which converts `Postable` to `Messageable`. Reason being that posts are a subset of messages. Although I feel it's a bit premature right now. Once we have tested this in prod a bit, and we know for a fact that posts are indeed a true subset of messages, we can do this convergence. 

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [x] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2024-01-30 21:00:50 +00:00
Keepers
80d7d5c63d
boilerplate teamschat collection package (#5087)
seems like a lot of code, but this is 95% boilerplate additions copied from groups collections packages, with a find-replace for names.

Some noteworthy differences:
* teamsChats does not handle metadata, so all metadata, delta, and previous path handling was removed
* teamsChats does not produce tombstones
* chats are never deleted, so no "removed" items are tracked
* all chats gets stored at the prefix root, so no "containers" are iterated, and therefore only one collection is ever produced.

This means that, overall, while the boilerplate here is still the same, it's much reduced compared to similar packages.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🌻 Feature

#### Issue(s)

* #5062

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2024-01-30 20:04:49 +00:00
ashmrtn
ca3ca60ba4
Update repo init for partial init case (#5060)
Unclear if this is precisely the path we want to take to handle possible
partial repo initialization so feel free to leave suggestions if a
different behavior would be better

Update the repo init logic to continue if the repo is marked as already existing. If it already exists then the process will try to update persistent config info in the repo. If the repo already exists but some connection credential is incorrect then stack errors such that both the reason for the connect failure and the fact that the repo already exists are both visible.

Manually tested attempting to initialize a repo that was already
initialized but using a different passphrase and got the output:
```text
Error: initializing repository: a repository was already initialized with that configuration: connecting to kopia repo: unable to create format manager: invalid repository password: repo already exists: repository already initialized
exit status 1
```

Attempting to initialize a repo again using the same passphrase resulted
in a success message with no errors reported

Unfortunately I can't really add unit tests to ensure the config ends up
the way we expect because I have no way to trigger an error partway through
kopia's init logic (or even just after it) without some non-trivial code
refactoring

---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Test Plan

- [x] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2024-01-30 18:27:35 +00:00
Abhishek Pandey
50ba30539a
Persist metadata files for group mailbox posts (#5135)
<!-- PR description-->

* Each post now has a `.data` and `.meta` file.
* I've only made changes to groups lazy reader to keep this PR short. Note that channels don't have meta/data file concepts, we don't want to accidentally enable this for channels. Given channels doesn't use lazy reader, this is safe to do short term.
* I'll be adding small follow up PRs to 1) Make sure channels files don't get assigned `.data` suffix. 2) Support for data and meta files for prefetch conversations backup. 

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2024-01-30 11:10:07 +00:00
ashmrtn
6ef2c2d494
Use type switch instead of strings of types (#5156)
Instead of relying on strings pulled from the graph SDK to identify the types of the items we're dealing with use the type of the interface in golang.

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #5124

#### Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2024-01-29 19:53:22 +00:00
Abin Simon
8a7a61f05d
CI to catch incorrect use of clues (#5144)
<!-- PR description-->

Catches issues where we use `WC` variant of clues when `ctx` is passed and as well as cases where we don't use WC variant when `ctx` is not passed in.

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [x] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2024-01-29 18:26:42 +00:00
ashmrtn
85aaa448c5
Call config verify command during maintenance (#5123)
Add the wiring to actually call the new config verify command during
maintenance

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

merge after:
* #5117
* #5118
* #5122

#### Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2024-01-29 16:25:25 +00:00
Abin Simon
79194c44df
Fix event cancellation status in export (#5107)
<!-- PR description-->

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2024-01-29 04:38:09 +00:00
ashmrtn
5f036a0cc1
Use fault bus and alerts instead of error for config verify (#5122)
Switch to using alerts and the fault bus instead of errors. Hopefully
this will make it easier to ensure this verify code doesn't fail
maintenance overall and that callers can consume the info in a
standardized manner

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

merge after:
* #5117
* #5118

#### Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2024-01-27 04:04:12 +00:00
ashmrtn
d426250931
Validate kopia config (#5118)
Add function that returns errors if it finds issues with common config
info in the kopia repo. Parameters that are currently checked are:

* kopia global policy:
    * kopia snapshot retention is disabled
    * kopia compression matches the default compression for corso
    * kopia scheduling is disabled
* object locking:
    * maintenance and blob config blob parameters are consistent (i.e. all
    enabled or all disabled)

Note that tests for this will fail until alcionai/clues#40 is
merged and clues is updated in corso

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

merge after:
* #5117
* alcionai/clues#40

#### Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2024-01-27 01:53:11 +00:00
Abhishek Pandey
8133da3087
Centralize metadata suffix definitions (#5134)
<!-- PR description-->

Centralizing the suffixes so that group mailbox and future services can also use them.

* Move `.meta`, `.data`, `.dirmeta` definitions from `internal/m365/collection/drive/metadata` pkg to `pkg/services/m365/api/graph/metadata`.
* Adjust package names in misc places.
* No logic change. 
* We should probably keep all metadata definitions and code in one place, i.e. relocate drive metadata definitions to `pkg/services/m365/api/graph/metadata`. This is a bigger change, we can do it later.
---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2024-01-26 22:06:04 +00:00
ashmrtn
683f163df1
Code and tests to validate retention config (#5117)
Basic validation checks that make sure maintenance and retention options are consistent with each other. This at least makes sure retention is either enabled and maintenance extends locks or nothing is locked at all.

Going to be part of a set of PRs for validating the kopia config

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2024-01-26 00:09:35 +00:00
ashmrtn
e4e4bae850
Add a pin to kopia snapshots so they won't be GCed by kopia accidentally (#5090)
Internally kopia has the notion of a retention policy. This adds a pin to each snapshot that corso uses kopia to create so that those snapshots won't be subject to kopia's rentention policy. As we disable kopia's retention policy during repo init this just acts as another layer of security to ensure snapshots won't be deleted unexpectedly.

Note that this PR only applies to _new_ snapshots that corso creates.
It doesn't go back and pin older snapshots for already existing backups

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2024-01-24 21:55:58 +00:00
ashmrtn
cb4006eb60
Configure new corso repos with 8 hr minEpochDuration (#5086)
We've found that reducing the MinEpochDuration parameter in kopia
can reduce GET requests sent to the object store by a large amount
(>50% in some cases) for repos that run many backups each day

This PR updates the init repo code path to set this parameter to
8hrs instead of the default 24hrs to bring in some of those savings

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2024-01-24 21:18:22 +00:00
Abin Simon
214d8a6030
Fixing handling of permissions for external users (#5097)
<!-- PR description-->

---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* fixes https://github.com/alcionai/corso/issues/5046

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2024-01-24 19:32:03 +00:00
Abin Simon
c526ced23e
Skip attachments without any content for events (#5110)
<!-- PR description-->

Known issue, tracking in https://github.com/alcionai/corso/issues/4772

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2024-01-24 18:50:51 +00:00
ashmrtn
963d0e3fef
Skip base finding steps if we don't need bases (#5066)
Minor optimization where we skip trying to find BackupBases if we know
we're going to end up dropping them anyway. This can save us from
loading all the kopia manifests only to discard the information we
pulled from them. Note that this optimization only kicks in for
backups that request no merge bases and no assist bases
("full backup")

Existing test coverage in `operations/manifests_test.go` ensures this
doesn't break base selection altogether

As this is an optimization we don't need to merge this PR if it seems
like it will make code maintenance more difficult going forward

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

* #<issue>

#### Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2024-01-24 17:05:34 +00:00
Abhishek Pandey
b073c204c8
Store parent post content (#5096)
<!-- PR description-->

* Expand inReplyTo property to additionally get the parent post contents. This will be persisted as part of post data.
* This additional data will be useful for building a reply tree if we decide to do in-order restore/export in future.


---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2024-01-24 06:36:44 +00:00
ashmrtn
5a05223e90
Use more unique data in the kopia SourceInfo for snapshots (#5093)
Use unique info for the kopia snapshot source information to help
isolate snapshots for different resources from each other. This will
come in handy both for checkpoint snapshots and regular snapshots if
kopia's retention policy happens to look for things.

Integration tests for this probably won't pass until the next PR in
the series is merged into this PR

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

merge after:
* #5091
* #5092

#### Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2024-01-24 02:46:57 +00:00
Keepers
d501a30761
use the tree-based drive backup by standard (#4980)
reverses the drive backup process toggle by making the tree-based backup standard, and the old, linear enumeration the toggleable fallback.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🌻 Feature

#### Issue(s)

* #4690

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2024-01-23 18:57:11 +00:00
Hitesh Pattanayak
8ca1a05400
generates test lists for sanity test (#5035)
generates test lists for CI sanity tests

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)
#4754 

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2024-01-23 18:16:59 +05:30
Hitesh Pattanayak
dfb4a73f56
modifies list restore handler and handles lists advanced restore (#5024)
- includes GetList in list restore handler
- includes GetListsByCollisionKey in list restore handler
- updates mock list restore handler


#### Does this PR need a docs update or release note?
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature

#### Issue(s)
#4754 

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2024-01-23 16:18:47 +05:30
Abhishek Pandey
90648ef564
Augment post details with recipient information (#5027)
<!-- PR description-->
Posts are always sent to the group email address, along with additional
recipients if any. Currently, we don't have a way to get the unique
recipient list for individual posts due to graph api limitations.

Store the group mail address so that we can use it to populate the 'To'
field during Post -> EML exports.


---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2024-01-23 05:18:37 +00:00
Abin Simon
deead262ae
Allow for timezone from TZ database (#5078)
<!-- PR description-->

Graph sometimes just send timezone value which are available in TZ
database. This PR makes sure that we don't always try to convert, but
check if the timezone is already in the format that we need first.

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic
words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E

---------

Co-authored-by: zackrossman <zrossman@alcion.ai>
2024-01-22 14:11:39 -08:00
Abin Simon
f1f805b3f8
Export event data for invites (#5049)
<!-- PR description-->

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* https://github.com/alcionai/corso/issues/5039

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2024-01-22 05:58:49 +00:00
Abin Simon
6d2d9c0099
Log more clues for export operations (#5068)
<!-- PR description-->

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2024-01-22 04:36:52 +00:00
ashmrtn
a6ec08375f
Fix in-place contacts restore when restore destination is empty (#5053)
Fix bug where we tried to create folders with an empty display name
when the user did a restore with `/` or `""` as the restore destination
for contacts

This PR is safe with the current flow for backing up contact folders
because nested folders aren't backed up at all (at least nesting beyond
the first level). It is unclear, though possible, that this patch will
continue to work if we backup nested contact folders

Also adds basic smoke tests for this issue for all exchange data types

---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Test Plan

- [x] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2024-01-18 23:50:50 +00:00
ashmrtn
b8b1299514
Cleanup immutable ID config passing (#5013)
This value was present in the control.Options in the struct and
passed as a parameter. This PR removes the parameter so it's clear
the control.Options one is used

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #5012

#### Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2024-01-17 00:51:11 +00:00
Keepers
6e50d5216e
remove builder.toServicePath from paths (#5028)
path.Build produces the same result with more standard and centralized behavior.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #4025

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2024-01-16 16:42:49 +00:00
Keepers
dab53502ae
minor logging and cli output tweaks (#5016)
just touchups for supportability and tracking, plus a ux tweak here and there.  No real logic changes.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🤖 Supportability/Tests

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2024-01-12 22:25:07 +00:00
Abin Simon
6beb774ba8
Enable events export (#5015)
<!-- PR description-->

---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* closes https://github.com/alcionai/corso/issues/3890

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2024-01-12 20:06:50 +00:00
Abin Simon
722c69b3cc
Generate text in case of utf8 descriptions (#5014)
The idea is to mimic the format in which graph convert html into text when generating ics which gets embedded into eml file.

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [x] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2024-01-12 18:35:46 +00:00
Abin Simon
985220562c
Few more fixes to ics export (#5004)
<!-- PR description-->

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* https://github.com/alcionai/corso/issues/3890

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2024-01-12 14:28:25 +00:00
Abin Simon
da8466ae0b
Fix handling of recurring events with end date (#4997)
Previously we were not accurately handling the end date of recurring
events. This was because we treated end date to be start of the day
instead of the end of day.
<!-- PR description-->

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* https://github.com/alcionai/corso/issues/3890

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2024-01-12 12:25:27 +00:00
Abin Simon
303b8c31ce
Cancelled and modified events in ics export (#4996)
<!-- PR description-->

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* https://github.com/alcionai/corso/issues/3890

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2024-01-12 10:41:10 +00:00
Keepers
1694581e26
make a not found core error (#4975)
make a canonical 'not found' error for our core sentinels. This replaces the graph.DeletedInFlight error, which acted as an interpretation of "not found".

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #4685

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2024-01-12 01:29:39 +00:00