1734 Commits

Author SHA1 Message Date
ashmrtn
01a6304a62
Split some CI tests for OneDrive into nightly and regular CI (#2907)
Try to reduce the overall number of tests run during
regular CI by splitting most of the OneDrive
RestoreAndBackup tests into a new "nightly" test suite.
Regular CI will now only run against the latest backup
version to make sure nothing immediate broke

This should only be merged once we have a GitHub action
for nightly tests and some way of reporting the outcome
of the action

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Supportability/Tests
- [x] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* closes #2756

#### Test Plan

- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-03-22 21:51:44 +00:00
neha_gupta
adc4bb068a
continue if collection path is nil (#2912)
Currently code panics if it tries to find FullPath for a drive item and its not present.
Added a continue and skipped the item if FullPath is not present.

Root cause of bug is caused by having a folder in the previous paths map, having
the delta token expire, and seeing the folder be deleted in the delta results

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-03-22 21:11:21 +00:00
Abin Simon
e09a98f462
Fix permissions restore for version 6 (#2908)
Paths were not properly augmented for newer versions which removed the name from the `.dirmeta` file. This fixes that. It also adds a tests to ensure that the code will fail whenever we add a new version forcing us to revisit and update.

---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* https://github.com/alcionai/corso/pull/2906
* fixes https://github.com/alcionai/corso/issues/2909

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-03-22 20:27:17 +00:00
Keepers
e77a5e5a7e
additional logging when creating backup model (#2916)
#### Type of change

- [x] 🤖 Supportability/Tests

#### Issue(s)

* #2913
2023-03-22 20:00:33 +00:00
Keepers
532c5c017b
update backup cli output (#2903)
changes a couple things in the backup cli output:
* appends the backup id after completion
* prints an error if the backup fails
* replaces multierr with a clues error
* logs each backup failure individually

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #1970

#### Test Plan

- [x] 💪 Manual
2023-03-22 16:01:02 +00:00
Abin Simon
8bebf579a4
Remove global 10 day timeout (#2875)
This has been fixed upstream with https://github.com/microsoft/kiota-http-go/pull/71 . We don't need this hack anymore.

---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* https://github.com/alcionai/corso/issues/2849
* https://github.com/alcionai/corso/pull/2694
* closes https://github.com/alcionai/corso/pull/2893
* closes https://github.com/alcionai/corso/pull/2892
* https://github.com/alcionai/corso/pull/2891

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-03-22 13:37:30 +00:00
Vaibhav Kamra
f5bdbc7be2
Update CHANGELOG.md (#2905)
Update CHANGELOG - remove v0.6.0 and update with v0.6.1

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-03-22 04:00:47 +00:00
ashmrtn
1aa315412a
Add more logging to middleware and incremental backup operation logic (#2899)
Ensures:
* context clues are added to logged messages
* kv-pairs added with `log.With` are added to logged messages
* add log messages for how many details items were sourced from a base snapshot during merging
* ensure log messages for base selection have all the info they should

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Test Plan

- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
v0.6.1
2023-03-21 22:33:19 +00:00
Keepers
cb50386b4b
log item path on restore (#2901)
adds the item path to the restore logger, plus some errant ctx usage fixes

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🤖 Supportability/Tests
2023-03-21 19:00:16 +00:00
Keepers
285f8f01ef
update sharepoint cli examples (#2822)
Updates the sharepoint cli examples to better
reflect the current support and utilities.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🌻 Feature

#### Issue(s)

* #2786

#### Test Plan

- [x] 💪 Manual
- [x] 💚 E2E
2023-03-21 17:33:24 +00:00
Keepers
c7d282ce0c
cli: use withTimeout, not withDeadline (#2865)
#### Type of change

- [x] 🧹 Tech Debt/Cleanup
2023-03-21 14:51:57 +00:00
neha_gupta
7fbe6d1099
Update CHANGELOG.md (#2894)
<!-- PR description-->

update changelog for latest release

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-03-21 08:01:44 +00:00
Niraj Tolia
18843ac1e2
Group CLI docs by service for better accessibility (#2890)
This commit makes the CLI docs more accessible by grouping them by service

---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included

#### Type of change

- [x] 🗺️ Documentation
2023-03-21 07:56:25 +00:00
Niraj Tolia
5e36ae5789
Add SharePoint CLI docs (#2889)
Adds auto-generated SharePoint docs to the website

---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🗺️ Documentation
2023-03-21 07:39:04 +00:00
neha_gupta
5951cbc289
handle endpoint flag to check http:// or https:// (#2874)
<!-- Insert PR description-->

add CLI check to handle http or https protocol.  

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🐛 Bugfix

#### Issue(s)

* https://github.com/alcionai/corso/issues/2330

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
2023-03-21 07:35:05 +00:00
Keepers
bdc854c1da
scopes typing rename, filter -> info (#2842)
Renames a lot of the names in scopes that contain
the word "filter" with the word "info".  The basis here
is to reduce confusion in the libraries.  Filter as a term
is overloaded.  It currently means: 1/
the set of scopes that comparisons _must_ pass
to succeed; 2/ the pkg and  struct that handles
comparison logic; and 3/  the set of scopes that
compare values in details.ItemInfo instead of in
the item's storage or logical path.  The third
category is the primary overload here, and is
better off with a different name.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Test Plan

- [x]  Unit test
2023-03-21 06:51:14 +00:00
Georgi Matev
7b26a7213a
Distinguish CI output between Exchange and OneDrive (#2885)
The CI output between Exchange and Onedrive is easily distinguished when calling multiple scripts from the same action step. Use separate steps for clarity. 

Also removes now unnecessary Go setup step

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [x] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)


#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
v0.6.0
2023-03-21 06:17:47 +00:00
Vaibhav Kamra
c49e4bbef4
Handle 503 retries where the Retry-After header exceeds what the Graph SDK will wait for (#2888)
This logic will only trigger if the Graph retry handler does not handle the 503 (it is invoked first).

We remove the check in Corso for `absoluteMaxDelay` because our retry config will not trigger that limit (180s).
This allows the retry to kick in if the 503 response contains a `Retry-After` value > 180s

This could help with issues like https://github.com/alcionai/corso/issues/2876

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #2876 

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-03-21 05:37:24 +00:00
Abin Simon
ef5178668f
Log user name when enumerating drive items (#2872)
This mimics the behavior for exchange where we print out which user and what content is currently being enumerated. While this might not be useful when we are only backing up a single user, it can come in handy when we run backups for multiple users.

<!-- Insert PR description-->

---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* fixes https://github.com/alcionai/corso/issues/2871

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-03-21 03:51:16 +00:00
ashmrtn
a798932e4d
Add hidden flag to set fetch parallelism for Exchange from CLI (#2879)
#### Does this PR need a docs update or release note?

- [x]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

* closes #2877

#### Test Plan

- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-03-20 21:48:21 +00:00
Keepers
8b1c3a7073
add labels to logged errors (#2824)
#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup
2023-03-20 21:12:05 +00:00
Keepers
c47ee74837
add supportability to type of change (#2881)
#### Type of change

- [x] 💻 CI/Deployment
2023-03-20 20:37:18 +00:00
Georgi Matev
6c989dbd78
Adding new OneDrive and Sharepoint CI cleanup (#2870)
New script for OneDrive and Sharepoint cleanup 

Commit by commit review is encouraged for readability since this involves renames of older scripts.


---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [x] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)


#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E

Tested this with `nektos/act `so action logic should be mostly sound. There is one issue with the script installed for exchange that I believe is just a difference the container image used by GHA and the local I could use for `act`
2023-03-20 20:36:27 +00:00
ashmrtn
2176f4b831
Add option to control parallelism getting items (#2864)
This option is only consumed by Exchange right now and will control the number of items fetched in parallel per collection. Note that the total number of parallel item fetches may be higher is kopia is concurrently uploading multiple collections.

This PR does not expose this option at the CLI layer

Manually tested to ensure it
* uses the default if options value is 0
* uses the default if options value > default
* uses option value otherwise

There's another PR open for this but it's not expected to main. We still
want to make sure the change goes in though, hence this PR.

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Test Plan

- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-03-20 20:04:49 +00:00
Keepers
fa2cf046bb
fixes, logging improvements for fault.Errors (#2848)
A last-second change in 2708 caused us to
pass along the wrong fault.Errors into backup
persistence, thus slicing the count of skipped
items.  That's been fixed, along with improved
end-of-operation logging of fault errors.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🐛 Bugfix

#### Issue(s)

* #2708

#### Test Plan

- [x] 💪 Manual
- [x] 💚 E2E
2023-03-20 06:01:04 +00:00
Georgi Matev
bf1c824f4a
Once more cleanup action typo (#2868)
Final CI cleanup fix
---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-03-19 10:47:09 +00:00
Georgi Matev
4d89626071
Trying to fix a break in the action format (#2866)
---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [x] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-03-19 09:17:10 +00:00
ashmrtn
a13479a0f6
Enable OneDrive incrementals, disable incrementals for permission backup (#2642)
Permissions backup requires more work, so we only support delta incrementals for OneDrive if that's not enabled.

---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #2333

Pending merge of
* #2650
* #2496

#### Test Plan

- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-03-18 17:37:44 +00:00
Keepers
c28d5399ef
mark event instance loss as known (#2837)
#### Does this PR need a docs update or release note?

- [x]  Yes, it's included

#### Type of change

- [x] 🗺️ Documentation

#### Issue(s)

* #2835
2023-03-18 17:04:15 +00:00
ashmrtn
56cd045bff
Add a test with duplicate item IDs for OneDrive (#2845)
Ensure that as long as drive IDs are unique we can handle
items with the same ID if they're in different drives.

This is a direct result of observing SharePoint sites that
have multiple drives and the root of each drive has the
same ID

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #2732

#### Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-03-18 03:01:40 +00:00
Georgi Matev
df1ddb94f3
Improved CI cleanup script (#2851)
Improved this significantly based on some learnings from trying to get this in better shape
* Reduce folder enumerations significantly
* Address issue when cleanup gets backlogged and cannot enumerate all items to clean in a single job
* Remove the need to launch one job per prefix or folder name
* The only way to clean /Audits reliably seems to be emptying (with hard delete) of /Revoverable Items (parent of Audits)
* Eliminated need to control concurrency and overhead of job spin up
* Attempt to set explicit network timeout for SOAP calls to prevent long hangs being observed

NOTE: This change affect both cleanup script and the corresponding GHA workflow
---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [x] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-03-18 02:13:45 +00:00
ashmrtn
60d173b25b
Log retry after for throttled requests (#2847)
This seems to be the header that is used to communicate the delay in seconds to wait.

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* closes #2846

#### Test Plan

- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-03-18 00:14:03 +00:00
Keepers
621e850531
add 10 day deadline to cli context (#2852)
Ensures all CLI usage has a context deadline so that we
don't bump into the graph api 100 second deadline.

---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included

#### Type of change

- [x] 🐛 Bugfix

#### Test Plan

- [x] 💪 Manual
2023-03-17 23:48:13 +00:00
Keepers
91328d9898
cleanup graph logging, remove 2xx test logs (#2836)
Removes 2xx query logging from CI testing, and
slightly refactors the logging middleware to make
better use of clues and minimize boilerplate.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🤖 Test
- [x] 🧹 Tech Debt/Cleanup

#### Test Plan

- [x] 💚 E2E
2023-03-17 16:00:22 +00:00
Vaibhav Kamra
83bddf79a0
Use AWS keys for long running test (#2817)
This switches the sanity test to use static keys since the test runs longer than an hour

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
2023-03-17 15:36:29 +00:00
Abin Simon
0e1064078a
Backup owner and empty permissions (#2833)
Backup owner and empty permissions, we cannot restore them though (refer https://github.com/alcionai/corso/issues/2789#issuecomment-1471787829) . Previous we were just skipping even backing them up. It might be good idea to have this data in case we find that we are able to restore them in the future.

---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* closes https://github.com/alcionai/corso/issues/2789

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-03-17 09:42:23 +00:00
ashmrtn
9fd3b81f6e
More rigorous email test with repeat folder names (#2792)
Expand coverage for email tests by creating a layout that has duplicate folder names at different levels of the hierarchy.

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Test
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* closes #1228

#### Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-03-17 05:31:26 +00:00
Keepers
968f51616f
add outputs to script (#2834)
Some quick QoL
2023-03-17 05:07:01 +00:00
Keepers
8dfb00f308
add fault errors streamstore (#2731)
Adds a new streamstore controller for fault.Errors. This provides large scale, extensible file storage for fault errors to be persisted, much like we do for backup details.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🌻 Feature

#### Issue(s)

* #2708

#### Test Plan

- [x] 💚 E2E
2023-03-17 01:07:07 +00:00
Georgi Matev
bf2f66398b
Prevent the static purge scripts running in combination without a user (#2832)
---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-03-17 00:07:34 +00:00
Keepers
26bf24c77e
add load tester suite (#2809)
#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Test Plan

- [x] 💚 E2E
2023-03-16 23:04:35 +00:00
Keepers
4515d3a4b2
remove unused test scoping envs (#2648)
Doesn't seem like we're using these

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🤖 Test
- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #2373

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-03-16 08:00:49 +00:00
Abin Simon
4303c78f3e
Add option to specify user to run sanity tests with in GHA (#2806)
![2023-03-15-17-51-55](https://user-images.githubusercontent.com/14259816/225307537-91e3754e-c08a-4457-9482-999cc9d5b084.png)


---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [x] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-03-16 07:23:16 +00:00
Georgi Matev
b1566ed3d3
Fix CI-Test-Cleanup workflow definition (#2830)
Incorrect workflow definition
---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-03-16 04:10:39 +00:00
Georgi Matev
dc8ec86dd3
Integrate Exchange retention reset in CI (#2807)
Adds retention to current CI action
---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [x] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)


#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-03-15 23:51:49 +00:00
Keepers
1b6ad654e8
fix sharepoint filter consts (#2776)
#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🐛 Bugfix

#### Issue(s)

* #2757

#### Test Plan

- [x]  Unit test
2023-03-15 20:58:43 +00:00
Keepers
76b3fe3b86
append ToCore to all errors tests (#2793)
In order to retrieve all clues structured error data in tests, we need to extract it from the error using the clues library.

This change appends `clues.ToCore(err)` to all
variations of `assert.NoError(t, err)`.  The only
other changes are those necessary to preserve
linting, or to produce an error variable for the
ToCore call.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🤖 Test
- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #1970

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-03-15 19:02:47 +00:00
Abin Simon
0125876192
Skip items returning 404 (#2805)
Treat a 404 as an unavailable item and skip it similar to malware items.

---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
Treat a 404 as an unavailable item and skip it similar to malware items.
2023-03-15 17:52:22 +00:00
Nočnica Mellifera
766dbb2c94
new blog post with all recent video tutorials (#2653)
new blog post with all recent video tutorials

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [x] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic
words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E

---------

Co-authored-by: Niraj Tolia <ntolia@users.noreply.github.com>
2023-03-15 09:35:22 -07:00
Georgi Matev
41f0b86073
Script to set retention for all mailboxes in a tenant (#2798)
The script requires authenticating with Exchange admin user name and password and will set retention to 0 for all tenant mailboxes. This helps with CI cleanup of test data

The script can be integrated with some low frequency invocation with the CI pipeline to make sure the retention is properly set. Would require setting the credentials as Git secrets

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [x] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)


#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-03-15 16:07:39 +00:00