BUG FIX: Adds clause to return an error if experienced during path building of a collection.
## Description
<!-- Insert PR description-->
## Does this PR need a docs update or release note?
- [x] ⛔ No
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [x] 🐛 Bugfix
- related to #2257
## Test Plan
<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
## Description
separates item get/retry when iterating over
items in exchange collection streams, and adds
a mocked unit test on that func.
## Does this PR need a docs update or release note?
- [x] ⛔ No
## Type of change
- [x] 🤖 Test
## Issue(s)
* closes#2299
## Test Plan
- [x] ⚡ Unit test
## Description
Create a shared set of helper functions and interfaces for getting next and delta links from Graph API responses. This helps standardize how they are handled with respect to nil/empty values and makes comparisons easier in other code
## Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
## Type of change
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup
## Issue(s)
* #2264
## Test Plan
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [x] 💚 E2E
## Description
Make a separate interface for fetching drive information from Graph API. Interface allows for better testing via mocks
Merges SharePoint and OneDrive code for getting drives.
Also fixes potential bug where not all drives would be fetched. This could have occurred because the previous implementation for both SharePoint and OneDrive were not checking for paginated results
Viewing by commit is recommended
## Does this PR need a docs update or release note?
- [x] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ] ⛔ No
## Type of change
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Test
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup
## Issue(s)
* #2264
## Test Plan
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
## Description
Three changes: adds a LargeItem servicer to
the exchange api client. Migrates attachment
downloads in mail and events out of serialize
and into the GetItem func. Finally, utilizes the
largeItem servicer to download attachments,
instead of the standard servicer.
A follow-up PR will add mocked test cases for
these changes.
## Does this PR need a docs update or release note?
- [x] ✅ Yes, it's included
## Type of change
- [x] 🌻 Feature
## Issue(s)
* #2299
## Test Plan
- [x] 💚 E2E
## Description
A couple minor refactors. First, combines the
graph/service_helper.go and graph/service.go
files into a single file (just lift'n'shift, no logic
changes). Second, renames CreateHTTPClient
to just HTTPClient. Third, replaces the new
LargeItemClient constructor with extensible
options for the http client, starting with a
NoTimeout option. This will provide longer
term maintenance over multiple constructors.
This change sets up QoL for building multiple services
and clients within api layer handlers, which will appear
in an immediately following PR.
## Does this PR need a docs update or release note?
- [x] ⛔ No
## Type of change
- [x] 🧹 Tech Debt/Cleanup
## Issue(s)
* #2299
## Test Plan
- [x] ⚡ Unit test
- [x] 💚 E2E
## Description
Expose the global exclude list mechanism in kopia BackupCollections to other components. Add tests for BackupCollections testing new exclude list functionality
Currently wired to always be passed nil
## Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
## Type of change
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
## Issue(s)
* #2243
merge after:
* #2143
## Test Plan
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
## Description
The current interface makes no sense. This
change refactors the funcs to standard boolean
comparators.
## Does this PR need a docs update or release note?
- [x] ⛔ No
## Type of change
- [x] 🧹 Tech Debt/Cleanup
## Description
Expand error checking for kopia wrapper backup. Kopia may report a number of errors for a snapshot operation but may not give error structs for them. Handle this by checking if there were > 0 errors reported and making our own error struct for this if there are.
This will keep the system from reporting a successful backup when kopia had problems reading or uploading data. Errored files in kopia do not become part of the snapshot
## Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
## Type of change
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
## Issue(s)
* closes#2280
## Test Plan
- [ ] 💪 Manual
- [ x ⚡ Unit test
- [x] 💚 E2E
## Description
If a drive item goes over its 1 hour jwt expiration to download the backing file, re-fetch the item
and use the new download url to get the file.
## Does this PR need a docs update or release note?
- [x] ⛔ No
## Type of change
- [x] 🌻 Feature
## Issue(s)
* #2267
## Test Plan
- [x] 💪 Manual
## Description
This is a quick hack to satisfy a primary case of PII scrubbing. We expect to revisit it in the future.
## Does this PR need a docs update or release note?
- [x] ⛔ No
## Type of change
- [x] 🌻 Feature
## Issue(s)
* #2284
## Test Plan
- [x] 💪 Manual
- [x] ⚡ Unit test
## Description
Wraps the onedrive item download control into
the lazy reader creation, so that items are not
fetched from graph until kopia decides it wants
the bytes. This should only occurr after other
checks, like mod-time comparison, have passed,
thus giving us kopia-assists for bakup.
## Does this PR need a docs update or release note?
- [x] ⛔ No
## Type of change
- [x] 🐛 Bugfix
## Issue(s)
* closes#2262
## Test Plan
- [x] ⚡ Unit test
- [x] 💚 E2E
## Description
This PR reintroduces the changes from #2266 with a change to *not* reset the transport
when initializing the shared client.
Doing so was removing the retry and other middlewares
and also resulting in throttled requests being masked as success
Also - we now decorate our download traffic with an ISV tag as recommended [here](https://learn.microsoft.com/en-us/sharepoint/dev/general-development/how-to-avoid-getting-throttled-or-blocked-in-sharepoint-online#how-to-decorate-your-http-traffic)
## Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [x] 🕐 Yes, but in a later PR
- [ ] ⛔ No
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #2266
## Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [ ] ⚡ Unit test
- [x] 💚 E2E
## Description
This check is brittle and shouldn't be needed. It should be sufficient to check whether a user has a
drive or not (similar to what we now do for exchange mailboxes).
Validated with the existing integration test.
## Does this PR need a docs update or release note?
- [x] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ] ⛔ No
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #2271
## Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
## Description
onedrive currently constructs a new http client
for every file it downloads. This causes the OS
to generate extra sockets, and hang onto them
after the download is complete. Replacing these
one-off clients with a singular, re-used client-
which is the behavior and standard suggested
for golang http clients- minimizes system
resource consumption.
## Does this PR need a docs update or release note?
- [x] ⛔ No
## Type of change
- [x] 🐛 Bugfix
## Issue(s)
* closes#2262
## Test Plan
- [x] 💪 Manual
- [x] ⚡ Unit test
- [x] 💚 E2E
## Does this PR need a docs update or release note?
- [x] ⛔ No
## Type of change
- [x] 🧹 Tech Debt/Cleanup
## Issue(s)
* #1969
## Test Plan
- [x] 💪 Manual
## Description
Tackles a few things:
* normalize case when going from string->path.Service or path.Category
* removes duplicate helper function for string->path.Category in favor of path package one
## Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
## Type of change
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup
## Issue(s)
* closes#2259
## Test Plan
- [x] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
## Description
Begin populating a global exclude set with files deleted from OneDrive. The set contains the IDs of files that have been deleted.
This is actually safe to return from GraphConnector now (though it's not returned) as it uses item IDs instead of item names.
Future PRs will need to address handling of (potentially) moved files
## Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
## Type of change
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
## Issue(s)
* #2242
## Test Plan
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
## Description
Some external services like OneDrive do not have the ability to determine the path a deleted item used to exist at, they only know the item's old ID. This patch allows Kopia Wrapper to handle those items by implementing a global exclude set. The patch assumes that items in base directories are unique as the items in every base directory are checked against the set.
This is not wired to anything outside of Kopia Wrapper. Currently this feature is disabled as the passed value is always nil.
## Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
## Type of change
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
## Issue(s)
* closes#2121
## Test Plan
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
## Description
If gc.stats reports a non-zero error count at the
end of a backup, retrieve the error from the
status and return it as the backup operation err.
## Does this PR need a docs update or release note?
- [x] ⛔ No
## Type of change
- [x] 🐛 Bugfix
## Test Plan
- [x] ⚡ Unit test
- [x] 💚 E2E
## Description
Update kopia version since upstream now includes recent PRs from our repo
Upstream now includes the following:
* io.ReadCloser interface for StreamingFiles
* properly pass file size for cached StreamingFiles
## Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
## Type of change
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup
## Issue(s)
* closes#2234
* closes#1732
## Test Plan
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [x] 💚 E2E
## Description
Fixes and unskips the tests incremental integration tests
for exchange calendar events.
Currently blocked on LynneR displaying the calendar
fetch issues where it's only possible to retrieve her default
Calendar, Birthdays, and US Holidays calendars.
## Does this PR need a docs update or release note?
- [x] ⛔ No
## Type of change
- [x] 🤖 Test
## Issue(s)
* #2022
## Test Plan
- [x] 💪 Manual
- [x] 💚 E2E
## Description
When fetching item data, don't backoff and retry if Graph reported the item was not found. In this case, we mark it as succeeded as we can never get the data anyway.
## Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
## Type of change
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup
## Issue(s)
* closes#2217
## Test Plan
- [x] 💪 Manual
- [ ] ⚡ Unit test
- [ ] 💚 E2E
## Description
Report an error if an item in details that was sourced from a base snapshot doesn't have a previous path. Items that don't have a previous path cannot be sourced from a base snapshot's backup details meaning the item will be stored but will not be searchable/restorable by users
Logging was not used because no context is available in the kopia callback that is checking if the previous path is nil
## Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
## Type of change
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
## Issue(s)
* closes#1915
## Test Plan
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
## Description
Adds the api client pkg pattern to the connector/
discovery package. Most code changes are plain
lift-n-shift, with minor clean-ups along the way.
User retrieval is now filtered to only include
member and on-premise accounts.
## Does this PR need a docs update or release note?
- [x] ✅ Yes, it's included
## Type of change
- [x] 🌻 Feature
## Issue(s)
* #2094
## Test Plan
- [x] 💪 Manual
- [x] ⚡ Unit test
## Description
Track folder moves and update the persisted metadata accordingly. Moves need to take into account subtree changes because OneDrive will not notify us of subfolders moving if nothing else was updated on the subfolder.
Handle updates by making a copy of the folder map for ease of updating and later on ease of finding out which collection paths have changed
## Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
## Type of change
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
## Issue(s)
* #2120
## Test Plan
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
## Description
If GraphConnector reported errors while trying to fetch item data, fail the entire operation. This stops silent failures, but is a big hammer at the moment because there's no checks for items no longer being available.
Error reporting is minimal as well, but some info should be in the log.
**This effectively disables best-effort as any failure will result in the operation being reported as failed**
## Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
## Type of change
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
## Issue(s)
* #2196
## Test Plan
- [x] 💪 Manual
- [ ] ⚡ Unit test
- [ ] 💚 E2E
## Description
Track folder and package deletions in OneDrive metadata for incremental backups. Add test for deletions
## Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
## Type of change
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
## Issue(s)
* #2120
## Test Plan
- [x] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
## Description
Record information about the error Graph returned if something went wrong when fetching an item.
## Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
## Type of change
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup
## Issue(s)
* closes#2194
## Test Plan
- [x] 💪 Manual
- [ ] ⚡ Unit test
- [ ] 💚 E2E
## Description
Item not found errors may appear because an item was moved or deleted between enumerating the items in a folder and fetching the data for the items in the folder. This keeps them from being reported as there is nothing we can do if the item data is no longer available.
This applies to all Exchange data categories
## Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
## Type of change
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
## Issue(s)
* closes#2198
## Test Plan
- [x] 💪 Manual
- [ ] ⚡ Unit test
- [ ] 💚 E2E
## Description
Make error appending thread-safe since multiple goroutines may attempt to add an error at the same time.
## Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
## Type of change
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
## Issue(s)
* closes#2197
## Test Plan
- [x] 💪 Manual
- [ ] ⚡ Unit test
- [ ] 💚 E2E
## Description
Images copy/pasted into a message were not being saved previously. Verified with Corso binary
<!-- Insert PR description-->
## Does this PR need a docs update or release note?
- [x] ✅ Yes, in the ChangeLog.
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [x] 🐛 Bugfix
## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* closes #2163<issue>
## Test Plan
<!-- How will this be tested prior to merging.-->
- [x] ⚡ Unit test
## Description
Adds a little extra process into the calendar
resolver so that it mimics the mail and contact
resolvers. This will allow us to collapse the
three resolvers into a more common handler
or interface.
We can take this change or drop it. I added
the code in exploration of the event test failures,
and figured I'd throw it out just in case.
## Does this PR need a docs update or release note?
- [x] ⛔ No
## Type of change
- [x] 🧹 Tech Debt/Cleanup
## Issue(s)
* #2022
## Test Plan
- [x] 💚 E2E
## Description
CI Secrets will need to be updated to complete this change.
## Does this PR need a docs update or release note?
- [x] ⛔ No
## Type of change
- [x] 🤖 Test
- [x] 💻 CI/Deployment
## Issue(s)
* #2022
## Test Plan
- [x] 💪 Manual
- [x] 💚 E2E
## Description
Follow up for https://github.com/alcionai/corso/pull/2156. We might deice to change where we add the logic to handle this, as in if we should move it to discovery module.
## Does this PR need a docs update or release note?
- [x] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ] ⛔ No
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* https://github.com/alcionai/corso/issues/2145
* https://github.com/alcionai/corso/pull/2156
## Test Plan
<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ] ⚡ Unit test
- [ ] 💚 E2E
## Description
SharePointInfo extended to include SitePageable objects.
**NOTE:** Merge includes the beta graph package and will affect the compile time.
<!-- Insert PR description-->
## Does this PR need a docs update or release note?
- [x] ⛔ No
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* closes #2069<issue>
* related #2071
## Test Plan
- [x] ⚡ Unit test
## Description
Fix to allow selectors to include `SharePoint.Pages` during backup without causing backup workflow to fail.
<!-- Insert PR description-->
## Does this PR need a docs update or release note?
- [x] ⛔ No
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [x] 🐛 Bugfix
## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
*closes #2113<issue>
## Test Plan
<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
## Description
Branch treats Drive `Site Pages` as a restricted directory. All `pages` within the directory, will be backed up via the Site Pages API. This adds an additional call to per collection to obtain the parent `driveName`.
<!-- Insert PR description-->
TODO: Create a Pipeline that distinguishes between SitePages, Libraries, and List for SharePoint
## Does this PR need a docs update or release note?
- [x] 🕐 Yes, but in a later PR
The documentation should state the assumption is that only `.aspx` files are to be found in the s
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* closes #2072<issue>
## Test Plan
- [x] ⚡ Unit test
## Description
Hacks the beta version call into the events api
when iterating through items, allowing us to run
delta-based queries for calendar events.
## Does this PR need a docs update or release note?
- [x] ⛔ No
## Type of change
- [x] 🌻 Feature
## Issue(s)
* #2022
## Test Plan
- [x] ⚡ Unit test
- [x] 💚 E2E
## Description
Use same tags for checkpoints and snapshots as having different tags won't get us too much benefit. We can revisit this if we find we're really doing too many manifest fetches, but until then the current code to fetch manifests is pretty good.
## Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
## Type of change
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup
## Issue(s)
* #1674
## Test Plan
<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ] ⚡ Unit test
- [ ] 💚 E2E
## Description
Make some parts of the lookup logic more uniform as to when reasons are added to things.
Add logging to make the process easier to debug
## Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
## Type of change
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup
## Issue(s)
* #1675
* #2149
## Test Plan
- [x] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
## Description
Adds info logging on all throttling responses,
and some extra error handling in container
resolvers.
## Does this PR need a docs update or release note?
- [x] ⛔ No
## Type of change
- [x] 🧹 Tech Debt/Cleanup
## Test Plan
- [x] 💪 Manual
## Description
Adds additional definitions for context timeouts for middleware usage
<!-- Insert PR description-->
## Does this PR need a docs update or release note?
- [x] ⛔ No
## Type of change
- [x] 🐛 Bugfix
## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
*related #2047<issue>
## Test Plan
<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
## Description
This commit adds logic in discovery and backup to check whether the specified user has
an exchange mailbox that is available/enabled.
If so - the backup is short-circuited to succeed but with "no data"
Going forward - we should be able to move the logic in the OneDrive connector that checks
for a valid drive and license in here.
## Does this PR need a docs update or release note?
- [x] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ] ⛔ No
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #2145
## Test Plan
<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ] ⚡ Unit test
- [ ] 💚 E2E
## Description
Tags allow searching for previous snapshots (complete or otherwise). This in turn provides the ability to use kopia-assisted incrementals in more situations as partially completed backups will be reused for their cached data.
Streaming files that have not been completely uploaded are not added to a checkpoint so there is no possibility of accidentally triggering kopia-assisted incrementals thus causing us to treat a partially uploaded file in a checkpoint as a "cached" version of the complete file
## Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
## Type of change
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
## Issue(s)
* closes#1674
## Test Plan
- [x] 💪 Manual
- [ ] ⚡ Unit test
- [ ] 💚 E2E
## Description
Add log statements noting which bases were used for kopia assisted incrementals and which bases were merged into the hierarchy. Also record the reasons a base was chosen.
Log statements when searching for previous snapshots will be added when that code is refactored
## Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
## Type of change
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup
## Issue(s)
* #2149
## Test Plan
- [x] 💪 Manual
- [ ] ⚡ Unit test
- [ ] 💚 E2E
## Description
Adds mocked unit tests for produceManifestsAnd-
Metadata. For cleanliness, moves that func, and
any funcs called within it, to their own file within operations
## Does this PR need a docs update or release note?
- [x] ⛔ No
## Type of change
- [x] 🤖 Test
## Issue(s)
* #2062
## Test Plan
- [x] ⚡ Unit test
## Description
Start persisting the folder path maps and delta URLs for backed up OneDrive/SharePoint drives. Delta URLs are saved in a map[drive ID]deltaURL while folder IDs are in a map[driveID]map[folder ID]folder path
Needs another patch to properly save the path for folders that match the selector, currently the selector comparison is only on the parent of an item
Later PRs can get the new folder map by taking the map from the previous backup and making changes to it when folder deletions/moves are encountered
## Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
## Type of change
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
## Issue(s)
* #2120
## Test Plan
- [x] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
