adds the getPostIDs func to ensure conversations
complies with standard data paging patterns
---
#### Does this PR need a docs update or release note?
- [x] ⛔ No
#### Type of change
- [x] 🌻 Feature
#### Issue(s)
* #4536
#### Test Plan
- [x] ⚡ Unit test
- [x] 💚 E2E
<!-- PR description-->
With #4497 changes, any counters that we log in graph middlewares ( e.g. `APICallTokensConsumed` or `ThrottledAPICalls`) will no longer show up in `BackupOperation.Results.Counts`.
This is because graph adapters and backup operation will be initialized with different count busses. Since the backup result counts are captured from the operation bus, we will lose out on any graph metrics captured in middleware layers. This PR unifies the busses.
Ill also add a regression check for graph metrics shortly
---
#### Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
#### Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
#### Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>
#### Test Plan
<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ] ⚡ Unit test
- [ ] 💚 E2E
Previously if a folder with the same name was deleted and recreated, we would error out. This fixes that by ensuring that only the final collection makes it into the final set.
---
#### Does this PR need a docs update or release note?
- [x] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ] ⛔ No
#### Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
#### Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>
#### Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
adds basic get requests for the conversations api set to the api package. Plust some other minor change for formatting, correctness, or necessity.
---
#### Does this PR need a docs update or release note?
- [x] ⛔ No
#### Type of change
- [x] 🌻 Feature
#### Issue(s)
* #4536
#### Test Plan
- [x] 💪 Manual
- [x] ⚡ Unit test
- [x] 💚 E2E
We were previously exporting json but without the proper file extension.
---
#### Does this PR need a docs update or release note?
- [x] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ] ⛔ No
#### Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
#### Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* closes https://github.com/alcionai/corso/issues/4581
#### Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
Add an option to request a preview backup and tag the resulting backup
as a preview if the flag is set. Preview backups must complete
successfully with no errors in order to be tagged
This does not update the item selection logic, so right now preview
backups will contain all items that normal backups do. Item selection
will be refined in upcoming PRs
---
#### Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
#### Type of change
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
#### Test Plan
- [ ] 💪 Manual
- [ ] ⚡ Unit test
- [x] 💚 E2E
Add a comment and test ensuring preview backups are excluded during
base selection as expected. It's still not possible to make a
backup tagged with the preview tag right now
---
#### Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
#### Type of change
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
#### Test Plan
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
First step in reducing the number of places we have to check the service type manually. Create a way to get a handle to a service specific handler and implement exports for those handlers
---
#### Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
#### Type of change
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup
#### Issue(s)
* #4254
#### Test Plan
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [x] 💚 E2E
Use the new helper function to get the backup type and rewrite logic
to streamline checks/actions on the backup type
Hoists a check on the backup's snapshot ID higher in the code and
adds a test case for that
---
#### Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
#### Type of change
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
#### Test Plan
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
Store more info in the context so that log statements need to add fewer additional fields.
---
#### Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
#### Type of change
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup
#### Test Plan
- [x] 💪 Manual
- [ ] ⚡ Unit test
- [ ] 💚 E2E
Rename variable in test struct to be more in line
with other variable names and also make the use
clearer
---
#### Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
#### Type of change
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
#### Test Plan
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
These tests were originally written before we checked backup
models when searching for bases. This refactor updates the
tests with helper functions that create self-consistent
backup and manifest models. Overall this pattern makes it
easier to declare the data the test uses, easier to read
test data, and makes the tests more compact
Reviewing this PR by commit may be easier
---
#### Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
#### Type of change
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
#### Test Plan
- [x] 💪 Manual
- [ ] ⚡ Unit test
- [ ] 💚 E2E
#### Does this PR need a docs update or release note?
- [x] ⛔ No
#### Type of change
- [x] 🧹 Tech Debt/Cleanup
#### Test Plan
- [x] ⚡ Unit test
- [x] 💚 E2E
Forgot it increments the version as it goes
---
#### Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
#### Type of change
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
#### Issue(s)
* #4569
#### Test Plan
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
Bump the backup version and force a full backup if
there's a backup for teams/groups that has base(s)
from an older version of corso
This will avoid propagating older details formats
forward. Those formats don't have all the data
newer formats do
This is mostly a stop-gap, a more robust solution
can be added later
Manually tested that it forces a full backup
---
#### Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
#### Type of change
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
#### Issue(s)
* #4569
#### Test Plan
- [x] 💪 Manual
- [ ] ⚡ Unit test
- [ ] 💚 E2E
Adds two-step processing to the html previews for channel messages and replies. First, all inline attachments are replaced with the string `[attachment:name]`.
Second, remaining html is stripped out, leaving only plaintext.
This transformation is applied to both the exported content and the preview content in details.
---
#### Does this PR need a docs update or release note?
- [x] ⛔ No
#### Type of change
- [x] 🌻 Feature
#### Issue(s)
* #4546
#### Test Plan
- [x] ⚡ Unit test
- [x] 💚 E2E
Export doesn't pull meaningful stats from the controller. Just remove
the collection/usage of these stats since it's confusing. Substitute the
check for not doing anything from a check on stats to a check on the
number of collections returned by kopia.
---
#### Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
#### Type of change
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup
#### Issue(s)
* #4254
#### Test Plan
- [x] 💪 Manual
- [ ] ⚡ Unit test
- [ ] 💚 E2E
Update the BackupBases API to return BackupBase
structs that contain both the snapshot and backup
model instead of having separate functions for
each. Minor logic updates to accommodate these
changes
This PR also updates tests and mock code for the
new API
Suggest viewing by commit
---
#### Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
#### Type of change
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup
#### Issue(s)
* closes#3943
#### Test Plan
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [x] 💚 E2E
builds out more details for channel messages and replies
---
#### Does this PR need a docs update or release note?
- [x] ⛔ No
#### Type of change
- [ ] 🌻 Feature
#### Issue(s)
* #3988
#### Test Plan
- [x] ⚡ Unit test
- [x] 💚 E2E
No logic changes. Just code movement/renaming.
---
#### Does this PR need a docs update or release note?
- [x] ⛔ No
#### Type of change
- [x] 🧹 Tech Debt/Cleanup
Will be useful for later testing that needs BackupBase
---
#### Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
#### Type of change
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
#### Issue(s)
* #3943
#### Test Plan
- [x] 💪 Manual
- [ ] ⚡ Unit test
- [ ] 💚 E2E
<!-- PR description-->
Add a check to make sure url cache's previous delta base doesn't change after refresh
---
#### Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
#### Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
#### Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>
#### Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [ ] ⚡ Unit test
- [ ] 💚 E2E
Merge fields in the backup bases struct since
we assume they need to be tracked together
anyway
This PR attempts to keep the API as close to
what it currently is as possible. A future PR
will go through and update the
API/tests/mocks
---
#### Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
#### Type of change
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup
#### Issue(s)
* #3943
#### Test Plan
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [x] 💚 E2E
removes the cache container transformation funcs
from the exchange api container enumeration params. This ensures a cleaner separation of ownership where the api is only responsible for enumerating containers, and the caller is responsible for transforming them.
---
#### Does this PR need a docs update or release note?
- [x] ⛔ No
#### Type of change
- [ ] 🧹 Tech Debt/Cleanup
#### Test Plan
- [x] ⚡ Unit test
- [x] 💚 E2E
marks the full subtree of a drive package item
as being within the package. This ensures
that all collections whose directory is a subdir of a package-type folder are themselves marked as package-typed.
---
#### Does this PR need a docs update or release note?
- [x] 🕐 Yes, but in a later PR
#### Type of change
- [x] 🐛 Bugfix
#### Test Plan
- [x] ⚡ Unit test
- [x] 💚 E2E
As the title suggests.
---
#### Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
#### Type of change
<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
#### Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>
#### Test Plan
<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [x] ⚡ Unit test
- [x] 💚 E2E
renames methods in the group handler interface
to replace the channel-specific vocabulary with the more standard and generic "containers" and
"items" naming conventions.
---
#### Does this PR need a docs update or release note?
- [x] ⛔ No
#### Type of change
- [x] 🧹 Tech Debt/Cleanup
#### Test Plan
- [x] ⚡ Unit test
- [x] 💚 E2E
uses the count bus in the kopia backup package.
This currently duplicates counts that we're getting
from the kopia stats. A later pr will remove the old
stats entirely in favor of the counter.
---
#### Does this PR need a docs update or release note?
- [x] ⛔ No
#### Type of change
- [x] 🧹 Tech Debt/Cleanup
#### Test Plan
- [x] ⚡ Unit test
- [x] 💚 E2E
We aren't using this property as an enum, nor does it look like we have reason to do so in the future. Currently, the enumeration values obscure the point of tracking the property. This value change should make that more clear.
Next PR will handle identifying parent-child package relationships.
---
#### Does this PR need a docs update or release note?
- [x] ⛔ No
#### Type of change
- [x] 🐛 Bugfix
#### Test Plan
- [x] ⚡ Unit test
- [x] 💚 E2E
<!-- PR description-->
Recent PR #4456 rolled back some of newer test additions. This PR reintroduces these tests.
---
#### Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
#### Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
#### Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>
#### Test Plan
<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ] ⚡ Unit test
- [ ] 💚 E2E
<!-- PR description-->
URL cache should use the same delta token base used by `ProduceCollections`. This is because we want to update URLs for the set of items which was discovered by `ProduceCollections`. Otherwise, we'll have url cache misses & we'll have to resort to doing GetItem graph calls for each item which means we lose all goodness of the url cacche.
This reverts a change introduced in a recent [PR](5215e907b0 (diff-f79ab9c9b31db3600d4e14b6df4d4c679b4356e32c3a68eaeaeb934c1c5a13a7R175)) where we are updating cache's delta token base to a newer one.
---
#### Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
#### Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
#### Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>
#### Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [ ] ⚡ Unit test
- [ ] 💚 E2E
Previously we were trying to fetch channels of Groups without channels.
---
#### Does this PR need a docs update or release note?
- [x] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ] ⛔ No
#### Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
#### Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* closes https://github.com/alcionai/corso/issues/4519
#### Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [ ] ⚡ Unit test
- [x] 💚 E2E
This enhances the check that allows skipping OneNote files that cannot be downloaded
with a mime type check.
This handles cases where the folder the item is in does not have the `package` facet.
---
#### Does this PR need a docs update or release note?
- [x] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ] ⛔ No
#### Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
#### Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>
#### Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
<!-- PR description-->
---
#### Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
#### Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [x] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
#### Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>
#### Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [ ] ⚡ Unit test
- [ ] 💚 E2E
Protective measure to prevent further details merging issues.
will need a follow-up to enable via a feature flag for testing.
---
#### Does this PR need a docs update or release note?
- [x] ⛔ No
#### Type of change
- [x] 🐛 Bugfix
#### Issues
- #2023
#### Test Plan
- [x] ⚡ Unit test
- [x] 💚 E2E
corrects the prior PRs full-scale batching of driveItems by adding a channel to the api layer's enumerateItems and driveEnumerateItems funcs. To avoid changes
in most parts of code, also adds batch wrappers
that hide these changes from existing pager usage.
DriveItem collection processing now ranges over
a channel of pager results, allowing it to process pages in a stream rather than batching all items.
---
#### Does this PR need a docs update or release note?
- [x] ⛔ No
#### Type of change
- [x] 🧹 Tech Debt/Cleanup
#### Test Plan
- [x] ⚡ Unit test
- [x] 💚 E2E
refactors the common drive sanity checks to use the sanitree data container. Also expands the sanitree in two ways:
1. adds leaves (individual items) to nodes for granular data comparison
2. adds multi-type support for comparing nodes of different types.
---
#### Does this PR need a docs update or release note?
- [x] ⛔ No
#### Type of change
- [x] 🤖 Supportability/Tests
#### Issue(s)
* #3988
#### Test Plan
- [x] 💪 Manual
- [x] 💚 E2E
To assist in debugging via logs, add the set of
categories an operation is acting on to the
clues set in the context
---
#### Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
#### Type of change
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
#### Test Plan
- [x] 💪 Manual
- [ ] ⚡ Unit test
- [ ] 💚 E2E
#### Does this PR need a docs update or release note?
- [x] ✅ Yes, it's included
#### Type of change
- [x] 🐛 Bugfix
#### Test Plan
- [x] ⚡ Unit test
- [x] 💚 E2E
enforce the following:
* no mod times returned from the api layer are zero-valued
* dttm.OrNow() provides a zero value time check
* ptr.OrNow() calls dttmOrNow on non-nil time values.
* itemPagers for getAddedAndRemoved call ptr.OrNow()
* callers to getAddedAndRemoved provide a transformer func which produces the mod time from the paged item, instead of assuming all items use the same interface for mod time.
---
#### Does this PR need a docs update or release note?
- [x] ✅ Yes, it's included
#### Type of change
- [x] 🐛 Bugfix
#### Test Plan
- [x] ⚡ Unit test
- [x] 💚 E2E
Move the Reasoner implementation from the kopia package to the identity
package. This will help avoid import cycles if we want to start
persisting Reason information in the backup model.
---
#### Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
#### Type of change
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
#### Test Plan
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [x] 💚 E2E
Renames focus on OneDrive and Libraries handlers.
Files are also renamed to match
item -> userDrive
library -> site
If diff is large due to file renames view by
commit since file renames are done in a separate
commit
---
#### Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
#### Type of change
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup
#### Issue(s)
* #4254
#### Test Plan
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [x] 💚 E2E
Do a couple of high-level things with drive handlers:
* pull out generic functionality into a base handler
* use handler-specific implementations of AugmentItemInfo
* remove old switch-based augmentItemInfo
Overall goal is to reduce places we use service type
comparisons to handle behavior differences
---
#### Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
#### Type of change
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup
#### Issue(s)
* #4254
#### Test Plan
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [x] 💚 E2E
Have a way to gather stats about the exported data.
Users can now call `ExportOperation.GetStats()` at the end of the run to get the stats for the operations. The data will be in the format `map[path.CategoryType]data.KindStats` whre `KindStats` is:
```go
type KindStats struct {
BytesRead int64
ResourceCount int64
}
```
---
#### Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
#### Type of change
<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
#### Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* https://github.com/alcionai/corso/issues/4311
#### Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
some additional logging context for collection scope.
---
#### Does this PR need a docs update or release note?
- [x] ⛔ No
#### Type of change
- [x] 🤖 Supportability/Tests
#### Test Plan
- [x] ⚡ Unit test
- [x] 💚 E2E
<!-- PR description-->
This fixes a perf regression in #4456
Context:
URL cache only needs a subset of drive item properties while doing delta queries. See https://github.com/alcionai/corso/pull/4074 for details. Changes in #4456 were applying the default item property set for all delta enumerator consumers including URL cache. This PR fixes the memory regression.
---
#### Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
#### Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
#### Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>
#### Test Plan
<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ] ⚡ Unit test
- [ ] 💚 E2E
handle cases where a resource is found, but is not accessible due to being locked out by an administrator or msoft process.
---
#### Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
#### Type of change
- [x] 🐛 Bugfix
#### Issue(s)
* #4464
#### Test Plan
- [x] ⚡ Unit test
- [x] 💚 E2E
<!-- PR description-->
**Changes:**
1. Count item download requests under drive rate limit quota . Currently this is under exchange limiter.
2. Use 1 rate limit token instead of 2 for drive calls by default.
3. Use 2 tokens instead of 1 for initial delta query ( which has no token).
Sharing internal docs separately to go along with the review.
---
#### Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
#### Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
#### Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>
#### Test Plan
<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ] ⚡ Unit test
- [ ] 💚 E2E
<!-- PR description-->
`TestControllerIntegrationSuite/TestRestoreAndBackup_core/MultipleContactsInRestoreFolder` is failing right now for CI. We are failing while comparing contact categories [here](8e4d320b21/src/internal/m365/helper_test.go (L321)).
expected: empty
got: Slice of 1 -> `"Corso_Restore_10-Oct-2023_13-54-32"`
We started hitting this failure today in both CI as well as local test runs. This appears to be caused by a graph transition. Disabling the category checks temporarily to unblock PRs. Meanwhile, will keep an eye on graph behavior for the next few days.
---
#### Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
#### Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
#### Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>
#### Test Plan
<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ] ⚡ Unit test
- [ ] 💚 E2E
