<!-- PR description-->
**Changes:**
1. Count item download requests under drive rate limit quota . Currently this is under exchange limiter.
2. Use 1 rate limit token instead of 2 for drive calls by default.
3. Use 2 tokens instead of 1 for initial delta query ( which has no token).
Sharing internal docs separately to go along with the review.
---
#### Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
#### Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
#### Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>
#### Test Plan
<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ] ⚡ Unit test
- [ ] 💚 E2E
hands the backup resource into the drive collection for the handler to use to record as the siteID
---
#### Does this PR need a docs update or release note?
- [x] ⛔ No
#### Type of change
- [x] 🐛 Bugfix
#### Issue(s)
* #3988
#### Test Plan
- [x] 💪 Manual
- [x] ⚡ Unit test
- [x] 💚 E2E
Temporarily remove the large file size check that controls skipping a OneNote file download that failed with a
503 error.
We've observed this error condition to be persistent even with smaller OneNote files.
This allows us to unblock backup for these resources while we investigate the issue.
---
#### Does this PR need a docs update or release note?
- [x] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ] ⛔ No
#### Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
#### Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>
#### Test Plan
<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ] ⚡ Unit test
- [ ] 💚 E2E
Leverage the generic item struct to inject
serialization format information for all
items
Unwires the old code that injected versions
in kopia wrapper but leaves some code in
the wrapper to strip out the serialization
format during restore
Future PRs should move the process of
pulling out serialization format to
individual services
Viewing by commit may make review
easier
---
#### Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
#### Type of change
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
#### Issue(s)
* #4328
#### Test Plan
- [x] 💪 Manual
- [x] ⚡ Unit test
- [x] 💚 E2E
Switch all metadata files (of all types) to use the generic unindexed
item type. Transitioned items include:
* previous paths and deltas for all services
* site list for groups
* drive .meta files
---
#### Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
#### Type of change
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup
#### Issue(s)
* #4191
#### Test Plan
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [x] 💚 E2E
Use generic lazy item implementation for drive
backups. This is a bit of a stop-gap
implementation since the drive logic is more
complex than exchange logic
Refactoring the drive logic to streamline it
would help reduce the number of functions
called to download a single item etc.
---
#### Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
#### Type of change
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup
#### Issue(s)
* #4191
#### Test Plan
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [x] 💚 E2E
groups collection item streaming is missing
recoverable errors, and is only logging at this time. This may be causing backups to succeed even
when errors should cause a failure.
---
#### Does this PR need a docs update or release note?
- [x] ⛔ No
#### Type of change
- [x] 🐛 Bugfix
#### Issue(s)
* #3988
#### Test Plan
- [x] ⚡ Unit test
- [x] 💚 E2E
This was an accidental mis-use of the skipped item pattern. Items deleted during in flight during backup due to race conditions do not count as permanent
skips in the way that other skipped items do.
---
#### Does this PR need a docs update or release note?
- [x] ⛔ No
#### Type of change
- [x] 🧹 Tech Debt/Cleanup
#### Issue(s)
* closes#4044
#### Test Plan
- [x] ⚡ Unit test
- [x] 💚 E2E
<!-- PR description-->
---
#### Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
#### Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup
#### Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>
#### Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [ ] ⚡ Unit test
- [ ] 💚 E2E
This was somehow sliced out of changes persisted
in prior branch merges. It re-adds persisting replies as part of message content retrieval.
---
#### Does this PR need a docs update or release note?
- [x] ⛔ No
#### Type of change
- [x] 🐛 Bugfix
#### Issue(s)
* #3989
#### Test Plan
- [x] 💪 Manual
- [x] ⚡ Unit test
Update the API for Item.Info to return an error.
This can then be leveraged to add lazy readers
to exchange backups
See #2023 for more info on how to add lazy
readers to exchange
---
#### Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
#### Type of change
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup
#### Issue(s)
* #2023
#### Test Plan
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [x] 💚 E2E
A Stream is a continuous transmission of data.
An item is a single structure. Crossing the two
definitions generates confusion.
Primarily code movement/renaming. Though there
is also some reduction/replacement of structs
where we'd made a variety of testable Item implementations
instead of re-using the generic mock.
---
#### Does this PR need a docs update or release note?
- [x] ⛔ No
#### Type of change
- [x] 🧹 Tech Debt/Cleanup
#### Test Plan
- [x] ⚡ Unit test
- [x] 💚 E2E
Only code movement, no code changes.
Moved services to `/internal/m365/services/{onedrive,sharepoint,exchange}`
Moved collections to `/internal/m365/collection/{drive,site}`
---
#### Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
#### Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup
#### Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>
#### Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [ ] ⚡ Unit test
- [ ] 💚 E2E
