63 Commits

Author SHA1 Message Date
ashmrtn
dcf02f6256
Minor refactoring of graph error checking code (#3686)
Minor code cleanup to error checking and calling code:
- Remove unused function
- Update error check for folder not found
- Remove duplicate check in SDK function

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* fixes #3684

#### Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-06-28 21:21:28 +00:00
Keepers
d9f6a9297f
fix bad user info conditional on spo license (#3699)
Primary change is fixing the spo license check in
GetUserInfo from a || to a && !.  Currently the check returns the spo license error instead of treating it as a drive access condition.

Additionally adds a root site check in the m365
isServiceEnabled call, instead of always returning true for sharepoint.  This should catch spo license issues on a per-site basis, if someone makes it that far.

Finally, adds extra testing (plus some test code
consolidation and refactoring) to ensure GetUserInfo is properly tested.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🐛 Bugfix

#### Issue(s)

* #3671

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-06-28 18:03:17 +00:00
Keepers
df100b9b3b
use errors.As to extract odataError (#3690)
casting only the top level error to an odataError
will slice out useful data when the odataError is
wrapped, stacked, or otherwise nested.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🐛 Bugfix

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-06-27 23:56:27 +00:00
Keepers
c7ce9f4c0e
make tests less sus (#3689)
#### Type of change

- [x] 🤖 Supportability/Tests

#### Test Plan

- [x]  Unit test
2023-06-27 22:16:13 +00:00
ashmrtn
ce3ecacbe2
Add lint rule to use any instead of interface{} (#3685)
#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #3654

#### Test Plan

- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-06-27 20:34:30 +00:00
ashmrtn
4f9793a824
Handle broader set of user not found error codes (#3678)
graph has been inconsistent about what error code is returned
when the user doesn't exist. Expand the set of things we
check so that hopefully we'll get them all

Always return an error if the user doesn't exist as determined
by checking for exchange folders during GetInfo calls

Also add a little more stack trace info to returned errors

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

* fixes #3672

#### Test Plan

- [x] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-06-27 18:59:33 +00:00
Keepers
0451e933d5
handle no sharepoint license err (#3673)
adds handling for graph api responses
where the tenant does not have an active
sharepoint license.  For user's drives, this
will fall under the same "service not enabled"
behavior we use to determine if the drive can
get backed up.  For sites, this causes the
"get all sites" call to return a serviceNotEnabled error.

---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included

#### Type of change

- [x] 🐛 Bugfix

#### Issue(s)

* #3671

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-06-27 02:26:47 +00:00
Keepers
1bfa195b33
move default rate limit to per-second (#3637)
The current rate limit default maximizes the number of calls we can make in the first 10 minutes, but the larger initial bucket size means we have to drop the per-second token burst.  As a result, any process longer than 10 minutes is going to see unnecessary rate limiting by ~100 calls per minute.

This change swaps the large initial bucket for a smaller one, and a larger token burst to increase throughput across long runtimes.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-06-23 00:25:16 +00:00
Abin Simon
d9b5cda8f1
Fix uploading large attachments for exchange (#3634)
Upload of large attachments were broken previously.

Previously we were relying on the size reported by Graph API, but
that is not reliable and we were not completing uploads because of that.

<!-- PR description-->

---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [ ]  Unit test
- [x] 💚 E2E
2023-06-21 07:40:41 +00:00
Keepers
416383a99c
cascade restoreCfg collision policy into onedrive (#3623)
Adds collision policy handling to onedrive item posts.  This allows us to override the default "replace" behavior that currently returns a 409 for the creation endpoint, in case we want to use skip (ie: fail) or copy handling.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🌻 Feature

#### Issue(s)

* #3562

#### Test Plan

- [x] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2023-06-16 00:07:09 +00:00
Abin Simon
6e2d72509c
Skip any attachment fetches that fail with OLE conversion error (#3607)
These files from what we can understand are not available and thus can't be fetched. This change ensures that we don't fail the backup just because of this error.

<!-- PR description-->

---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-06-14 05:09:07 +00:00
Keepers
ce72acbcc1
auto-log recoverable errors with stack (#3598)
automatically log when we add a recoverable error or a skipped item to fault.  This log will include a stack trace of the call from the location of the logged recoverable.  Clues does not have a method for pulling a stack trace out of an error yet; that can be added at a future date.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🤖 Supportability/Tests

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-06-13 22:18:18 +00:00
Keepers
2f6d731993
rename connector -> m365 (#3600)
renames /internal/connector to /internal/m365.  No logic changes in this PR.  Only the dir rename, import renames, and one linter shadowing rename.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #1996

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-06-13 18:35:39 +00:00