929 Commits

Author SHA1 Message Date
Keepers
1f8c7598b4
separate fault bus in fetch manifests (#3090)
The global aggregation of recoverable errors
causes us to mark operations as failed even in
conditions where they should have succeeded,
because higher level callers don't have sufficient
control over ignorable error conditions.  This
is a short-term fix that allows us to log and dump
fault error accumulation in a specific, scoped instance.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🐛 Bugfix

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-04-12 01:16:40 +00:00
Keepers
30c86797d3
always include delta metadata, additional logging (#3092)
Ensure exchange always includes the delta metadata entry, even if it has zero deltaURLs to record.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🐛 Bugfix

#### Issue(s)

* fixes #3019

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-04-12 00:43:55 +00:00
ashmrtn
ab3011cab4
Log resource owner name, not full selector during backup (#3098)
#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

* closes #3096

#### Test Plan

- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-04-12 00:05:39 +00:00
Keepers
50e92b65c6
add Concealer compliance to paths (#3017)
Adds compliance with clues.Concealer to the paths
package.  Also introduces a new struct in the same
space: Elements, which is a thin wrapper around
a slice of strings so that subsections of a path
or builder can carry the same pii behavior
without additional work on the consumer's end.

---

#### Does this PR need a docs update or release note?

- [x] 🕐 Yes, but in a later PR

#### Type of change

- [x] 🌻 Feature
- [x] 🤖 Supportability/Tests

#### Issue(s)

* #2024

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-04-11 22:04:34 +00:00
Keepers
10a5d21cc9
remove "errors" from log message (#3087)
#### Type of change

- [x] 🤖 Supportability/Tests
2023-04-11 19:26:03 +00:00
ashmrtn
cae3debd28
Lint for using suite.T().Run instead of testify's suite.Run (#2664)
Includes fixups for calls that need updated

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Test
- [x] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #2373

#### Test Plan

- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-04-11 16:39:39 +00:00
Keepers
21cd7210cb
remove weburl suffix support from sharepoint (#3082)
We're dropping support for selection-by-url-suffix in sharepoint.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #2825

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-04-11 16:09:58 +00:00
ashmrtn
2ebab1a78b
Use prefix matcher when merging backup details (#3055)
Store all locations in the prefix matcher
and then look them up when merging details

Intermediate step to get things setup for
having OneDrive locations during merging

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #2486

#### Test Plan

- [x] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-04-11 03:23:13 +00:00
Keepers
7c9eada5a9
optionally look up and union fallback manifests (#3057)
Fallback manifests allow us to find manifests and
metadata from snapshots that utilized earlier
versions of kopia Reasons, primarily across
resource owner deltas.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🌻 Feature

#### Issue(s)

* #2825

#### Test Plan

- [x] 💪 Manual
- [x]  Unit test
2023-04-11 02:47:11 +00:00
Abhishek Pandey
2300727b09
Sanitize recurrenceTimezone prior to restoring calendar event (#3064)
<!-- PR description-->
* Do not specify recurrenceTimeZone if it's not set in the item being
restored. "" is not a valid value
---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->

- [x] 🐛 Bugfix


#### Issue(s)
* COR-53

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💚 E2E
2023-04-10 13:17:53 -07:00
ashmrtn
89ac00e64e
Add basic prefix matcher type (#3054)
Adds a basic generic prefix matcher type. The
first implementation just uses a map and iterates
through things

This is expected to be used for at least the
exclude list for backups (inner type of
`map[string]struct{}`) and logic for merging
backup details (inner type still TBD a bit,
but either `*path.Builder` or a custom struct)

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #2486

#### Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-04-10 19:11:53 +00:00
ashmrtn
27909592b8
Exit early if context is marked as done (#2883)
Keeps the code from blocking and then failing immediately without sending a retry.

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* closes #2882

#### Test Plan

- [ ] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-04-10 18:38:14 +00:00
Georgi Matev
b6ae9c6d07
Switching user identification and hashing (#3071)
Encompasses the following changes:
* distinct ID for users and events set to m365 tenant ID hash
* still record local repoId
* switches the hashing method for the m365 tenant ID to tuncated sha256
* continue logging deprecated md5 hash for a few releases to facilitate event merge

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-04-10 16:44:23 +00:00
Keepers
896c05f623
look up users and sites by id or name (#2973)
Adds a lookup step to graph connector to find
an owner's id and name given some identifier.
The identifier, for either sites or users, can be a
well formed id or name.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🌻 Feature

#### Issue(s)

* #2825

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-04-07 22:29:09 +00:00
Keepers
3cc29649ed
Add GetUserInfo to services/m365 (#2991)
Adds an interface for GetUserInfo, which gets
corso-specific metadata for a single user.  Also
does some refactoring around discovery for
better interface consistency, both as variables
and as access layers.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🌻 Feature

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-04-07 00:36:50 +00:00
Keepers
0fd1314479
use clues with plaintext as default (#3058)
#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🤖 Supportability/Tests

#### Issue(s)

* #2024

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-04-06 23:15:20 +00:00
Abin Simon
baddd9fc83
Get OneDrive item with getM365 (#2791)
Add support for onedrive in getM365

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-04-06 05:32:03 +00:00
ashmrtn
59e17757bf
Switch LocationPath return type (#3038)
LocationRefs don't need prefix directories, so using a path.Path is
overkill. This will also fit better with changes to come where we need
to parse and update the LocationRef of merged backup details entries.

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #2486

#### Test Plan

- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-04-05 21:10:42 +00:00
Keepers
0d521bb5a7
move sharepoint onto map-based lookup (#2958)
Refactors the sharepoint site lookup to use the
id-to-name maps.  This has a momentary regression
that will get solved in the next PR: we no longer
match on weburl suffixes, and instead require
a complete match.

Also, migrates the sharepoint lookup code out
of GC and into Discovery.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🌻 Feature

#### Issue(s)

* #2825
* #1995
* #1547

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-04-05 19:38:22 +00:00
Keepers
fac5a0f3d2
No warn logs (#3024)
#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Test Plan

- [x] 💪 Manual
2023-04-05 17:43:02 +00:00
Keepers
74e4a094e0
conceal url pii (#3015)
Adds a new package to common (pii) with a clues.
Concealer compliant struct that can log and report
url values with a mix of hidden pii and exposed
public values.  Exposed values are based on
well-known graph api path and query params.

---

#### Does this PR need a docs update or release note?

- [x] 🕐 Yes, but in a later PR

#### Type of change

- [x] 🤖 Supportability/Tests

#### Issue(s)

* #2024

#### Test Plan

- [x]  Unit test
2023-04-04 21:01:37 +00:00
Keepers
2341d61842
introduce id-name lookup maps (#2955)
Adds two maps to resource-owner handling:
id-to-name and name-to-id.  Expectation is that
these maps will either get populated by a caller
as a pre-process before initializing the gc client, or
gc will (later pr) be able to look up the owner and
populate those maps itself.  The maps are
used to set the selector id and name for iface
compliance.  Only supported by exchange in this PR.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🌻 Feature

#### Issue(s)

* #2825

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-04-04 20:21:58 +00:00
Keepers
fbb867a08d
add clues.Hide to various logs (#2994)
Begins hiding PII with clues.Hide injections in our structured logging aggregators.  This is a Non-
comprehensive change, and improvements here will be ongoing.

---

#### Does this PR need a docs update or release note?

- [x] 🕐 Yes, but in a later PR

#### Type of change

- [x] 🤖 Supportability/Tests

#### Issue(s)

* #2024

#### Test Plan

- [x] 💪 Manual
- [x] 💚 E2E
2023-04-04 16:59:31 +00:00
Abin Simon
4e1a44fbee
Remove unnecessary indirection for retries (#3021)
This was previously used to add retry logic which is no more the case as is a completely unnecessary indirection.

<!-- PR description-->

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-04-04 16:05:25 +00:00
Abin Simon
c696e7b7ff
Enable permissions backup by default (#2981)
Previously we had a flag that needed to be flipped to backup permissions. Now we always backup permissions.

- Should we wait till https://github.com/alcionai/corso/issues/2976 is done to do this
- Users will have to do a non incremental backup first as of now to get permissions. We should probably have some way to force this.

---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* closes https://github.com/alcionai/corso/issues/2409

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2023-04-04 03:55:44 +00:00
Keepers
99774e754b
introduce idnamer (#2934)
Adds a common interface: idNamer, which is used
to pass around tuples of an id and a name for some
resource.  Also adds compliance to this iface in
selectors, where a selector's ID and Name are the
DiscreteOwner values.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🌻 Feature

#### Issue(s)

* #2825

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-04-03 22:42:48 +00:00
ashmrtn
16d1e68ebf
Populate location for all Exchange data types (#2829)
Always populate the location field of backup details
for Exchange data types

Fix bug where ParentPath for calendar items used
folder IDs instead of display names

---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

- [x] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #2486
* closes #2827

merge after:
* #2826
* #2808
* #2828

#### Test Plan

- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-04-03 20:22:31 +00:00
Abin Simon
4cc0397075
Log responses for 503 (#3012)
Currently we were logging rate limiting values for 503 which is not available and end up with empty values. Switching it to logging the response, similar to 404 as that can be more useful.

<!-- PR description-->

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-04-03 17:32:44 +00:00
Keepers
dcf5ea2415
update failure logs (#2998)
Co-authored-by: aviator-app[bot] <48659329+aviator-app[bot]@users.noreply.github.com>
2023-04-03 14:40:01 +00:00
neha_gupta
4087b69a39
Enhancement Sanity Test: verify permissions and defined data (#2938)
<!-- PR description-->
onedrive -  
- verify permissions - grantedToV2 and roles 
- recursively check subfolder's data and permission

Exchange -
- since the data in email is huge, create custom data and check if that data is getting restored properly
 
#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Supportability/Tests
- [x] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-04-03 10:43:48 +00:00
Keepers
f2bf0ee685
introduce fail-after-recovery option (#3005)
While fail-fast and best-effort make for good
categories at the extreme ends of error handling, 
we keep finding outselves wanting to operate
in a middle ground.  This change introduces a new
error handling category: FailAfterRecovery.  This
option tells corso to complete as much of its
process as it can, even if it recovers from errors.
But at the end of processing, if it recovered from
any errors, an error is returned for the operation.

This behavior is the new failure handling default,
instead of failFast.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🌻 Feature

#### Test Plan

- [x] 💪 Manual
- [x]  Unit test
2023-04-01 00:43:52 +00:00
Keepers
983aaabdb2
restrict sharepoint test backups to test folders (#2969)
To avoid over-production of data, and thus longer
CI times, any sharepoint integration test that can scope
library usage down to only the "test" folders should do so.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🤖 Supportability/Tests

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-03-31 23:47:34 +00:00
Keepers
b81a527c10
add clues.Hide to replace observe.PII (#2993)
The observe PII handlers are getting replaced by
the more standardized clues secrets.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #2024

#### Test Plan

- [x] 💪 Manual
- [x] 💚 E2E
2023-03-31 22:14:09 +00:00
Keepers
6190b1ba05
use equalfold in filter equals checks (#2995)
Because it's nifty.

-----

#### Type of change

- [x] 🧹 Tech Debt/Cleanup
2023-03-31 16:36:19 +00:00
Keepers
7e04cefa2c
replace graphConnector with interface (#2932)
Replaces the operations graphConnector reference
with an interface.  Restore and Backups have
separate, unique interfaces.

<!-- PR description-->

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🌻 Feature

#### Issue(s)

* #2825

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-03-31 15:56:19 +00:00
Abin Simon
446ccfe491
Skip large OneNote items that cannot be downloaded (#2925)
OneNote files >2GB cannot be downloaded via the graph API. This change will gracefully skip them and prevent the backup from erroring out.

A OneNote file is represented by a Package(folder) which contains two file within it, `<file>.one` and `<file>.onetoc2`.  From what I can tell the `onetoc2` file is the metadata file and the `one` file is the one that contains data and can get big.

**The current implementation has a limitation(or feature depending on how you see it) in that we will end up doing a partial backup with just the `onetoc2` file but avoiding the big `one` file if that is big.**

I did not try to change the behaviour as even currently if for somehow we ended up with an error like 404 when trying to backup one of these files, we do skip it. I've added a comment in for now, let me know if we should go back and revisit the behaviour. *The main issue is the fact that we only hit this problem once it reaches kopia and removing it at that point is harder.*

---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* fixes https://github.com/alcionai/corso/issues/2910

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-03-31 04:43:34 +00:00
Keepers
25cffcdf8d
create a new http client on every delete (#2970)
in order to avoid errors cascading from graph API's
populated 204 responses containing `0/r/n/r/n`,
in particular in response to deletes, this change
hacks in the creation of a unique http client for
each delete call.

This is dangerous to do, since we already know
that generating clients per calls can leak
consumption of system resources such as sockets.
This change presumes that runtime deletes are
minimally, if ever, called during standard backup
and restore.  The permission deletion in onedrive
is the stand-out exception that might need some
additional investigation.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🤖 Supportability/Tests

#### Issue(s)

* #2707

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-03-30 15:56:53 +00:00
Abin Simon
d99b125087
Do not try to call syscall.Kill on Windows (#2986)
Will create a followup to make sure we are doing the right thing on Windows. This will unblock builds for now, although we don't get the metrics for Windows(which was not working anyways).

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [x] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-03-30 08:14:33 +00:00
Keepers
4a395d44a8
move graph connector creation to new op (#2923)
Moves the generation of graphConnector to the
NewBackupOp and NewRestoreOp constructors
inside the repository.  This removes gc creation
from the backup and restore operations, and
requires a reference to gc to exist in operations.

This sets up two changes: 1/ mocking of GC
within operation (the next PR will replace it with
an interface).  2/ the ability to perform a gc
validation step within the operation construction,
and thus the capacity to look up a resource owner
display name or id, and vice versa.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🌻 Feature

#### Issue(s)

* #2825

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-03-29 22:00:30 +00:00
Keepers
e09c120778
add linter for using clues to create and wrap errs (#2959)
#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #1970

#### Test Plan

- [x]  Unit test
2023-03-29 21:16:05 +00:00
Abin Simon
6b00548a0b
Integration test for permissions changes (#2965)
This adds basic integration tests for backing up permissions in OneDrive.
*https://github.com/alcionai/corso/issues/2790 could be a good follow up to this.*

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* fixes https://github.com/alcionai/corso/issues/2772

#### Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [ ]  Unit test
- [x] 💚 E2E
2023-03-29 04:45:25 +00:00
ashmrtn
91e0a27831
Populate location in Exchange container resolver (#2828)
Always generate both a path of IDs and a path of
display names for folders in the container resolver.

Changes in internal/connector/exchange/service_functions.go
keep the behavior of the overall system from changing

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #2486

merge after:
* #2808

#### Test Plan

- [x] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-03-29 01:19:59 +00:00
Keepers
940892fc07
correct input aliasing (#2960)
Updates and corrects input aliasing according to
the following rules (in priority order):
1. if the library name is usable, use it
2. if not, alias to the package name
3. if the package name is weird, alias sensibly
4. in case of collision, alias more distant imports
5. aliases should be consistent throughout

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #1970

#### Test Plan

- [x]  Unit test
2023-03-28 20:48:22 +00:00
Keepers
214206b75e
avoid segfaults in metrics middleware (#2972)
#### Type of change

- [x] 🐛 Bugfix
2023-03-28 20:13:58 +00:00
ashmrtn
79b6c0957f
Re-enable import order linting (#2928)
Fixes current errors as well

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* closes #2927

#### Test Plan

- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-03-28 16:45:28 +00:00
Keepers
9f0e259764
metrics PoC for github.com/armon/go-metrics (#2884)
Adds a logger-oriented metrics tracker to aggregate statistics like requests-per-minute and average
http request time.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🤖 Supportability/Tests

#### Issue(s)

* #2212

#### Test Plan

- [x] 💪 Manual
2023-03-28 15:57:55 +00:00
ashmrtn
172155654d
File name compare (#2956)
Check the file name recorded in the ItemInfo (and used during restore) matches the expected file name for the item. We can't directly compare file names because files are now stored by ID in kopia.

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #2897

#### Test Plan

- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-03-28 06:03:23 +00:00
Keepers
9d73d0c8e2
replace errors.* with clues.* (#2924)
Mostly find/replace on errors.N and errors.W. Also turns all wrapf into wrap, and removes as many errorf calls as possible.

Might follow up with a linter to enforce this change.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #1970

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-03-28 04:01:54 +00:00
Keepers
8acff1ad93
fix sharepoint library folder selection (#2962)
Bugfix:
Sharepoint LibraryFolder scopes always return
zero matches in filtering with a prefix due to the
path not having the drive root prefix truncated, as is
expected of onedrive-based storage paths.

---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included

#### Type of change

- [x] 🐛 Bugfix

#### Test Plan

- [x] 💪 Manual
- [x]  Unit test
2023-03-28 01:21:20 +00:00
ashmrtn
1548d51004
Add basic RestoreAndBackup test for SharePoint (#2935)
Reuses a lot of the OneDrive logic to run
the test. Only tests basic things as it's
assuming the OneDrive set of tests will do
a more thorough job testing the overall
logic for backup/restore and permissions

Doesn't test permissions as those aren't
yet enabled for SharePoint

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #2897

#### Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-03-27 22:41:50 +00:00