744 Commits

Author SHA1 Message Date
Danny
afe8f04a45
Merge branch 'beta-library-service' into backup-page-flow 2023-01-27 11:02:10 -05:00
Danny Adams
946767edcc Add file /graph/betasdk/beta_service.go 2023-01-27 10:50:46 -05:00
Danny Adams
3228bfbe1e Updates to /graph/betasdk/beta_client.go
Comments updated for clarity on package setup.
2023-01-27 10:37:42 -05:00
Danny Adams
447ae1f3bb Updates to /graph/betasdk/kiota-lock.json
Key/Value pair added for Beta version.
2023-01-27 10:35:52 -05:00
Danny Adams
2133cc41f1 Update to /graph/betasdk/kiota-lock.json
Changes to JSON fields for compliance with parsing.
2023-01-27 10:23:00 -05:00
Danny
9deb1ce9a2
GC: Backup: SharePoint: Test Fetch/Get Functions (#2182)
## Description
Tests the functionality of GC's retrieval functions
2023-01-27 09:40:32 -05:00
Danny
9aa7c1b667
Merge branch 'beta-library-switch' into backup-page-flow 2023-01-27 09:34:40 -05:00
Danny Adams
d7e077ad72 Merge branch 'beta-library-switch' of github.com:alcionai/corso into beta-library-switch 2023-01-26 17:58:54 -05:00
Danny Adams
db9a91fb87 Updates to /betasdk/sites/kiota-lock.json
Details the required files for the subdirectories of models and sites.
2023-01-26 17:58:40 -05:00
Danny
57ce0be82a
Merge branch 'main' into beta-library-switch 2023-01-26 17:17:00 -05:00
Danny Adams
b3aec1b40f Updates /graph/beasdk/beta_service_test.go
File renamed to beta_client_test.go.
Comments updated.
2023-01-26 16:16:43 -05:00
Danny Adams
f8d65d66de Updates to /graph/betasdk/beta_client.go
Registers serializers for connector.
Adds explicit comments on connector usage and patterns.
Removes Sites() call from supported commands.
2023-01-26 16:08:25 -05:00
Danny Adams
817a7a2bc6 UPdates to /graph/betasdk/sites package
Final sweep to bring src files down to bare minimum for Site Page management.
2023-01-26 15:51:39 -05:00
Danny Adams
8a474af189 Revert back to msgraph-sdk-go library for /support/errors.go 2023-01-26 14:02:55 -05:00
Danny Adams
b7b2de096b Revert back to msgraph-sdk-go package for /connector/graph/errors.go 2023-01-26 14:02:15 -05:00
Danny Adams
03ec1dd2e9 Updates to remove beta library from operations/backup_integration_test.go 2023-01-26 14:01:24 -05:00
Danny Adams
263197bbab Updates to /betasdk/sites/ for GOFMT:
Fixes for linter failures --> ignore statements.
2023-01-26 11:39:37 -05:00
Vaibhav Kamra
bdb7f2b109
Re-use client for OneDrive item downloads (#2276)
## Description

This PR reintroduces the changes from #2266 with a change to *not* reset the transport
when initializing the shared client. 

Doing so was removing the retry and other middlewares
and also resulting in throttled requests being masked as success

Also - we now decorate our download traffic with an ISV tag as recommended [here](https://learn.microsoft.com/en-us/sharepoint/dev/general-development/how-to-avoid-getting-throttled-or-blocked-in-sharepoint-online#how-to-decorate-your-http-traffic)

## Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [x] 🕐 Yes, but in a later PR
- [ ]  No 

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #2266 

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [ ]  Unit test
- [x] 💚 E2E
2023-01-26 15:05:12 +00:00
Danny Adams
bcd379f596 Updates to betasdk/sites package
Dependencies updated and minimal files supplied.
2023-01-26 08:24:09 -05:00
Vaibhav Kamra
0587df91ef
Remove OneDrive license GUID check (#2275)
## Description

This check is brittle and shouldn't be needed. It should be sufficient to check whether a user has a 
drive or not (similar to what we now do for exchange mailboxes).

Validated with the existing integration test.

## Does this PR need a docs update or release note?

- [x]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No 

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #2271

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-01-26 08:52:39 +00:00
Vaibhav Kamra
e9354a8429
Revert "add itemClient for re-usable od item downloads" (#2273)
Reverts alcionai/corso#2266. Am seeing issues on data restore that need to be debugged.
2023-01-26 07:52:56 +00:00
Keepers
bda8a5c60c
add itemClient for re-usable od item downloads (#2266)
## Description

onedrive currently constructs a new http client
for every file it downloads.  This causes the OS
to generate extra sockets, and hang onto them
after the download is complete.  Replacing these
one-off clients with a singular, re-used client-
which is the behavior and standard suggested
for golang http clients- minimizes system
resource consumption.

## Does this PR need a docs update or release note?

- [x]  No 

## Type of change

- [x] 🐛 Bugfix

## Issue(s)

* closes #2262

## Test Plan

- [x] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2023-01-26 04:27:52 +00:00
Danny Adams
aca2eb5d65 Updates to betasdk/sites package
Package only to support SitePageable objects. Process begun to removal
of queries that do not support these operations.
2023-01-25 21:34:36 -05:00
Danny Adams
2f02efe489 Reduction to /graph/betasdk/models/package
Referenced models from v1.0 and retained minimial for Page retrieval.
2023-01-25 20:52:07 -05:00
Danny Adams
3e4fae6a56 Removal of odataerrors withing betasdk.
Files are present in main package.
2023-01-25 20:16:49 -05:00
Danny Adams
775f5ce4e3 Updates to /connector/graph/betasdk/sites
Removal of Async calls and file permissions for adapter compatibility.
2023-01-25 17:57:11 -05:00
Danny Adams
8f22c4f776 Non-Page files removed. 2023-01-25 17:18:46 -05:00
Keepers
78b9f2752e
adds clues package to begin structured logging (#2214)
## Does this PR need a docs update or release note?

- [x]  No 

## Type of change

- [x] 🧹 Tech Debt/Cleanup

## Issue(s)

* #1969

## Test Plan

- [x] 💪 Manual
2023-01-25 21:39:38 +00:00
Danny Adams
9dc487b39f Test file created. 2023-01-25 16:39:28 -05:00
Danny Adams
9bbb479fa3 Changes to /connector/discovery/api/users_test.go
Reverts library back to msgraph-sdk-go.
2023-01-25 15:30:50 -05:00
ashmrtn
bb73d142c5
Remove duplicate string->path.CategoryType helper function (#2260)
## Description

Tackles a few things:
* normalize case when going from string->path.Service or path.Category
* removes duplicate helper function for string->path.Category in favor of path package one

## Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No 

## Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup

## Issue(s)

* closes #2259

## Test Plan

- [x] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-01-25 20:13:50 +00:00
Danny Adams
918fd835ad Updates library to use betasdk package. 2023-01-25 14:33:11 -05:00
Danny Adams
42a9499b71 Add file to /graph/betasdk/kiota-lock.json
File shows how app.kiota generator created the minimal SDK.
2023-01-25 14:31:16 -05:00
Danny Adams
740c8dc3e2 Updates to /graph/betasdk/beta_client.go
Removes the admin and groups properties for minimal beta connector.
2023-01-25 14:12:34 -05:00
Danny Adams
d2fb427146 Hydrate sites command directory. 2023-01-25 14:07:06 -05:00
Danny Adams
4fdd165afd Hydrate model directory. 2023-01-25 14:06:41 -05:00
Danny Adams
9a0dd673c6 Kiota library mismatch from preview. 2023-01-25 13:13:10 -05:00
Danny Adams
b4118627a6 Kiota library mismatch from preview. 2023-01-25 13:12:37 -05:00
Danny Adams
a51b6e8444 Kiota library mismatch from preview. 2023-01-25 13:11:03 -05:00
ashmrtn
a4f2f0a437
Add deleted files to OneDrive excluded file list (#2250)
## Description

Begin populating a global exclude set with files deleted from OneDrive. The set contains the IDs of files that have been deleted.

This is actually safe to return from GraphConnector now (though it's not returned) as it uses item IDs instead of item names.

Future PRs will need to address handling of (potentially) moved files

## Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No 

## Type of change

- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

## Issue(s)

* #2242 

## Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-01-25 16:57:30 +00:00
ashmrtn
46d61c7246
Add a set of items that will be excluded from base directories during backup (#2143)
## Description

Some external services like OneDrive do not have the ability to determine the path a deleted item used to exist at, they only know the item's old ID. This patch allows Kopia Wrapper to handle those items by implementing a global exclude set. The patch assumes that items in base directories are unique as the items in every base directory are checked against the set.

This is not wired to anything outside of Kopia Wrapper. Currently this feature is disabled as the passed value is always nil.

## Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No 

## Type of change

- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

## Issue(s)

* closes #2121 

## Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-01-25 16:28:09 +00:00
Danny Adams
1acc6b9e18 Hydrate sites package. 2023-01-25 09:51:12 -05:00
Danny Adams
9a6d58b18c Inflate admin package. 2023-01-25 09:50:15 -05:00
Danny Adams
b27ce3d0e2 Inflate Models package. 2023-01-25 09:46:34 -05:00
Danny Adams
f5d514470d Inflate admin package. 2023-01-25 09:44:25 -05:00
Danny Adams
0a1e0d249f Package configuration. 2023-01-25 09:34:15 -05:00
Danny Adams
b1f7ab51e9 beta_client beta client formatting. 2023-01-25 09:23:13 -05:00
Danny Adams
b13f5b6fb6 Update shuffle. 2023-01-25 09:21:18 -05:00
Danny Adams
e51f654e46 Delete /betaSDK/sites package. 2023-01-25 09:02:25 -05:00
Danny Adams
50da437421 Updates package name for betasdk package. 2023-01-25 08:47:45 -05:00