1504 Commits

Author SHA1 Message Date
Danny Adams
da2d0d1779 Update /internal/connector/support/m365Support.go
Issue #2169: Serialization logic for SitePageable data types.
2023-01-21 19:54:37 -05:00
Danny Adams
e6c7bbbb9e Update pkg/selectors/sharepoint.go
Bug Fix: Carried forward into package. Allows for display of backed up data within details.
2023-01-21 17:36:30 -05:00
Danny Adams
7fbb06a69f Troubleshooting complete. 2023-01-21 17:35:37 -05:00
Danny Adams
13c9a847e2 Merge branch 'backup-page-flow-3' into backup-page-flow-3-cliPicked 2023-01-21 02:13:44 -05:00
Danny Adams
4b67bb4684 Bug Fix: Initialize value at 0 2023-01-21 02:13:16 -05:00
Danny Adams
f036d9dfb8 Updated to /connector/sharepoint/data_collections.go
Control.Options added to NewCollection() calls.
2023-01-21 02:08:29 -05:00
Danny Adams
f9e3a57ec3 Updates to /internal/connector/sharepoint/collection_test.go
Control option added to NewCollection() calls within tests.
2023-01-21 02:07:34 -05:00
Danny Adams
a0ea09e9a1 Update to /connector/sharepoint/collection.go
Ctrl option added to Collection Constructor.
2023-01-21 02:06:03 -05:00
Danny Adams
226ce05637 Linter Fix: Wording for documentation. 2023-01-21 01:19:13 -05:00
Danny Adams
806b4d3a86 Updates to /cli/backup/sharepoint.go
PR Suggestions applied.
2023-01-21 01:19:01 -05:00
Danny Adams
04f22b6070 Updates to /cli/backup/sharepoint.go
Adds PageItem option option.
2023-01-21 01:18:48 -05:00
Danny Adams
7076bc0e7f Updates /cli/backup/sharepoint_test.go
Test suite expanded to include sharePoint data selection.
2023-01-21 01:18:25 -05:00
Danny Adams
5abc237ac6 Updates to /cli/backup/sharepoint.go
Adds functionality for Selecting SharePoint Page as a data selector.
2023-01-21 01:18:07 -05:00
Danny
6d615810cd
CLI: Backup: SharePoint: Pages Enable (#2213)
## Description
Adds the functionality to select SharePoint Pages for backup.
<!-- Insert PR description-->

## Does this PR need a docs update or release note?
SharePoint commands remain hidden in terms of pages. 
- [x] 🕐 Yes, but in a later PR

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* closes  #2216 <issue>
* closes #2107

## Test Plan

<!-- How will this be tested prior to merging.-->
- [x]  Unit test
2023-01-21 05:50:23 +00:00
Danny Adams
0f16e5768f Update to /connector/sharepoint/data_collections.go
Finalize the backup workflow for SharePoint.Pages.
2023-01-21 00:40:33 -05:00
Danny Adams
3c7c156e72 Updates to /connector/sharepoint/collection_test.go TestSharePointPageCollection_Populate() added. Ensures serialization of single page completes without issue. 2023-01-20 23:57:15 -05:00
Danny Adams
d2c5ad45bf Update /internal/connector/sharepoint/collection.go
Logic for serialization of models.Pages completed.
2023-01-20 22:49:27 -05:00
Danny Adams
1b4d3c2b08 GOFMT: Name update. 2023-01-20 22:47:51 -05:00
Danny Adams
e153c43811 Updates to /internal/connector/sharepoint/collection.go
Abstraction applied to pipeline to dynamically populate List objects. Page objects are currently stubbed out.
2023-01-20 22:15:41 -05:00
Danny Adams
10ae8fd102 Merge branch 'backup-page-flow-2' of github.com:alcionai/corso into backup-page-flow-2 2023-01-20 22:03:34 -05:00
Danny Adams
d5e0801219 Updates to abstraction model. 2023-01-20 21:33:17 -05:00
Danny
b192f9866c
Merge branch 'backup-page-flow' into backup-page-flow-2 2023-01-20 20:47:27 -05:00
ashmrtn
ea2d9ceceb
Report errors when building details (#2206)
## Description

Report an error if an item in details that was sourced from a base snapshot doesn't have a previous path. Items that don't have a previous path cannot be sourced from a base snapshot's backup details meaning the item will be stored but will not be searchable/restorable by users

Logging was not used because no context is available in the kopia callback that is checking if the previous path is nil

## Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No 

## Type of change

- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

## Issue(s)

* closes #1915

## Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-01-20 21:40:48 +00:00
Danny
cd77f43fb2
CLI: Updates for Exchange Help (#2191)
## Description
Removes references to M365 ID as instruction.
<!-- Insert PR description-->

## Does this PR need a docs update or release note?

- [x]  No, self-documenting

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🐛 Bugfix


## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* closes #2190

## Test Plan

- [x]  Unit test
2023-01-20 21:01:20 +00:00
Niraj Tolia
7872201f73
Move to wow.js fork (#2210)
## Description

wow.js got relicensed. Switch to the MIT fork (the old license)

## Does this PR need a docs update or release note?

- [x]  No 

## Type of change

- [x] 🐛 Bugfix

## Test Plan

- [x] 💪 Manual
2023-01-20 20:13:25 +00:00
Keepers
b3d4b4687b
discovery api, filter guest and external users (#2188)
## Description

Adds the api client pkg pattern to the connector/
discovery package.  Most code changes are plain
lift-n-shift, with minor clean-ups along the way.

User retrieval is now filtered to only include
member and on-premise accounts.

## Does this PR need a docs update or release note?

- [x]  Yes, it's included

## Type of change

- [x] 🌻 Feature

## Issue(s)

* #2094

## Test Plan

- [x] 💪 Manual
- [x]  Unit test
2023-01-20 20:09:59 +00:00
ashmrtn
bb7f54b049
Update OneDrive metadata with folder moves (#2193)
## Description

Track folder moves and update the persisted metadata accordingly. Moves need to take into account subtree changes because OneDrive will not notify us of subfolders moving if nothing else was updated on the subfolder.

Handle updates by making a copy of the folder map for ease of updating and later on ease of finding out which collection paths have changed

## Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No 

## Type of change

- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

## Issue(s)

* #2120

## Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-01-20 19:18:59 +00:00
ashmrtn
7f44db6c7c
Fail operation if we failed to fetch item data (#2199)
## Description

If GraphConnector reported errors while trying to fetch item data, fail the entire operation. This stops silent failures, but is a big hammer at the moment because there's no checks for items no longer being available.

Error reporting is minimal as well, but some info should be in the log.

**This effectively disables best-effort as any failure will result in the operation being reported as failed**

## Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No 

## Type of change

- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

## Issue(s)

* #2196

## Test Plan

- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-01-20 18:59:15 +00:00
Danny
10efa650c7
Merge branch 'beta-library-switch' into backup-page-flow 2023-01-20 13:59:01 -05:00
dependabot[bot]
fd32770d2a
⬆️ Bump github.com/spf13/viper from 1.14.0 to 1.15.0 in /src (#2204)
Bumps [github.com/spf13/viper](https://github.com/spf13/viper) from 1.14.0 to 1.15.0.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/spf13/viper/releases">github.com/spf13/viper's releases</a>.</em></p>
<blockquote>
<h2>v1.15.0</h2>
<!-- raw HTML omitted -->
<h2>What's Changed</h2>
<h3>Exciting New Features 🎉</h3>
<ul>
<li>feat: add multiple endpoints support for remote by <a href="https://github.com/mozartz"><code>@​mozartz</code></a> in <a href="https://github-redirect.dependabot.com/spf13/viper/pull/1464">spf13/viper#1464</a></li>
</ul>
<h3>Enhancements 🚀</h3>
<ul>
<li>Add DocBlock to WatchConfig by <a href="https://github.com/glebik000"><code>@​glebik000</code></a> in <a href="https://github-redirect.dependabot.com/spf13/viper/pull/1467">spf13/viper#1467</a></li>
</ul>
<h3>Breaking Changes 🛠</h3>
<ul>
<li>Drop YAML v2 and TOML v1 by <a href="https://github.com/sagikazarmark"><code>@​sagikazarmark</code></a> in <a href="https://github-redirect.dependabot.com/spf13/viper/pull/1493">spf13/viper#1493</a></li>
<li>Drop support for Go 1.16 by <a href="https://github.com/sagikazarmark"><code>@​sagikazarmark</code></a> in <a href="https://github-redirect.dependabot.com/spf13/viper/pull/1494">spf13/viper#1494</a></li>
</ul>
<h3>Dependency Updates ⬆️</h3>
<ul>
<li>build(deps): bump github.com/spf13/afero from 1.9.2 to 1.9.3 by <a href="https://github.com/dependabot"><code>@​dependabot</code></a> in <a href="https://github-redirect.dependabot.com/spf13/viper/pull/1465">spf13/viper#1465</a></li>
<li>build(deps): bump github.com/magiconair/properties from 1.8.6 to 1.8.7 by <a href="https://github.com/dependabot"><code>@​dependabot</code></a> in <a href="https://github-redirect.dependabot.com/spf13/viper/pull/1475">spf13/viper#1475</a></li>
<li>build(deps): bump github.com/pelletier/go-toml/v2 from 2.0.5 to 2.0.6 by <a href="https://github.com/dependabot"><code>@​dependabot</code></a> in <a href="https://github-redirect.dependabot.com/spf13/viper/pull/1466">spf13/viper#1466</a></li>
<li>build(deps): bump mheap/github-action-required-labels from 2 to 3 by <a href="https://github.com/dependabot"><code>@​dependabot</code></a> in <a href="https://github-redirect.dependabot.com/spf13/viper/pull/1482">spf13/viper#1482</a></li>
<li>build(deps): bump github.com/subosito/gotenv from 1.4.1 to 1.4.2 by <a href="https://github.com/dependabot"><code>@​dependabot</code></a> in <a href="https://github-redirect.dependabot.com/spf13/viper/pull/1488">spf13/viper#1488</a></li>
<li>build(deps): bump github.com/sagikazarmark/crypt from 0.8.0 to 0.9.0 by <a href="https://github.com/dependabot"><code>@​dependabot</code></a> in <a href="https://github-redirect.dependabot.com/spf13/viper/pull/1490">spf13/viper#1490</a></li>
</ul>
<h2>New Contributors</h2>
<ul>
<li><a href="https://github.com/choar816"><code>@​choar816</code></a> made their first contribution in <a href="https://github-redirect.dependabot.com/spf13/viper/pull/1481">spf13/viper#1481</a></li>
<li><a href="https://github.com/lol768"><code>@​lol768</code></a> made their first contribution in <a href="https://github-redirect.dependabot.com/spf13/viper/pull/1485">spf13/viper#1485</a></li>
<li><a href="https://github.com/mozartz"><code>@​mozartz</code></a> made their first contribution in <a href="https://github-redirect.dependabot.com/spf13/viper/pull/1464">spf13/viper#1464</a></li>
<li><a href="https://github.com/glebik000"><code>@​glebik000</code></a> made their first contribution in <a href="https://github-redirect.dependabot.com/spf13/viper/pull/1467">spf13/viper#1467</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a href="https://github.com/spf13/viper/compare/v1.14.0...v1.15.0">https://github.com/spf13/viper/compare/v1.14.0...v1.15.0</a></p>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="5182412574"><code>5182412</code></a> Add DocBlock to WatchConfig (<a href="https://github-redirect.dependabot.com/spf13/viper/issues/1467">#1467</a>)</li>
<li><a href="0755477020"><code>0755477</code></a> feat: add multiple endpoints support for remote</li>
<li><a href="a7ad8cfcc4"><code>a7ad8cf</code></a> build(deps): bump github.com/sagikazarmark/crypt from 0.8.0 to 0.9.0</li>
<li><a href="63630c2fcd"><code>63630c2</code></a> build(deps): bump github.com/subosito/gotenv from 1.4.1 to 1.4.2</li>
<li><a href="996ab10214"><code>996ab10</code></a> ci: update golangci-lint</li>
<li><a href="982460e8e1"><code>982460e</code></a> feat!: drop Go 1.16 support</li>
<li><a href="2f5cb62e1f"><code>2f5cb62</code></a> ci: cleanup tags</li>
<li><a href="77fc1c3084"><code>77fc1c3</code></a> feat!: drop toml v1 support</li>
<li><a href="e42b933dbe"><code>e42b933</code></a> feat!: drop yaml v2 support</li>
<li><a href="9e46b7652f"><code>9e46b76</code></a> build(deps): bump mheap/github-action-required-labels from 2 to 3</li>
<li>Additional commits viewable in <a href="https://github.com/spf13/viper/compare/v1.14.0...v1.15.0">compare view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=github.com/spf13/viper&package-manager=go_modules&previous-version=1.14.0&new-version=1.15.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

You can trigger a rebase of this PR by commenting `@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


</details>
2023-01-20 17:28:05 +00:00
ashmrtn
6651305d4f
Update OneDrive metadata with folder deletions (#2192)
## Description

Track folder and package deletions in OneDrive metadata for incremental backups. Add test for deletions

## Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No 

## Type of change

- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

## Issue(s)

* #2120

## Test Plan

- [x] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-01-20 17:02:23 +00:00
Danny Adams
f44607ca73 Additonal swap space required for CI compilation. 2023-01-20 11:40:39 -05:00
Danny Adams
898191f9a9 Removal of whitespace. 2023-01-20 11:23:08 -05:00
Danny Adams
c0112d917e Revert back to original ci.yml. 2023-01-20 11:21:35 -05:00
Danny Adams
f425d24364 Update support library to beta. 2023-01-20 11:11:11 -05:00
Danny Adams
585ce939ed Merge branch 'beta-library-switch' into backup-page-flow 2023-01-20 10:39:57 -05:00
Danny Adams
2f125d44a6 Merge branch 'main' into beta-library-switch 2023-01-20 09:40:57 -05:00
Danny Adams
2cd64c4775 Updates to internal/connector/graph_connector_helper_test.go
git status
2023-01-20 09:31:14 -05:00
Danny Adams
b4eae746c4 Update to /pkg/services/m365/m365.go
Change to msgraph beta library.
2023-01-20 09:27:43 -05:00
Danny Adams
9b30a03c31 File added in error. 2023-01-20 09:26:48 -05:00
Danny Adams
77c3b3b7d7 Update /internal/operations package
Beta Library replaces msgraph v1.0
2023-01-20 09:25:44 -05:00
Danny Adams
4009a5c2d4 Update /connector/ package
Beta Library replaces msgraph v1.0
2023-01-20 09:25:03 -05:00
Danny Adams
4687b134ef Update /connector/support package
Beta Library replaces msgraph v1.0
2023-01-20 09:24:11 -05:00
Danny Adams
2929c9f25e Update /connector/sharepoint package
Beta Library replaces msgraph v1.0
2023-01-20 09:23:41 -05:00
Danny Adams
7404c4c8b0 Update /connector/onedrive package
Beta Library replaces msgraph v1.0
2023-01-20 09:22:40 -05:00
Danny Adams
135c07a41a Update /connector/exchange package
Beta Library replaces msgraph v1.0
2023-01-20 09:22:15 -05:00
Danny Adams
3ccdda8ef0 Update /connector/mockconnector package
Change to Beta library.
2023-01-20 09:21:08 -05:00
Danny Adams
4952b81333 Update /connector/graph package
Change to Beta library.
2023-01-20 09:20:31 -05:00
Danny Adams
b82a6349f4 Update /connector/exchange/api package
Change to Beta library.
2023-01-20 09:19:38 -05:00
Danny Adams
519a449b2f Update /connector/discovery package
Change to Beta library.
2023-01-20 09:18:49 -05:00