1100 Commits

Author SHA1 Message Date
Abin Simon
c9a3884289
Add check using staticcheck (#1649)
## Description

Add linting using staticcheck. This is added as a separate check from golangci-lint as we were running into memory issues when running it via golangci-lint.

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [x] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* https://github.com/alcionai/corso/pull/1634

## Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2022-12-02 03:28:11 +00:00
Keepers
b52a3323c0
add shell to onedrive purge (#1666)
## Type of change

- [x] 🐛 Bugfix
2022-12-02 01:09:04 +00:00
Keepers
412b73bf74
purge onedrive folders on gh action (#1655)
## Description

runs the golang purge script for onedrive folders
when running the pwsh purge scripts.  This will
move test-generated folders in onedrive into
a recycle bin.  However, that recycle bin cannot
be purged via api, and must be manually emptied
periodically for each user.

## Type of change

- [x] 💻 CI/Deployment

## Issue(s)

* #1525

## Test Plan

- [x] 💚 E2E
2022-12-01 19:38:50 +00:00
ashmrtn
e9afbaf0ec
Find previous kopia snapshots for given set of tags (#1427)
## Description

Helper functions that allow finding the most recent complete
(and maybe incomplete) snapshots for a given set of tags. The
returned snapshots can later be used to do more efficient kopia
snapshots by allowing kopia to determine it's already uploaded
a file and skip uploading it again.

The number of most recent snapshots to return can be tuned (right
now it returns 1 most recent), but may cause more memory usage
during backups.

Kopia currently has some oddities when getting snapshot
manifests via tags:
* tag values cannot be empty or the comparison returns always
  true, selecting all snapshot manifests
* kopia does not currently tag snapshot manifests made during
  checkpoints. A patch to upstream kopia is needed to fix this.
  All other manifests that match the tags will be returned though

Added code is not currently connected to any backup logic.

Viewing by commit may help as there was a tad of lift'n'shift code
movement for the sake of organization

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

* #1404 

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-12-01 19:07:44 +00:00
dependabot[bot]
2687ecf961
⬆️ Bump go.uber.org/zap from 1.23.0 to 1.24.0 in /src (#1647)
Bumps [go.uber.org/zap](https://github.com/uber-go/zap) from 1.23.0 to 1.24.0.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/uber-go/zap/releases">go.uber.org/zap's releases</a>.</em></p>
<blockquote>
<h2>v1.24.0</h2>
<p>Enhancements:</p>
<ul>
<li><a href="https://github-redirect.dependabot.com/uber-go/zap/issues/1148">#1148</a>[]: Add <code>Level</code> to both <code>Logger</code> and <code>SugaredLogger</code> that reports the
current minimum enabled log level.</li>
<li><a href="https://github-redirect.dependabot.com/uber-go/zap/issues/1185">#1185</a>[]: <code>SugaredLogger</code> turns errors to zap.Error automatically.</li>
</ul>
<p>Thanks to <a href="https://github.com/Abirdcfly"><code>@​Abirdcfly</code></a>, <a href="https://github.com/craigpastro"><code>@​craigpastro</code></a>, <a href="https://github.com/nnnkkk7"><code>@​nnnkkk7</code></a>, and <a href="https://github.com/sashamelentyev"><code>@​sashamelentyev</code></a> for their
contributions to this release.</p>
<p><a href="https://github-redirect.dependabot.com/uber-go/zap/issues/1148">#1148</a>: <a href="https://github-redirect.dependabot.com/uber-go/zap/pull/1148">uber-go/zap#1148</a>
<a href="https://github-redirect.dependabot.com/uber-go/zap/issues/1185">#1185</a>: <a href="https://github-redirect.dependabot.com/uber-go/zap/pull/1185">uber-go/zap#1185</a></p>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a href="https://github.com/uber-go/zap/blob/master/CHANGELOG.md">go.uber.org/zap's changelog</a>.</em></p>
<blockquote>
<h2>1.24.0 (30 Nov 2022)</h2>
<p>Enhancements:</p>
<ul>
<li><a href="https://github-redirect.dependabot.com/uber-go/zap/issues/1148">#1148</a>[]: Add <code>Level</code> to both <code>Logger</code> and <code>SugaredLogger</code> that reports the
current minimum enabled log level.</li>
<li><a href="https://github-redirect.dependabot.com/uber-go/zap/issues/1185">#1185</a>[]: <code>SugaredLogger</code> turns errors to zap.Error automatically.</li>
</ul>
<p>Thanks to <a href="https://github.com/Abirdcfly"><code>@​Abirdcfly</code></a>, <a href="https://github.com/craigpastro"><code>@​craigpastro</code></a>, <a href="https://github.com/nnnkkk7"><code>@​nnnkkk7</code></a>, and <a href="https://github.com/sashamelentyev"><code>@​sashamelentyev</code></a> for their
contributions to this release.</p>
<p><a href="https://github-redirect.dependabot.com/uber-go/zap/issues/1148">#1148</a>: <a href="https://github.coml/uber-go/zap/pull/1148">https://github.coml/uber-go/zap/pull/1148</a>
<a href="https://github-redirect.dependabot.com/uber-go/zap/issues/1185">#1185</a>: <a href="https://github.coml/uber-go/zap/pull/1185">https://github.coml/uber-go/zap/pull/1185</a></p>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="a55bdc32f5"><code>a55bdc3</code></a> Release v1.24.0 (<a href="https://github-redirect.dependabot.com/uber-go/zap/issues/1208">#1208</a>)</li>
<li><a href="ed5598a9e2"><code>ed5598a</code></a> Use ReplaceAll instead of Replace with -1 pos arg (<a href="https://github-redirect.dependabot.com/uber-go/zap/issues/1195">#1195</a>)</li>
<li><a href="da406e3622"><code>da406e3</code></a> Chore: Fix deprecation docs and links (<a href="https://github-redirect.dependabot.com/uber-go/zap/issues/1193">#1193</a>)</li>
<li><a href="9b86a50a3e"><code>9b86a50</code></a> SugaredLogger: Turn error into zap.Error  (<a href="https://github-redirect.dependabot.com/uber-go/zap/issues/1185">#1185</a>)</li>
<li><a href="9137e0e842"><code>9137e0e</code></a> go.mod: Bump go directive to 1.19 (<a href="https://github-redirect.dependabot.com/uber-go/zap/issues/1181">#1181</a>)</li>
<li><a href="7cabba7950"><code>7cabba7</code></a> Open absolute paths as files, limited Windows support (<a href="https://github-redirect.dependabot.com/uber-go/zap/issues/1159">#1159</a>)</li>
<li><a href="7681a0abf6"><code>7681a0a</code></a> Export objectMarshalerPtr (<a href="https://github-redirect.dependabot.com/uber-go/zap/issues/1171">#1171</a>)</li>
<li><a href="9abd14b74c"><code>9abd14b</code></a> chore: remove duplicate word in comments (<a href="https://github-redirect.dependabot.com/uber-go/zap/issues/1169">#1169</a>)</li>
<li><a href="eae3743bc3"><code>eae3743</code></a> Logger, SugaredLogger: Add Level method (<a href="https://github-redirect.dependabot.com/uber-go/zap/issues/1148">#1148</a>)</li>
<li><a href="0d6a75bccf"><code>0d6a75b</code></a> tests: assert.Error/Contains =&gt; assert.ErrorContains (<a href="https://github-redirect.dependabot.com/uber-go/zap/issues/1158">#1158</a>)</li>
<li>See full diff in <a href="https://github.com/uber-go/zap/compare/v1.23.0...v1.24.0">compare view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=go.uber.org/zap&package-manager=go_modules&previous-version=1.23.0&new-version=1.24.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

You can trigger a rebase of this PR by commenting `@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


</details>
2022-12-01 18:32:56 +00:00
dependabot[bot]
3ded147000
⬆️ Bump @iconify/react from 4.0.0 to 4.0.1 in /website (#1653) 2022-12-01 17:51:23 +00:00
dependabot[bot]
015b0874ea
⬆️ Bump minimatch, recursive-readdir and serve-handler in /website (#1654) 2022-12-01 17:35:10 +00:00
Abin Simon
a5ad6e6788
Rework CI to handle combined website and docs (#1568)
## Description

This reworks CI now that we have merged docs and website into a single deployment.

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [x] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* Fixes https://github.com/alcionai/corso/issues/1551

## Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2022-12-01 16:50:04 +00:00
Abin Simon
c58240ea3d
Parallelize OneDrive queries to get URL (#1607)
## Description

This improves the initial backup speed for OneDrive. OneDrive backup was mostly slow when we had a lot of tiny files. Case where we had mostly large files was pretty much the best case scenario and we were throttled by purely how fast we can get the files from MS and how fast kopia can process and upload it. But of small files, we were slowed by the loop which was taking quite a bit of time to fetch the download urls. We have now parallelized the query for getting the download URL. Under best case scenarios, I was able to speed it up to under 20s from ~4-5m starting point. That said, MS graph api still seems to throttle us and when that happen we still go back to around ~2m for worst case scenario. I've added 3 retries as some requests were failing when we continuously making many requests.

This should also take care of the issue of url expiring mentioned in https://github.com/alcionai/corso/issues/581 as we are only prefetching a few urls ahed of time.

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* https://github.com/alcionai/corso/issues/1595
* fixes https://github.com/alcionai/corso/issues/581

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2022-12-01 06:11:11 +00:00
Abin Simon
784db9e12e
Speed up Exchange backup by parallelizing url fetch (#1608)
## Description

From rough numbers we can speedup an account with ~3000 emails, ~1000 contacts and ~1000 events from ~18m to <3m.

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* https://github.com/alcionai/corso/issues/362
* https://github.com/alcionai/corso/issues/1595
* https://github.com/alcionai/corso/pull/1607

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2022-12-01 04:46:24 +00:00
Keepers
58e70166d4
migrate tenant id out of event repoID (#1619)
## Description

Moves the tenantID out of the repoID hash and
into its own value (also hashed) in the event log.

Unique user identification to follow.

## Type of change

- [x] 🌻 Feature

## Issue(s)

* #1602

## Test Plan

- [x] 💚 E2E
2022-12-01 00:03:21 +00:00
ashmrtn
27d0980526
Split backups by (resource owner, service) at CLI layer (#1609)
## Description

Sketch of how to split backups by (resource owner, service) for exchange. Needs more support from selectors package to be complete.

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

* closes #1505

## Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2022-11-30 21:50:49 +00:00
Vaibhav Kamra
c4333ad531
Use latest golangci-lint version (#1634)
## Description

Use latest golangci-lint version. Also updates cache flags.

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [x] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

* #1629 

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2022-11-30 21:17:01 +00:00
ashmrtn
bd373bbfd4
Custom function to send get requests on delta queries (#1643)
## Description

The current version of graph SDK (v0.40.0) Corso uses does not support adding query parameters to delta requests on exchange mail folder messages or contact folder contacts. This reimplements the logic from a newer version of graph SDK so that we can add those parameters. Much of the code is lifted from the updated version of the SDK, with some combining of types that have the same fields.

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

* #1612 

## Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2022-11-30 20:24:31 +00:00
ashmrtn
17a8c87168
Helper to fetch backups by ID (#1635)
## Description

Helper function to fetch backup models by ID. Also renames `Backups` to `BackupsByTag`

## Type of change

- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

* #1505

separated from
* #1609 

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-11-30 16:46:32 +00:00
Abin Simon
467c3c699c
Prevent Test-Suite-Fork from running on main (#1638)
## Description

As of now, it looks like we are running `Test-Suite-Fork` in `main` instead of `Test-Suite-Trusted`. This tweaks the conditions so as to restrict the `Test-Suite-Fork` to just fork PRs.

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [x] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2022-11-30 05:48:13 +00:00
Keepers
2778cd567e
add sharepoint details command (#1618)
## Description

Adds the `backup details sharepoint` command
to the CLI.  E2E support is already functional.
Additional filters will be added as needs are
identified.

## Type of change

- [x] 🌻 Feature

## Issue(s)

* #1614 

## Test Plan

- [x]  Unit test
2022-11-29 23:37:05 +00:00
Keepers
f1ea464ad6
add boilerplate integration tests for sharepoint (#1600)
## Type of change

- [x] 🤖 Test

## Issue(s)

* #1506

## Test Plan

- [x] 💚 E2E
2022-11-29 20:53:42 +00:00
Keepers
ff9a174d52
add Set func to logger package (#1631)
## Description

Add a Set(ctx, logger) func to the logger package
to allow SDK users to provide their own zap
logger instance to corso.

## Type of change

- [x] 🐹 Trivial/Minor

## Issue(s)

* #1630
2022-11-29 20:15:08 +00:00
Keepers
eb67db2fa5
add a discrete clone method to scopes (#1626)
## Description

Adds a method to all scopes that produces a
discrete clone of the original scope.  The result is a shallow clone of the scope, thus retaining all
original values, but is made discrete by replacing the resource owner target with a specific value.

## Type of change

- [x] 🌻 Feature

## Issue(s)

* #1621

## Test Plan

- [x]  Unit test
2022-11-29 18:32:52 +00:00
ashmrtn
1d977105c0
Fix bad comparison due to timezone mismatch (#1627)
## Description

Exchange stores start/end times for events in UTC time. This fixes test failures due to comparing hours extracted from time structs directly. Without the normalization to UTC, the initial time may be different.

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

* closes #1625 

## Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-11-29 17:51:34 +00:00
Niraj Tolia
cc35088be1
Add a thumbnail for the Corso demo video (#1613)
## Description

Fixes a Google search index issues

## Type of change

- [x] 🐛 Bugfix
- [x] 🗺️ Documentation
2022-11-29 01:21:27 +00:00
Danny
94145ecbf6
Path: Category expanded to include SharePoint Lists (#1610)
## Description
Backup of SharePoint requires the extension of `path.CategoryType` to be able to support the creation of `data.Collections` for `SharePoint.Lists`. 
## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature


## Test Plan

- [x]  Unit test
2022-11-28 20:12:57 +00:00
Dhruv Patel
f05d6dec43
fix: integration tests running on PRs from forks (#1504)
## Description

* Allows integration tests to run via `repository_dispatch` events when
PRs are opened from forks

Credit: https://github.com/imjohnbo/ok-to-test

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [X] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic
words" - "closes, fixes" to auto-close the Github issue. -->
* ALC-635

## Test Plan

<!-- How will this be tested prior to merging.-->
- [X] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2022-11-28 14:42:48 -05:00
Danny
d766c30231
GC: Populate Refactor (Contacts) (#1570)
## Description

Iterator Removal from Cache Folder Resolver.
The iterator overhead contributes to the complexities of the GC backup pipeline. Issue 1486's goal is to remove errors that are populated from non-error scenarios (e.g. container no longer exists).

Stage 1: Reduce the number of error clauses in Populate() function
Stage 2: Update the M365ID retrieval process per resolver
PR introduces the `Iterate` approach used in delta functionality to populate the contact folder cache resolver. Reduces the number of calls and variable creation during the populate function. The increase in efficiency will reduce the amount of time concurrent Mailbox queries are being made (application throttling).

## Type of change
- [x] 🐹 Trivial/Minor

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

## Test Plan

- [x]  Unit test
2022-11-28 14:38:03 +00:00
dependabot[bot]
b88d8a61ad
⬆️ Bump github.com/aws/aws-sdk-go from 1.44.144 to 1.44.145 in /src (#1604)
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.44.144 to 1.44.145.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/aws/aws-sdk-go/releases">github.com/aws/aws-sdk-go's releases</a>.</em></p>
<blockquote>
<h1>Release v1.44.145 (2022-11-23)</h1>
<h3>Service Client Updates</h3>
<ul>
<li><code>service/grafana</code>: Updates service API and documentation</li>
<li><code>service/rbin</code>: Updates service API and documentation</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="d9ed7d6c12"><code>d9ed7d6</code></a> Release v1.44.145 (2022-11-23) (<a href="https://github-redirect.dependabot.com/aws/aws-sdk-go/issues/4636">#4636</a>)</li>
<li>See full diff in <a href="https://github.com/aws/aws-sdk-go/compare/v1.44.144...v1.44.145">compare view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=github.com/aws/aws-sdk-go&package-manager=go_modules&previous-version=1.44.144&new-version=1.44.145)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

You can trigger a rebase of this PR by commenting `@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


</details>
2022-11-24 07:28:21 +00:00
Keepers
3f7860fd0f
retrieve all drives in each site (#1599)
## Description

Extends the site drive query to include a selector
for 'system' properties, which causes graph to
return an expanded set of drives including many
which are not normally visible to end users.

## Type of change

- [x] 🌻 Feature

## Issue(s)

* #1506

## Test Plan

- [x] 💚 E2E
2022-11-23 23:42:30 +00:00
Niraj Tolia
5c420892b5
Enable trailing slashes for docs/website (#1603)
## Description

This should fix a bunch of the errors seen by Google when indexing the website.

## Type of change

- [x] 🐛 Bugfix
- [x] 🗺️ Documentation
2022-11-23 23:29:20 +00:00
Niraj Tolia
3c32136100
Prep for docs versioning (#1598)
## Description

- Format config for consistency
- Use relative paths in MD
- Use absolute paths for imports

## Type of change

- [x] 🌻 Feature
- [x] 🗺️ Documentation
2022-11-23 21:07:32 +00:00
Niraj Tolia
d2b5e773e7
Fix README links to the website (#1597)
## Description

These got broken when we combined docs and the website

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🐛 Bugfix
2022-11-23 20:04:51 +00:00
Georgi Matev
108023efd2
Improve Rudderstack event logging (#1593)
## Description

* Fire of Identify event on RepoInit to identify user for analytics tools
* Flatten Payload structure for better mapping to analytics tools
* Snake case convention for event properties

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->

## Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2022-11-23 17:01:39 +00:00
Keepers
22ff7f5d98
errant fixes running sp backup cli (#1589)
## Type of change

- [x] 🐹 Trivial/Minor

## Issue(s)

* #1506

## Test Plan

- [x] 💪 Manual
2022-11-23 16:50:39 +00:00
dependabot[bot]
48e3855847
⬆️ Bump github.com/aws/aws-sdk-go from 1.44.143 to 1.44.144 in /src (#1592)
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.44.143 to 1.44.144.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/aws/aws-sdk-go/releases">github.com/aws/aws-sdk-go's releases</a>.</em></p>
<blockquote>
<h1>Release v1.44.144 (2022-11-22)</h1>
<h3>Service Client Updates</h3>
<ul>
<li><code>service/appflow</code>: Updates service API and documentation</li>
<li><code>service/kinesisanalyticsv2</code>: Updates service API and documentation</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="1356aada48"><code>1356aad</code></a> Release v1.44.144 (2022-11-22) (<a href="https://github-redirect.dependabot.com/aws/aws-sdk-go/issues/4633">#4633</a>)</li>
<li>See full diff in <a href="https://github.com/aws/aws-sdk-go/compare/v1.44.143...v1.44.144">compare view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=github.com/aws/aws-sdk-go&package-manager=go_modules&previous-version=1.44.143&new-version=1.44.144)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

You can trigger a rebase of this PR by commenting `@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


</details>
2022-11-23 12:28:32 +00:00
Niraj Tolia
c3ca63a98d
Rename home page sections to reflect contents (#1572)
## Description

Makes it easier to figure out where to make changes

## Type of change

- [x] 🗺️ Documentation
2022-11-22 22:21:37 +00:00
Keepers
d6522e01ae
filter out personal sites (#1554)
## Description

Adds a server-side filter that removes all
personal (aka: oneDrive) sites from the drive
retrieval.

The approach is not my preferred method.  I'd rather
have used a filter in the query parameters.  In fact, I
have a functioning query in postman:
`$filter=NOT contains(webUrl, 'sharepoint.com/personal/')`
But I can't seem to get the graph client query to build
when I add that to the query params.

## Type of change

- [x] 🌻 Feature

## Issue(s)

* #1506

## Test Plan

- [x] 💚 E2E
2022-11-22 22:17:55 +00:00
Keepers
162713dc54
add flag to retain all progress bars (#1582)
## Description

Adds the --save-progress-bars flag, which
prevents the observe package from removing
completed progress bars.

## Type of change

- [x] 🌻 Feature

## Issue(s)

* #1581

## Test Plan

- [x] 💪 Manual
2022-11-22 22:00:04 +00:00
Niraj Tolia
dea9d4d609
Remove empty page (#1576)
## Description

This is getting picked up by search engines because of the sitemap

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🐛 Bugfix
- [x] 🗺️ Documentation
2022-11-22 21:48:02 +00:00
Niraj Tolia
914bf0cb6f
Fix some font issues in the docs landing page (#1587)
## Description

Reflects former state of the landing page before combining into docs

## Type of change

- [x] 🐛 Bugfix
- [x] 🗺️ Documentation
2022-11-22 21:44:36 +00:00
Keepers
ce4a214761
observe always hides when told (#1578)
## Description

Fixes a bug where not all observer progress bars
were staying hidden if the 'hide' configuration
was flagged.

## Type of change

- [x] 🐛 Bugfix

## Issue(s)

* fixes https://github.com/alcionai/corso/issues/1569

## Test Plan

- [x] 💪 Manual
2022-11-22 20:35:18 +00:00
Keepers
5d13f4e6b8
add sharepoint backup to the cli (#1577)
## Description

Adds boilerplate backup (create, list, delete) cli
commands for sharepoint.

## Type of change

- [x] 🌻 Feature

## Issue(s)

* #1506

## Test Plan

- [x] 💪 Manual
- [x]  Unit test
2022-11-22 19:31:20 +00:00
Keepers
5b0549fb32
migrate sharepoint cat from file to library (#1555)
## Description

Adds a LibraryCategory in paths, and changes
the current sharepoint code to use that cat
instead of the onedrive files cat.  This is mostly to keep sharepoint library drives separate from
onedrive and sharepoint site list items.

## Type of change

- [x] 🌻 Feature

## Issue(s)

* #1506

## Test Plan

- [x]  Unit test
2022-11-22 19:01:05 +00:00
Keepers
10acf0ccf6
generic drive retrieval for sharepoint (#1536)
## Description

Adapts the graph onedrive library to handle
access to drive data across both onedrive and
sharepoint services.

## Type of change

- [x] 🌻 Feature

## Issue(s)

* #1506

## Test Plan

- [x] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2022-11-22 18:05:47 +00:00
Keepers
5f82edb783
fix resource owner display on backup list (#1580)
## Type of change

- [x] 🐛 Bugfix

## Issue(s)

* #1506

## Test Plan

- [x]  Unit test
2022-11-22 16:37:48 +00:00
dependabot[bot]
f3980f6bc9
⬆️ Bump github.com/aws/aws-sdk-go from 1.44.132 to 1.44.143 in /src (#1585)
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.44.132 to 1.44.143.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/aws/aws-sdk-go/releases">github.com/aws/aws-sdk-go's releases</a>.</em></p>
<blockquote>
<h1>Release v1.44.143 (2022-11-21)</h1>
<h3>Service Client Updates</h3>
<ul>
<li><code>service/route53</code>: Updates service API
<ul>
<li>Amazon Route 53 now supports the Asia Pacific (Hyderabad) Region (ap-south-2) for latency records, geoproximity records, and private DNS for Amazon VPCs in that region.</li>
</ul>
</li>
</ul>
<h1>Release v1.44.142 (2022-11-18)</h1>
<h1>Release v1.44.141 (2022-11-18)</h1>
<h3>Service Client Updates</h3>
<ul>
<li><code>service/appflow</code>: Updates service API and documentation</li>
<li><code>service/auditmanager</code>: Updates service API and documentation</li>
<li><code>service/chime-sdk-voice</code>: Adds new service</li>
<li><code>service/cloudfront</code>: Updates service API and documentation
<ul>
<li>CloudFront API support for staging distributions and associated traffic management policies.</li>
</ul>
</li>
<li><code>service/connect</code>: Updates service API and documentation</li>
<li><code>service/dynamodb</code>: Updates service API, documentation, waiters, paginators, and examples
<ul>
<li>Updated minor fixes for DynamoDB documentation.</li>
</ul>
</li>
<li><code>service/ec2</code>: Updates service API and documentation
<ul>
<li>This release adds support for copying an Amazon Machine Image's tags when copying an AMI.</li>
</ul>
</li>
<li><code>service/glue</code>: Updates service API and documentation
<ul>
<li>AWSGlue Crawler - Adding support for Table and Column level Comments with database level datatypes for JDBC based crawler.</li>
</ul>
</li>
<li><code>service/iot-roborunner</code>: Adds new service</li>
<li><code>service/quicksight</code>: Updates service API, documentation, and paginators
<ul>
<li>This release adds the following: 1) Asset management for centralized assets governance 2) QuickSight Q now supports public embedding 3) New Termination protection flag to mitigate accidental deletes 4) Athena data sources now accept a custom IAM role 5) QuickSight supports connectivity to Databricks</li>
</ul>
</li>
<li><code>service/sagemaker</code>: Updates service API and documentation
<ul>
<li>Added DisableProfiler flag as a new field in ProfilerConfig</li>
</ul>
</li>
<li><code>service/servicecatalog</code>: Updates service API and documentation
<ul>
<li>This release 1. adds support for Principal Name Sharing with Service Catalog portfolio sharing. 2. Introduces repo sourced products which are created and managed with existing SC APIs. These products are synced to external repos and auto create new product versions based on changes in the repo.</li>
</ul>
</li>
<li><code>service/ssm-sap</code>: Updates service API, documentation, paginators, and examples</li>
<li><code>service/states</code>: Updates service API and documentation
<ul>
<li>This release adds support for using Step Functions service integrations to invoke any cross-account AWS resource, even if that service doesn't support resource-based policies or cross-account calls. See <a href="https://docs.aws.amazon.com/step-functions/latest/dg/concepts-access-cross-acct-resources.html">https://docs.aws.amazon.com/step-functions/latest/dg/concepts-access-cross-acct-resources.html</a></li>
</ul>
</li>
<li><code>service/streams.dynamodb</code>: Updates service documentation</li>
<li><code>service/transfer</code>: Updates service API and documentation
<ul>
<li>Adds a NONE encryption algorithm type to AS2 connectors, providing support for skipping encryption of the AS2 message body when a HTTPS URL is also specified.</li>
</ul>
</li>
</ul>
<h1>Release v1.44.140 (2022-11-17)</h1>
<h3>Service Client Updates</h3>
<ul>
<li><code>service/amplify</code>: Updates service API and documentation</li>
<li><code>service/appflow</code>: Updates service API and documentation</li>
<li><code>service/appsync</code>: Updates service API and documentation</li>
<li><code>service/dms</code>: Updates service API and documentation
<ul>
<li>Adds support for Internet Protocol Version 6 (IPv6) on DMS Replication Instances</li>
</ul>
</li>
<li><code>service/ec2</code>: Updates service API and documentation</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="e1d2ecb3cc"><code>e1d2ecb</code></a> Release v1.44.143 (2022-11-21) (<a href="https://github-redirect.dependabot.com/aws/aws-sdk-go/issues/4632">#4632</a>)</li>
<li><a href="55b073d012"><code>55b073d</code></a> Release v1.44.142 (2022-11-18) (<a href="https://github-redirect.dependabot.com/aws/aws-sdk-go/issues/4630">#4630</a>)</li>
<li><a href="b18be4c13c"><code>b18be4c</code></a> Ssm sap rename (<a href="https://github-redirect.dependabot.com/aws/aws-sdk-go/issues/4628">#4628</a>)</li>
<li><a href="e979fb59f0"><code>e979fb5</code></a> Release v1.44.141 (2022-11-18) (<a href="https://github-redirect.dependabot.com/aws/aws-sdk-go/issues/4627">#4627</a>)</li>
<li><a href="00f6e7270f"><code>00f6e72</code></a> Release v1.44.140 (2022-11-17) (<a href="https://github-redirect.dependabot.com/aws/aws-sdk-go/issues/4626">#4626</a>)</li>
<li><a href="6232b911c5"><code>6232b91</code></a> Release v1.44.139 (2022-11-16) (<a href="https://github-redirect.dependabot.com/aws/aws-sdk-go/issues/4625">#4625</a>)</li>
<li><a href="e4e9a9808b"><code>e4e9a98</code></a> Release v1.44.138 (2022-11-15) (<a href="https://github-redirect.dependabot.com/aws/aws-sdk-go/issues/4624">#4624</a>)</li>
<li><a href="a11b2070d6"><code>a11b207</code></a> Release v1.44.137 (2022-11-14) (<a href="https://github-redirect.dependabot.com/aws/aws-sdk-go/issues/4622">#4622</a>)</li>
<li><a href="00c5fc72f8"><code>00c5fc7</code></a> Release v1.44.136 (2022-11-11) (<a href="https://github-redirect.dependabot.com/aws/aws-sdk-go/issues/4620">#4620</a>)</li>
<li><a href="2d26930f04"><code>2d26930</code></a> Release v1.44.135 (2022-11-10) (<a href="https://github-redirect.dependabot.com/aws/aws-sdk-go/issues/4619">#4619</a>)</li>
<li>Additional commits viewable in <a href="https://github.com/aws/aws-sdk-go/compare/v1.44.132...v1.44.143">compare view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=github.com/aws/aws-sdk-go&package-manager=go_modules&previous-version=1.44.132&new-version=1.44.143)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

You can trigger a rebase of this PR by commenting `@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


</details>
2022-11-22 08:55:56 +00:00
dependabot[bot]
9ad1b4d481
⬆️ Bump engine.io from 6.2.0 to 6.2.1 in /website (#1586) 2022-11-22 06:51:59 +00:00
dependabot[bot]
8fe82a79ab
⬆️ Bump swiper from 8.4.4 to 8.4.5 in /website (#1584) 2022-11-22 06:48:09 +00:00
Georgi Matev
31db484ab3
Add make serve to serve static build. (#1574)
## Description

Add `make serve` to serve static build.

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->

## Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2022-11-21 19:25:40 +00:00
Georgi Matev
16a18cb3b4
Tell Tailwind to use the Docusorus dark-mode attribute (#1573)
## Description

Tailwind needs to know what is the attribute specifying DarkMode. Default is `class="dark"` which is not what Docusaurus uses. 

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->

## Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2022-11-21 18:57:53 +00:00
Abin Simon
a2860eed10
Print version info to stdout instead of stderr (#1503)
## Description

Previously we were printing the version information with `corso --version` to stderr. This changes it to stdout.

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [x] 🐹 Trivial/Minor

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* fixes https://github.com/alcionai/corso/issues/1361

## Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2022-11-21 17:19:53 +00:00
Niraj Tolia
73b4aa6ff6
More website tweaks (#1563)
## Description

- Add quickstart button to homepage
- Tweak meta for better URL previews
- Add a robots.txt file

## Type of change

- [x] 🗺️ Documentation
2022-11-21 16:20:00 +00:00