394 Commits

Author SHA1 Message Date
ashmrtn
12f61dd71c
Switch to using path categories in graph connector (#734)
Will help make later PRs easier as the category will already be known.

## Description

<!-- Insert PR description-->

## Type of change

Please check the type of change your PR introduces:
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [x] 🐹 Trivial/Minor

## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
on path to:
* #456 

## Test Plan

<!-- How will this be tested prior to merging.-->

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-09-02 20:12:22 +00:00
Vaibhav Kamra
b1cd472483
Remove AWS credentials initialization (#610)
## Description

Instead of initializing static AWS credentials, we rely on the 
credential provider chain in [Kopia](https://github.com/kopia/kopia/pull/2213) to discover and initialize credentials.

This currently supports the following in this order:
- Static credentials
- Environment variables (what Corso used to implement)
- IAM

Going forward, this will also allow us to support shared credentials (cred file) once that is
added to the credential provider chain.

## Type of change

Please check the type of change your PR introduces:
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 🐹 Trivial/Minor

## Issue(s)
- #384 

## Test Plan

<!-- How will this be tested prior to merging.-->

- [x] 💪 Manual (see test output below)
- [ ]  Unit test
- [x] 💚 E2E
2022-09-02 18:38:01 +00:00
Vaibhav Kamra
ed1a4bebce
Use a WaitGroup for AwaitStatus (#716)
## Description

This builds on the `MergeStatus` proposal and `WaitGroup` discussion proposed in #494 
Required for OneDrive where we are operating on multiple collections

## Type of change

Please check the type of change your PR introduces:
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 🐹 Trivial/Minor

## Issue(s)
#494 

## Test Plan

<!-- How will this be tested prior to merging.-->

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-09-02 17:17:17 +00:00
ashmrtn
21c2e4af14
Make exchange mail path ctor a bit easier to use (#735)
## Description

Turn into single constructor with bool instead of separate item and
folder constructors.

## Type of change

Please check the type of change your PR introduces:
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [x] 🐹 Trivial/Minor

## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->

#671 

## Test Plan

<!-- How will this be tested prior to merging.-->

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-09-02 16:52:00 +00:00
Keepers
af9dfe6654
expose event info flags in cli (#727)
## Description

Adds the new event filter selectors to the CLI for
backup detail and restore commands.

## Type of change

Please check the type of change your PR introduces:
- [x] 🌻 Feature

## Issue(s)

#501

## Test Plan

- [x] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-09-02 16:25:58 +00:00
Niraj Tolia
568d8e2a0c
Add Corso logos (#736)
## Description

Add Corso logos

## Type of change

Please check the type of change your PR introduces:
- [x] 🌻 Feature
2022-09-02 15:56:58 +00:00
Niraj Tolia
67440951b7
Add a website README (#737)
Also nuke a CloudFront ID that isn't applicable to the Corso website

## Description

Adds a README for building the Corso website

## Type of change

Please check the type of change your PR introduces:
- [x] 🗺️ Documentation
2022-09-02 15:25:18 +00:00
Niraj Tolia
14cf0a13b2
Delete unused JS files (#740)
## Description

Delete unused JS files
2022-09-02 15:02:10 +00:00
Niraj Tolia
a7068566b2
Nuke non-optimal website pages and images (#729)
## Description

This cleanup leaves 5 potential templates to pick from. Further cleanup
will be done once we settle on a page format.

## Type of change

Please check the type of change your PR introduces:
- [x] 🐛 Bugfix

## Test Plan

- [x] 💪 Manual
2022-09-02 01:30:39 +00:00
ashmrtn
e325e36c04
Expand available path categories (#733)
## Description

Add events and contacts. Still cannot create paths of these types
though.

## Type of change

Please check the type of change your PR introduces:
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [x] 🐹 Trivial/Minor

## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->

#671 

## Test Plan

<!-- How will this be tested prior to merging.-->

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-09-02 00:55:53 +00:00
ashmrtn
61ce920972
Cleanup parameters to functions to appease linters (#730)
## Description

Group like parameters to graph functions so SonarCloud is happier

## Type of change

Please check the type of change your PR introduces:
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [x] 🐹 Trivial/Minor

## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
on the path to:
* #456 

## Test Plan

<!-- How will this be tested prior to merging.-->

- [ ] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2022-09-02 00:26:31 +00:00
Niraj Tolia
5e0fe9561c
Corso website template + build framework (#723)
## Description

This PR imports a Tailwind-based website template and adds a
build hardness for it.

Please check the type of change your PR introduces:
- [x] 🌻 Feature

## Test Plan

- [x] 💪 Manual
2022-09-02 00:03:10 +00:00
ashmrtn
ae28971be3
Thread contexts through some GC functions (#728)
## Description

Will be used later and just generally useful if we ever add tracing or
such to these functions.

## Type of change

Please check the type of change your PR introduces:
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [x] 🐹 Trivial/Minor

## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->

#456 

## Test Plan

<!-- How will this be tested prior to merging.-->

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-09-01 23:31:43 +00:00
ashmrtn
4a96f2571d
Second attempt at directory cache/resolver for a user (#724)
## Description

Directory cache capable of converting a folder ID to a complete path to the folder

## Type of change

Please check the type of change your PR introduces:
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 🐹 Trivial/Minor

## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->

#456 

## Test Plan

<!-- How will this be tested prior to merging.-->

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-09-01 23:03:35 +00:00
Keepers
784f006da5
don't split json output by type (#725)
## Description

Prevents output printing of detail entries from splitting into
multiple json arrays when the entries contains multiple item
type.s

## Type of change

Please check the type of change your PR introduces:
- [x] 🐛 Bugfix

## Issue(s)

#501 

## Test Plan

- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2022-09-01 22:06:35 +00:00
Keepers
da6288cb8b
minor value fixes in mail consts/types (#722)
- [x] 🐹 Trivial/Minor
- [x] 🐛 Bug

closes #726
2022-09-01 21:34:57 +00:00
Keepers
a043304e0b
add eventOrganizer selector (#720)
## Description
Adds the eventOrganizer exchange filter, wraps
up the addition of eventRecurs filtering, and fixes
a couple improperly designed filter constructors.

## Type of change

Please check the type of change your PR introduces:
- [x] 🌻 Feature

## Issue(s)
#501

## Test Plan

- [x] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-09-01 19:56:17 +00:00
Georgi Matev
137123294a
Auto-generate CLI docs in docs dev tooling (#721)
## Description

Auto-generate CLI docs in docs dev tooling. 
* Add new Makefile target to rebuild the `mdgen` binary, only if needed. This helps with slow in container builds. Relies on host caching of both modules and the actual binary. This approach is probably not suitable for the actual docs CI build. 
* Actual generation is fast and done every time when using relevant docs dev tooling commands. 

NOTE: .gitignore may need to be adjusted if we switch to creating the CLI docs as part of build and tooling as opposed to the current automated PR with changes. 

## Type of change

Please check the type of change your PR introduces:
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [x] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 🐹 Trivial/Minor

## Issue(s)

## Test Plan

<!-- How will this be tested prior to merging.-->

- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2022-09-01 18:50:09 +00:00
Keepers
a9e66b0db4
Issue 570 deploy sep b (#719)
Recovery action after losing this commit upon merging `issue-570-deploy`.
2022-09-01 16:40:36 +00:00
Danny
4398a67a5a
Bug fix: Addresses (#717)
## Description
When a mock event is created the event is sent to personnel. Requires that personnel have valid email addresses.
Addresses created and updated in the mockconnector package. 
## Type of change

Please check the type of change your PR introduces:
- [x] 🐛 Bugfix


## Test Plan

- [x] 💪 Manual
No emails populate when `TestRestoreEvents` completes
2022-09-01 16:11:08 +00:00
Keepers
c29d93e655
adds calendars as a folder structure to events (#714)
## Description

Exchange events were amended to use calendars
as a folder structure.  This updates the selector
to treat events as having folders similar to mail and
contacts.

## Type of change

Please check the type of change your PR introduces:
- [x] 🌻 Feature

## Issue(s)

#501

## Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-09-01 15:39:14 +00:00
Keepers
127b6d061a
introduce docker container production (#660)
Introduces the production of docker containers as a CI step.
Currently only provides a rolling-release version that builds
on every push to main.  Images are deployed to ghcr.io.

The PR includes two variations on building the images.  We'll
likely only want to stick with one or the other.
2022-09-01 15:06:57 +00:00
github-actions[bot]
7d5e72925a
Docs Auto-Generation (#618)
Automated changes by [create-pull-request](https://github.com/peter-evans/create-pull-request) GitHub action
2022-09-01 00:24:20 +00:00
Danny
7a25a7b70e
GC: Restore Event to M365 user account feature (#711)
## Description
logic and orchestration to take byte representation of M365 event, create a Corso calendar, and place the event and calendar into the M365 user's account. 
## Type of change

Please check the type of change your PR introduces:
- [x] 🌻 Feature

## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
closes #599 
closes #710
Requires PR #708 to ensure test folder is where events are created. 
## Test Plan

<!-- How will this be tested prior to merging.-->

- [x]  Unit test
2022-09-01 00:02:36 +00:00
Georgi Matev
3d9aa326e8
Missing env command in docs sidebar (#715)
## Description

Adds `corso env` auto-gen command docs to docs sidebar. 

## Type of change

Please check the type of change your PR introduces:
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 🐹 Trivial/Minor

## Issue(s)

## Test Plan

<!-- How will this be tested prior to merging.-->

- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2022-08-31 23:40:29 +00:00
Keepers
f0edf7b081
add the event-calendar flag to the cli (#712)
## Description

Adds the event-calendar flag to the cli to prepare
for selectors including the calendar as the events'
"folder" structure.

## Type of change

Please check the type of change your PR introduces:
- [x] 🌻 Feature

## Issue(s)

#501 

## Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-08-31 23:18:15 +00:00
Georgi Matev
dcbfed15df
Prepend Docusorus front matter to auto-gen CLI commands (#713)
* Pre-pend dark to auto-gen CLI commands

* After `gofmt -s`

* Update src/cmd/mdgen/mdgen.go

Co-authored-by: Keepers <ryanfkeepers@gmail.com>

* More gofmt

Co-authored-by: Keepers <ryanfkeepers@gmail.com>
2022-08-31 22:56:05 +00:00
Danny
8ead744e9f
GC: Restore: Create / Delete Calendar Feature (#708)
Feature creation: Create/Delete Calendar for aM365 user added to main branch
2022-08-31 22:03:54 +00:00
Keepers
5c59522fc2
print separate tables per entry type (#709)
## Description

Since a backup can encapsulate multiple data
types (ex: mail, contacts, and events), it doesn't
make sense to print one table with all disjoint
values.  This change splits up the print output
so that each data type in the details gets its
own table.

## Type of change

Please check the type of change your PR introduces:
- [x] 🌻 Feature

## Issue(s)

#501 

## Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-08-31 21:10:10 +00:00
Keepers
6de326a2c4
add event selector info filters (#703)
## Description

Adds filtering support for exchnage events in
the exchange selector.

## Type of change

Please check the type of change your PR introduces:
- [x] 🌻 Feature

## Issue(s)

#501

## Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-08-31 20:44:03 +00:00
ashmrtn
9fe6ca7f56
Logic to create Path structs from strings (#691)
## Description

* logic to split escaped path strings into elements
* wire up validity checks/struct construction
* tests for both invalid and valid path strings

## Type of change

Please check the type of change your PR introduces:
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 🐹 Trivial/Minor

## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->

part of #671 
merge after:
* #648
* #689 
* #690 

## Test Plan

<!-- How will this be tested prior to merging.-->

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-08-31 20:21:46 +00:00
Niraj Tolia
aa3cc21cdb
Docs: npm upgrades (#707)
* Docs: Fix mermaid install

* Docs: Upgrade clsx and prism-react-renderer
2022-08-31 19:57:57 +00:00
Keepers
cfda6ff643
add contactName filter in ExchangeSelector (#697) 2022-08-31 19:31:53 +00:00
Vaibhav Kamra
8634b1b1ac
Helpers to create OneDrive Collections (#692)
## Description

This adds the logic that materializes collections for a specified user

The collection paths are currently derived from OneDrive metadata but a future
PR will introduce `paths` pkg support to create these.

## Type of change

Please check the type of change your PR introduces:
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 🐹 Trivial/Minor

## Issue(s)

#388 

## Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-08-31 18:39:18 +00:00
Danny
305de77188
Iterate-Test Suite Move (#701)
Test suite move of iterator functions from `exchange-service-test.go` to `iterators_test.go`
2022-08-31 17:55:36 +00:00
Danny
b5a5378113
GC: Restore: Retrieve CalendarID Feature (#687)
Feature added to be able to retrieve containerID for `exchange.Event` types.
2022-08-31 17:28:57 +00:00
Keepers
7a8c6eaf78
clean up/simplify filter packagee (#695) 2022-08-31 16:43:54 +00:00
Danny
068f0560e4
Issue #501: Event support (#704)
Added EventRecurs as option for backup details.
2022-08-31 10:04:13 -06:00
Niraj Tolia
245bb6e73e
Misc doc changes (#667)
* Bounce package.json verison to 0.1.0

* Docs cleanup driven by Vale

* Clarifying language

* Clarify that permissions bit is important

* End of line white space linter warnings

* Straighten what seemed like awkward constructs in prerequisites

* Sync up with `docs/intro.md`

Co-authored-by: Georgi Matev <gmatev@alcion.ai>
2022-08-31 00:23:42 +00:00
Brent Checketts
d927ba1dda
[Snyk] Security upgrade mdx-mermaid from 1.2.2 to 1.3.0 (#698)
fix: docs/package.json & docs/package-lock.json to reduce vulnerabilities

The following vulnerabilities are fixed with an upgrade:
- https://snyk.io/vuln/SNYK-JS-MDXMERMAID-3009151

Co-authored-by: snyk-bot <snyk-bot@snyk.io>
2022-08-30 23:57:00 +00:00
ashmrtn
34697efbeb
Logic to validate service/category pairs for paths (#689)
* Logic to validate service/category pairs for paths

* convert string to service or category const
* check that a given service/category pair is a valid resource type

* Add basic tests for validateServiceAndCategory
2022-08-30 23:23:55 +00:00
ashmrtn
d1bf2b90a6
Re-add logic for getting Exchange mail paths (#648)
* Re-add logic for getting Exchange mail paths

Paths can either be for an item (email) or a folder. Returned struct
provides safe access to underlying information in the path once it is
created.

* Refine tests, add struct comment

Add test to make sure Item() and Folder() work properly for a resource
that has no parent folder, just the prefix stuff.

* Rework resource-specific paths based on comments

* use a single type to represent resource specific paths
* use service/category enums to denote the resource the path represents
* update tests to check for service/category of the path
* rename exchange_path*.go -> resource_path*.go
2022-08-30 22:45:47 +00:00
Danny
67bc038c55
GC: Add WSL Linting Formatting src/internal/connector (#694)
Lint remaining of `internal/connector` package for wsl
2022-08-30 22:12:38 +00:00
Keepers
6f04321a60
replace scope values with filters (#674)
The filters package allows callers to specify both a target
to match on, and behavior of the comparison.  While data-
type scopes always equate to "equals", the control over
different comparison behavior is useful for info-type
filters.  This change integrates filters into scopes for
built-in control of those comparisons.
2022-08-30 21:29:24 +00:00
ashmrtn
e3abc281d6
Enable wsl for most of internal/connector package (#680)
* Lint part of internal/connector package with wsl

* Cleanup some wsl lint errors in connector package
2022-08-30 13:13:00 -07:00
ashmrtn
ee57c7cb6c
Ignore multiple trailing '/' in path elements (#690)
Multiple unescaped trailing path separators should be trimmed from input elements so there are no empty elements or trailing unescaped separators at the end of the path or between elements
2022-08-30 19:30:28 +00:00
Danny
0b7fe0fdb9
GC: Small GC Refactor of Restore workflow (#682)
Removal of 'unused' if-clause and category variable being updated.
2022-08-30 18:50:31 +00:00
Danny
0b3f345727
GC: Restore Contact Interface (#642)
Functionality added to restore `exchange.Contact` to M365 back store. Unit tests added to support proper testing of new feature.
2022-08-30 17:46:47 +00:00
ashmrtn
ae06b36e7a
Enable wsl linter for most of codebase (#679)
Turn on now to keep from cycle of fixing most code, waiting for some PRs
in other packages, and then needing to do more fixups. Will slowly
enable on various files in connector package until we get full coverage.
2022-08-29 23:14:11 +00:00
ashmrtn
c3e89db0df
Lint doc for wsl (#678)
Some tips and examples on coexisting with the wsl linter
2022-08-29 22:46:57 +00:00