66 Commits

Author SHA1 Message Date
Georgi Matev
010d8d5df1
Final Examples indent fix to be consistent in docs and inline usage (#1056)
## Description

Previous indent fixed usage but screwed up generated docs. 

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)


## Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2022-10-05 20:33:47 +00:00
Georgi Matev
3a1eb1efd2
Add Examples section to CLI usage and docs (#1050)
## Description

Adds Examples to CLI usage output and the auto-gen docs

Also cleans up:
* OneDrive flags and flags help
* Addition of OneDrive generated files to docs 


## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [x] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* Fixes #529  

## Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2022-10-05 07:06:32 +00:00
Keepers
67215c9bf1
field NotFound err in kopia, handle in cli (#1027)
## Description

Replace the `manifest.NotFound` error production
in kopia wrapper with a local version of the error. This allows the cli to catch the corso error and
write a clearer response to the end user.

## Type of change

- [x] 🌻 Feature

## Issue(s)

* #976 

## Test Plan

- [x] 💪 Manual
- [x] 💚 E2E
2022-10-04 18:27:23 +00:00
Georgi Matev
81a259cb32
Clean up CLI flags, add extended usage, enhance docs pages (#1022)
## Description

* Clean up pass on all CLI output
* Added basic extended usage
* Mark required params in generated docs

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [x] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* fixes #527  

## Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2022-10-03 19:35:57 +00:00
Georgi Matev
5e8b67a606
Explicit ordering of flags based on importance and use frequency (#1002)
## Description

Disable automatic flag sorting and explicitly order based on something more sensible. Fixes the order in both inline help output and docs. 

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [x] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* Fixes # 783

## Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2022-09-30 19:01:39 +00:00
Georgi Matev
fe2516b9ba
Fix all Vale [error] level lint issues and all markdownlint issues (#997)
## Description

* fixes all `markdownlint` issues, includeing autogenerated docs
* fixes [error] level Vale problems 
*
## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [x] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [x] 🐹 Trivial/Minor

## Issue(s)

## Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2022-09-30 07:31:58 +00:00
ashmrtn
824f02469c
Backup details list exchange tests (#945)
## Description

Add tests for the `backup list exchange` subcommand. Tests mostly center around indirectly testing how selectors are created and used in this subcommand by checking the output of running the `Reduce` call on a known set of `details.DetailsEntry`s. Also check various error cases

Tests for invalid formats of flag values are disabled as that code does not currently exist. These tests can be enabled when #943 is resolved

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

* #913

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-09-23 16:51:08 +00:00
ashmrtn
744eadfd2f
Create ExchangeOpts struct for selector values (#941)
## Description

Consolidate function parameters by passing them as a single struct instead of a bunch of parameters. Helps keep function signatures shorter and also makes it easier to modify in the future (just change the struct rather than all call sites).

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [x] 🐹 Trivial/Minor

## Issue(s)

* #913 

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-09-22 22:51:25 +00:00
Keepers
76f2c5a632
improve language on selector reduce error (#794)
## Description

shorten and consolidate the error message when
the provided selectors match zero entries in the
backup entity list.

## Type of change

- [x] 🐹 Trivial/Minor

## Issue(s)

* #564

## Test Plan

- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2022-09-22 15:44:08 +00:00
Keepers
1f4a490c33
allow users to opt-out of metric gathering (#871)
## Description

Adds a flag and options control for disabling metrics.

## Type of change

- [x] 🌻 Feature

## Issue(s)

* #741

## Test Plan

- [x] 💪 Manual
2022-09-16 00:56:56 +00:00
ashmrtn
573f55686f
Make ExchangeDataCollection use path.Path internally (#820)
* Have exchange data collection store path.Path

Still complies with the old FullPath() string interface until we update
that.

* Pass path.Path to NewCollection for exchange

Basically fixes up errors introduced by previous commit.

* Fixup exchange recovery path indices

All exchange paths now use the path struct, meaning the service,
category, and user elements are in the standard positions.

* use path package in selector reduction (#822)

Currently, during a reduction process, scopes
compare their values to the raw split on repoRef.
This causes some brittle indexing to retrieve
values from the rr, carrying assumptions that are
difficult to track across changes.  This PR trades
the string split for the paths package to better
integrate identification of the path values.

Adds some mocks and amends some error
behaviors in order to fit paths into the current
testing schema.

Co-authored-by: Keepers <ryanfkeepers@gmail.com>
2022-09-13 13:58:20 -07:00
Keepers
c82dad6e40
flexible restore/details flags (#821)
removes the requriement that higher order data
must be specified in order to specify granular
data ids.  Ex: previously, cli users needed to add
--user and --folder flags in order to use a --mail
flag.  Now the user and folder will default to Any()
if they're not provided.
2022-09-12 18:13:50 -06:00
Keepers
25ce11b2c6
deduplicate shared exchange code in cli (#732) 2022-09-09 18:30:15 -06:00
Keepers
6f7cf00188
add cli integration tests for event/contact (#760)
## Description

Adds foundational cli integration tests for backup/restore
of events and calendars

## Type of change

- [x] 🤖 Test

## Issue(s)

#501 

## Test Plan

- [ ] 💪 Manual
- [ ]  Unit test
- [x] 💚 E2E
2022-09-09 14:15:36 +00:00
Keepers
6cdc691c6f
correct the corso module (#749)
swaps the corso go module from github.com/
alcionai/corso to github.com/alcionai/corso/src
to align with the location of the go.mod and
go.sum files inside the repo.

All other changes in the repository update the
package imports to the new module path.
2022-09-07 15:50:54 -06:00
Keepers
af9dfe6654
expose event info flags in cli (#727)
## Description

Adds the new event filter selectors to the CLI for
backup detail and restore commands.

## Type of change

Please check the type of change your PR introduces:
- [x] 🌻 Feature

## Issue(s)

#501

## Test Plan

- [x] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-09-02 16:25:58 +00:00
Keepers
a043304e0b
add eventOrganizer selector (#720)
## Description
Adds the eventOrganizer exchange filter, wraps
up the addition of eventRecurs filtering, and fixes
a couple improperly designed filter constructors.

## Type of change

Please check the type of change your PR introduces:
- [x] 🌻 Feature

## Issue(s)
#501

## Test Plan

- [x] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-09-01 19:56:17 +00:00
Keepers
c29d93e655
adds calendars as a folder structure to events (#714)
## Description

Exchange events were amended to use calendars
as a folder structure.  This updates the selector
to treat events as having folders similar to mail and
contacts.

## Type of change

Please check the type of change your PR introduces:
- [x] 🌻 Feature

## Issue(s)

#501

## Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-09-01 15:39:14 +00:00
Keepers
f0edf7b081
add the event-calendar flag to the cli (#712)
## Description

Adds the event-calendar flag to the cli to prepare
for selectors including the calendar as the events'
"folder" structure.

## Type of change

Please check the type of change your PR introduces:
- [x] 🌻 Feature

## Issue(s)

#501 

## Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-08-31 23:18:15 +00:00
Keepers
3f6f6604a2
remove extraneous debug logs from cli (#675) 2022-08-29 20:48:23 +00:00
ashmrtn
88a318a7c0
Fix wsl errors in cmd and cli packages (#651) 2022-08-26 09:13:39 -07:00
Keepers
f1370b36e4
add cli commands to delete backups (#641) 2022-08-24 18:26:14 +00:00
Danny
eca16e4a69
CLI: Contact and Event Flags active (#633)
Contact and Event flags restored
2022-08-23 12:29:19 +00:00
Keepers
3ac05acf64
introduce log-level control (#589)
Introduces the log-level flag, defaulting to info.  Also does
a minor refactor of how Print is called for backup results,
which moves the backup/details imports out of the cli/print,
and instead has thoses packages call a Print func.
2022-08-18 18:30:15 +00:00
Keepers
a1b9f876ae
add final basic cli tests (#578)
Adds the base CLI integration tests for backup, list, details,
and restore.  Also refactors out the global root command
value in favor of a ctx-bound reference so that tests may
control safely overwriting stdout to scrutinize output.
2022-08-17 18:13:55 +00:00
ashmrtn
e76860fd80
Set line length to 120 characters (#506)
* Enable line width linter

Set to 120 which should be long enough to not be annoying but keep
things from getting "too long." Adding to get rid of the subjectiveness
of what is "too long." Tabs count as a single character.
2022-08-12 16:05:46 +00:00
Keepers
5070296e18
add connect cli integration tests (#493) 2022-08-05 12:05:45 -07:00
ashmrtn
9e2e88f5f3
Enable some linters in revive (#486)
* Turn on revive linter, ignore only a few things

* Fix lint errors

Ignore shadowing of 'suite' in tests for now. Also move some constants
that had the same value to tester.
2022-08-04 14:42:51 -07:00
Keepers
342dd2e9f9
set up CLI for integration testing (#478)
Makes the necessary changes, including adding helper
funcs, to bring the CLI up to an integration-testable
state.  The changes made in this commit should be
sufficient for most other CLI tests.  Includes a single
test as verification.
2022-08-04 17:15:13 +00:00
Keepers
88a8e992fd
use StringSlice pflag func to split flag inputs (#468) 2022-08-03 16:32:49 -06:00
Keepers
a785cb5eb4
dedupe cli/print import (#464) 2022-08-02 18:40:49 +00:00
Keepers
b9171d01dc
remove help menu output from non-cli errors (#438)
Cobra's RunE causes every error response to display the
CLI help menu for the current command, after printing the
original error itself.  This made error outputs difficult to
read, and was generally unhelpful in most cases.  The exit
func in Print now prints the error and closes the CLI without
showing the help menu.
2022-07-29 19:47:08 +00:00
Keepers
7e60ec5073
move operation.Options into /pkg/control (#452)
In order for sdk users to build an operation using a repo,
without importing /inernal packages, the operation
options needs to be available via a /pkg package.
2022-07-29 12:52:19 -06:00
Keepers
9ea0409125
print backup details at the end of backup create (#439) 2022-07-28 17:53:49 -06:00
Keepers
4862d5b800
return errors if selectors reduce all entries (#427)
* return errors if selectors reduce all entries

During a `backup details` or `restore` command, if the
selectors produce 0 results, return an error rather than
running a no-op process.
2022-07-28 11:03:30 -06:00
Keepers
1929b2307f
reduce filter selector flags to single strings (#393)
* reduce filter selector flags to single strings

Since it doesn't make sense for many of the filter-type
selector flags to receive multiple input values (ex: what
would --received-after date1,date2 result in?), the filter
flags are getting reduced to single-string values.
2022-07-26 10:31:34 -06:00
Keepers
350d27dcb4
Issue-271-mustfilter: add must-pass filters to selectors (#392)
* add must-pass filters to selectors

Extends the selectors scope set to include Filters.  This
allows users to define all-pass matchers (filters),
separate from any-pass matchers (includes), and global
exclusions.
2022-07-25 22:41:49 +00:00
Keepers
9e7755cfa7
expand restore flags (#382)
* expand restore flags

Adds support for info filter flags to the restore command.
Also adds (hidden) support for event and contact data
types.  Finally, exchnages single-string flag data types for
multi-value string array support.
2022-07-25 09:43:12 -06:00
Keepers
0739ea7e09
add info-based selector flags to backup details (#380)
* filter backup details by flags

`backup details` should have its output filtered by the flags provided by
the user.  In addition, the selector's FilterDetails should maintain
information (esp service info) about the entries, rather than slicing them
down to only the path reference.
2022-07-22 13:23:16 -06:00
ashmrtn
395c7c8525
Add spell check linter and run gofmt in golangci-lint (#396)
* Basic golangci-lint config with gofmt and timeout

Remove corresponding sections from other CI config files.

* Enable comment and string spelling linter

* Fix spelling mistakes for linter

Co-authored-by: Danny <danny@alcion.ai>
2022-07-22 18:28:25 +00:00
Keepers
532922f662
introduce exchange info selector support (#379)
* introduce exchange info selector support

Adds support in selectors/exchange for queries based on
backup.ExchangeInfo entries.  This allows the declaration
of selectors based on non-identifier details such as sender,
subject, or receivedAt time.

Changes Exclude scope matching from being an Any-
match comparator to an All-match.  This keeps exclude
and include behavior identical, hopefully making less
confusion for users.
2022-07-21 15:08:03 -06:00
Keepers
6224a92e7a
filter backup details by flags (#371)
* filter backup details by flags

`backup details` should have its output filtered by the flags provided by
the user.  In addition, the selector's FilterDetails should maintain
information (esp service info) about the entries, rather than slicing them
down to only the path reference.
2022-07-20 16:25:28 -06:00
Keepers
37f2c44dc7
accept backupID flag, not backupDetailsID (#370) 2022-07-20 00:22:25 +00:00
Keepers
43c024f4b2
add per-data type flags to backup details (#355)
* add per-data type flags to backup details

Adds the contacts, contact folders, emails, email folders, events,
and users flag support to `corso backup create exchange`
to enable selector support.
2022-07-19 16:38:48 -06:00
Keepers
746c88a233
refactor selector scopes to accept slices (#352)
* refactor selector scopes to accept slices

Cli flag implementation was showcasing a toil issue: building
selectors required a lot of repetitious code for combining
inputs into sets of scopes.  Since all of these productions
were effectively identical (eg: for each user, then each folder,
create a scope with the ids), the cleaner solution is to pack
that behavior into the scope constructors themselves.
2022-07-19 16:04:07 -06:00
Keepers
625cf744a6
adds the onError option to operations (#347)
* adds the onError option to operations

Adds the OnError option to operations.Options.  OnError tells
corso whether to continue despite concurrent processing
errors, or to exit processing on any error.  Also includes flag
support for setting the option.  Only adds the options, does
not assert error handling behavior in corso.
2022-07-19 14:20:12 -06:00
Keepers
6c22d5c0ce
adds store package for wrapping model_store (#346)
* adds store package for wrapping model_store

Introduces the pkg/store package, which contains funcs
for wrapping the model_store with common requests.
This package choice was made for its combination
of being in an accessible place, centralizing functionality
and not introducing circular dependencies.
2022-07-19 11:54:53 -06:00
Keepers
efaa2da1bb
add --all and --data to backup create (#342)
* add --all and --data to backup create

Adds flags for backing up all exchange data, and
for isolating the data in the backup by data type.
Introduces validation and selector creation in
backup create.  Switches the --user flag variable
type from a string to a []string.
2022-07-15 10:27:20 -06:00
Keepers
39c85e1a84
add output formatting control to cli (#329)
* add output formatting control to cli

Adds the capacity for the CLI to output either a
text table or a json blob to the terminal.  Table is
the default behavior, json is toggled with the --json
flag.
2022-07-13 14:15:08 -06:00
Vaibhav Kamra
efe4319080
Fix for issues caught by go-lint (#327)
Error list visible [here](https://github.com/alcionai/corso/actions/runs/2653381666)
2022-07-12 11:19:32 -06:00