The filters package allows callers to specify both a target
to match on, and behavior of the comparison. While data-
type scopes always equate to "equals", the control over
different comparison behavior is useful for info-type
filters. This change integrates filters into scopes for
built-in control of those comparisons.
Introduces the log-level flag, defaulting to info. Also does
a minor refactor of how Print is called for backup results,
which moves the backup/details imports out of the cli/print,
and instead has thoses packages call a Print func.
Adds the base CLI integration tests for backup, list, details,
and restore. Also refactors out the global root command
value in favor of a ctx-bound reference so that tests may
control safely overwriting stdout to scrutinize output.
Adds the basic cli-based backup integration test.
Due to discovering some corner cases about panic
conditions and other error handling in bad runtime
state, updates many other packages for safety.
* Enable line width linter
Set to 120 which should be long enough to not be annoying but keep
things from getting "too long." Adding to get rid of the subjectiveness
of what is "too long." Tabs count as a single character.
* Turn on revive linter, ignore only a few things
* Fix lint errors
Ignore shadowing of 'suite' in tests for now. Also move some constants
that had the same value to tester.
Makes the necessary changes, including adding helper
funcs, to bring the CLI up to an integration-testable
state. The changes made in this commit should be
sufficient for most other CLI tests. Includes a single
test as verification.
* Add import order linter
Error messages aren't the most clear/helpful, but they will point you to
the lines that need modifying.
* Adjust max errors reported
* Fix lint errors
The alias of testing to ctesting prevented auto-importing
of the testing library. This change (arbitrarily) renames
the package so that it doesn't collide with the core pkg
for "testing".
Cobra's RunE causes every error response to display the
CLI help menu for the current command, after printing the
original error itself. This made error outputs difficult to
read, and was generally unhelpful in most cases. The exit
func in Print now prints the error and closes the CLI without
showing the help menu.
In order for sdk users to build an operation using a repo,
without importing /inernal packages, the operation
options needs to be available via a /pkg package.
* return errors if selectors reduce all entries
During a `backup details` or `restore` command, if the
selectors produce 0 results, return an error rather than
running a no-op process.
* add jsonDebug, MinimumPrintable to interface
We want to optionally print print either debugging
(containing all internal data) or user-friendly json
outputs. This requires a new flag: json-debug, and
an extension to the Print interface: minimumPrintable.
* store backup operation results in the backup manifest
Adds backup operation metadata like the outcome
statistics and selector definitions to the backup manifest
entry. These additional details will appear when users
call `corso backup list`.
* reduce filter selector flags to single strings
Since it doesn't make sense for many of the filter-type
selector flags to receive multiple input values (ex: what
would --received-after date1,date2 result in?), the filter
flags are getting reduced to single-string values.
* add must-pass filters to selectors
Extends the selectors scope set to include Filters. This
allows users to define all-pass matchers (filters),
separate from any-pass matchers (includes), and global
exclusions.
When a caller constructs an Users() exchange scope,
instead of building a single scope that contains all of the
provided users and all of the data beneath, follow the
pattern of the other scopes and generate one scope per
user-category pair.
* [Minor #323] add --success-on-exists flag to make idempotent
Co-authored-by: ryanfkeepers <ryanfkeepers@gmail.com>
Co-authored-by: Keepers <104464746+ryanfkeepers@users.noreply.github.com>
* expand restore flags
Adds support for info filter flags to the restore command.
Also adds (hidden) support for event and contact data
types. Finally, exchnages single-string flag data types for
multi-value string array support.
* filter backup details by flags
`backup details` should have its output filtered by the flags provided by
the user. In addition, the selector's FilterDetails should maintain
information (esp service info) about the entries, rather than slicing them
down to only the path reference.
* Basic golangci-lint config with gofmt and timeout
Remove corresponding sections from other CI config files.
* Enable comment and string spelling linter
* Fix spelling mistakes for linter
Co-authored-by: Danny <danny@alcion.ai>
* introduce exchange info selector support
Adds support in selectors/exchange for queries based on
backup.ExchangeInfo entries. This allows the declaration
of selectors based on non-identifier details such as sender,
subject, or receivedAt time.
Changes Exclude scope matching from being an Any-
match comparator to an All-match. This keeps exclude
and include behavior identical, hopefully making less
confusion for users.
* filter backup details by flags
`backup details` should have its output filtered by the flags provided by
the user. In addition, the selector's FilterDetails should maintain
information (esp service info) about the entries, rather than slicing them
down to only the path reference.
This allows the user to specify repo configuration via environment variables when a config file is not available.
This is a temporary fix - ideally we would leverage Viper for this but we currently only use Viper if a config file
is available so that needs to be refactored a bit.
[1]
`TENANT_ID` (already supported)
`BUCKET`
`ENDPOINT`
`PREFIX`
* add per-data type flags to backup details
Adds the contacts, contact folders, emails, email folders, events,
and users flag support to `corso backup create exchange`
to enable selector support.
* refactor selector scopes to accept slices
Cli flag implementation was showcasing a toil issue: building
selectors required a lot of repetitious code for combining
inputs into sets of scopes. Since all of these productions
were effectively identical (eg: for each user, then each folder,
create a scope with the ids), the cleaner solution is to pack
that behavior into the scope constructors themselves.
* adds the onError option to operations
Adds the OnError option to operations.Options. OnError tells
corso whether to continue despite concurrent processing
errors, or to exit processing on any error. Also includes flag
support for setting the option. Only adds the options, does
not assert error handling behavior in corso.
* adds store package for wrapping model_store
Introduces the pkg/store package, which contains funcs
for wrapping the model_store with common requests.
This package choice was made for its combination
of being in an accessible place, centralizing functionality
and not introducing circular dependencies.
