46 Commits

Author SHA1 Message Date
Georgi Matev
81a259cb32
Clean up CLI flags, add extended usage, enhance docs pages (#1022)
## Description

* Clean up pass on all CLI output
* Added basic extended usage
* Mark required params in generated docs

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [x] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* fixes #527  

## Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2022-10-03 19:35:57 +00:00
Georgi Matev
5e8b67a606
Explicit ordering of flags based on importance and use frequency (#1002)
## Description

Disable automatic flag sorting and explicitly order based on something more sensible. Fixes the order in both inline help output and docs. 

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [x] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* Fixes # 783

## Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2022-09-30 19:01:39 +00:00
Keepers
1f4a490c33
allow users to opt-out of metric gathering (#871)
## Description

Adds a flag and options control for disabling metrics.

## Type of change

- [x] 🌻 Feature

## Issue(s)

* #741

## Test Plan

- [x] 💪 Manual
2022-09-16 00:56:56 +00:00
Keepers
6cdc691c6f
correct the corso module (#749)
swaps the corso go module from github.com/
alcionai/corso to github.com/alcionai/corso/src
to align with the location of the go.mod and
go.sum files inside the repo.

All other changes in the repository update the
package imports to the new module path.
2022-09-07 15:50:54 -06:00
Vaibhav Kamra
b1cd472483
Remove AWS credentials initialization (#610)
## Description

Instead of initializing static AWS credentials, we rely on the 
credential provider chain in [Kopia](https://github.com/kopia/kopia/pull/2213) to discover and initialize credentials.

This currently supports the following in this order:
- Static credentials
- Environment variables (what Corso used to implement)
- IAM

Going forward, this will also allow us to support shared credentials (cred file) once that is
added to the credential provider chain.

## Type of change

Please check the type of change your PR introduces:
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 🐹 Trivial/Minor

## Issue(s)
- #384 

## Test Plan

<!-- How will this be tested prior to merging.-->

- [x] 💪 Manual (see test output below)
- [ ]  Unit test
- [x] 💚 E2E
2022-09-02 18:38:01 +00:00
Keepers
3f6f6604a2
remove extraneous debug logs from cli (#675) 2022-08-29 20:48:23 +00:00
ashmrtn
88a318a7c0
Fix wsl errors in cmd and cli packages (#651) 2022-08-26 09:13:39 -07:00
Keepers
a1b9f876ae
add final basic cli tests (#578)
Adds the base CLI integration tests for backup, list, details,
and restore.  Also refactors out the global root command
value in favor of a ctx-bound reference so that tests may
control safely overwriting stdout to scrutinize output.
2022-08-17 18:13:55 +00:00
Keepers
192c69c68f
add backup cli integration test (#517)
Adds the basic cli-based backup integration test.
Due to discovering some corner cases about panic
conditions and other error handling in bad runtime
state, updates many other packages for safety.
2022-08-12 20:04:44 +00:00
Vaibhav Kamra
88bcd053e5
Simplify test utilities (#532) 2022-08-12 02:49:22 +00:00
Keepers
0cb11e0cc5
clean up env var tester func for multiple slices (#508) 2022-08-12 00:08:00 +00:00
ashmrtn
da66cc4c2f
Enable more rigorous version of gofmt linting (#488)
* Enable a stricter linter

* Fix new lint errors
2022-08-05 13:33:20 -07:00
Keepers
5070296e18
add connect cli integration tests (#493) 2022-08-05 12:05:45 -07:00
Keepers
9326690338
autopopulate cli markdown on ci build (#453) 2022-08-04 16:46:14 -06:00
Keepers
342dd2e9f9
set up CLI for integration testing (#478)
Makes the necessary changes, including adding helper
funcs, to bring the CLI up to an integration-testable
state.  The changes made in this commit should be
sufficient for most other CLI tests.  Includes a single
test as verification.
2022-08-04 17:15:13 +00:00
Keepers
d7fe518007
rename /internal/testing to /tester (#463)
The alias of testing to ctesting prevented auto-importing
of the testing library.  This change (arbitrarily) renames
the package so that it doesn't collide with the core pkg
for "testing".
2022-08-02 12:21:14 -06:00
Keepers
b9171d01dc
remove help menu output from non-cli errors (#438)
Cobra's RunE causes every error response to display the
CLI help menu for the current command, after printing the
original error itself.  This made error outputs difficult to
read, and was generally unhelpful in most cases.  The exit
func in Print now prints the error and closes the CLI without
showing the help menu.
2022-07-29 19:47:08 +00:00
Sidhartha Mani
6a9ccf2a51
add --succeed-on-exists flag to make s3 repo init idempotent (#324)
* [Minor #323] add --success-on-exists flag to make  idempotent

Co-authored-by: ryanfkeepers <ryanfkeepers@gmail.com>
Co-authored-by: Keepers <104464746+ryanfkeepers@users.noreply.github.com>
2022-07-25 18:42:24 +00:00
Vaibhav Kamra
c1c9cfe118
Address issues flagged by golangci-lint (#341) 2022-07-13 22:21:04 +00:00
Vaibhav Kamra
efe4319080
Fix for issues caught by go-lint (#327)
Error list visible [here](https://github.com/alcionai/corso/actions/runs/2653381666)
2022-07-12 11:19:32 -06:00
Keepers
2aeafa36bd
add unit tests for cli command addition (#260)
* add unit tests for cli command addition
2022-07-01 08:37:50 -06:00
Keepers
8c399a6dc1
allow connect to create .corso config file (#256)
* allow connect to create .corso config file

Current bug: if no .corso config file exists, then repo connect
will always fail, even if it has the correct details to build
a new config file.  Solution: allow connect to build a .corso
config file when missing, so long as the operation succeeds
otherwise.

In tandem, return an error whenever a user attempts to
call repo connect with details that do not match the existing
.corso config file.
2022-06-30 13:03:26 -06:00
Keepers
ce52cc5f41
Add the backup list command to the cli (#254)
* Add the backup list command to the cli

Adds `corso backup list <service>` to the cli commands.
Currently does nothing, while we're waiting on
downstream implementation.  e2e hookup will arrive later.
2022-06-30 09:30:37 -06:00
Keepers
108065df75
remove repository.Account (#229)
* remove repository.Account

Now that account.Account exists, we can replace the
deprecated repository.Account struct with the account
version.
2022-06-23 16:02:12 -06:00
Keepers
243343c0e9
Replace fmt prints with the logger (#220)
Now that we have the logger in place, we should use it
in place of fmt.Print calls for development and debugging output.
2022-06-17 13:45:22 -06:00
Keepers
fcdb42bc89
add restore and restore exchange cli (#210)
Adds the `corso restore` command to the cli (which only displays
a help menu).  And adds the `corso restore exchange`
command (which also only displays a help menu) to the cli.
Further functionality (like the complete flags) will come later.
2022-06-15 15:46:09 -06:00
Keepers
340d875579
use config file for backup create (#171)
* use config file for backup create

Now that corso stores its prior connection details in a config
file, that file should get used to retrieve storage details as needed.
2022-06-09 14:28:26 -06:00
Keepers
f3866b7283
add pkg/log with zap log library (#155)
Adds a logger pkg using the zap log library.  Generally straight-
forward, though we'll have to watch out for gotchas with the
flush requirements.
2022-06-09 13:17:01 -06:00
Vaibhav Kamra
5dcb2b7579
Unit tests for config file helpers (#160)
Adds unit tests for the config file read/write helpers.

It also uncovered a bug in how viper handles writing a config file when a config file path is set
directly. This required a workaround in our init logic when we are using a custom config file name.

This commit does the following:
- Refactors the init logic into a InitConfig helper
- Adds a unit test to validate basic ReadRepoConfig behavior
- Adds a unit test that uses WriteReadConfig to write config and ReadRepoConfig to read it
2022-06-08 11:13:25 -07:00
Vaibhav Kamra
ebae1ed037
Utility methods to write and read local config file (#149)
Adds rudimentary support for writing/reading a local config file.
2022-06-07 17:25:14 -07:00
Keepers
21bf1d8b39
make consts camelCase (#153) 2022-06-07 11:44:14 -06:00
Keepers
b0fe422035
add pkg/credentials (#147)
credentials requriements surface in many places thorughout corso:
they can be sourced from many locations (envs, files, manually),
and used in many more (cli, repo, kopia).  This usage could blossom
into all kinds of duplicate structs sharing similar info.  The goal
of this change is to centralize where credentials are declared
and managed, and how they then cascade out to other packages.
2022-06-07 10:59:56 -06:00
Keepers
3be698eab9
return errors in cli handlers (#146)
* return errors in cli handlers

* return the wrapped errors
2022-06-07 09:15:00 -06:00
Keepers
d5d92e707d
add create and exchange backup commands (#145)
Adds the create subcommand for backups.  Adds the exchange
subcommand for backup creation.  Centralizes some
common code on top.
2022-06-06 16:33:04 -06:00
Keepers
8a9c1acc9e
add cli backup command (#142)
* add cli backup command

adds the `corso backup` command to the cli.  Currently
only displays the backup help.

* add backup to cli cmds, fix verbiage
2022-06-06 14:08:34 -06:00
ashmrtn
f68a4e3f46
Fix pointer receivers for KopiaWrapper handles (#131)
* Fix pointer receivers

* Some basic tests about closing KopiaWrapper

* Return pointers to structs
2022-06-03 16:15:51 -07:00
ashmrtn
f1550c967b
Add function to open kopia (#111)
* Add function to open kopia and keep handle

* Add kopia handle in Repository

* Have Repository manage lifetime of KopiaWrapper

* Have CLI manage lifetime of Repository
2022-06-03 10:34:28 -07:00
Keepers
cc3306e5e0
validate required storage props (#85) (#108)
* validate required storage props (#85)

Centralizes validation of required storage config properties within
the storage package.  Requiremens are checked eagerly at
configuration creation, and lazily at config retrieval.

Additionally, updates /pkg/storage tests to use suites
and assert funcs.

* add validation failure tests to storage
2022-06-02 15:33:19 -06:00
Keepers
535cb9e1f5
add common config and encryption passwd (#74) (#99)
* add common config and encryption passwd

Adds a provider-independent configuration handler, and the
the encryption password config property.  The password is used
to encrypt and decrypt the kopia repository properties file.

* fix corso_password in ci.yml

* actually use the corso password in testing

* replace passwd in ci.yml with a secret

* ci.yml secret typo fix
2022-06-01 16:00:41 -06:00
Keepers
596a4cc37b
initial repo init integration test (#90) (#96)
Implements an integration test in the repository that runs an end-
to-end test of an s3 repo initialization through kopia.
2022-05-31 18:32:26 -06:00
Keepers
eaf6018996
add prefix flag for s3 (#94)
* add prefix flag for s3

Adds the flag --prefix for s3 repo init and creation.

* fix missing prefix flag in repo s3 cli
2022-05-27 17:22:50 -06:00
Keepers
d923c4072b
expose s3 endpoint as flag (#87)
Exposes the s3 endpoint option as a cli flag and storage config
property.  Uses the kopia default s3.amazonaws.com as a fallback.
2022-05-27 09:11:26 -06:00
Keepers
e6663fd0f1
changes requried for manual repo init (#25) (#80)
A small collection of changes and code cleanup to successfully run
'corso repo init s3' manually.  Adds a default s3 url (might want
this configurable for local testing) and aws session token support.
2022-05-25 17:27:23 -06:00
Keepers
548eb1be72
add e2e wiring of cli to kopia (#77)
* add e2e wiring of cli to kopia

Now that pkg/storage and internal/kopia are in place, we
can wire up the init flow from the cli all the way to kopia.  Testing
harness for this functionality still needs investigation afterward.

* factor out awsVars struct for s3Cfg
2022-05-25 14:06:05 -06:00
Keepers
f2d5c878a3
adds s3 repo commands to the cli (#67)
Adds the s3 provider set to the repo cli commands.  Flag, env
vars, and other configuration is still incomplete and will require
iteration.  But this sets us up for quick testing of the commands.
2022-05-20 15:20:15 -06:00
Keepers
82c9a1f182
Add 'repo' cli command set (#31)
Adds the 'repo' command and its primary subcommands to the
Corso cli command list.
2022-05-09 16:15:07 -06:00