This is primarily an exercise in reducing the number of circular imports we get from adding the tester package to other packages.
No logic changes. Purely movement/renaming.
---
#### Does this PR need a docs update or release note?
- [x] ⛔ No
#### Type of change
- [x] 🤖 Supportability/Tests
#### Test Plan
- [x] ⚡ Unit test
- [x] 💚 E2E
<!-- PR description-->
don't check after fetching account type for repo init command
#### Does this PR need a docs update or release note?
- [x] ⛔ No
#### Type of change
<!--- Please check the type of change your PR introduces: --->
- [x] 🐛 Bugfix
#### Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
#### Test Plan
<!-- How will this be tested prior to merging.-->
- [x] ⚡ Unit test
<!-- PR description-->
Add test-cases to fetch config values from config file or flags
#### Does this PR need a docs update or release note?
- [ ] ⛔ No
#### Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🤖 Supportability/Tests
#### Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>
#### Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] ⚡ Unit test
<!-- PR description-->
---
#### Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
#### Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup
#### Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>
#### Test Plan
<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
This reverts commit 9a7213baa6f6858b92c47d5d28b8e39c180e64c0.
See sanity test failure: https://github.com/alcionai/corso/actions/runs/5425405323
---
#### Does this PR need a docs update or release note?
- [x] ⛔ No
#### Type of change
- [x] 🐛 Bugfix
#### Test Plan
- [x] 💚 E2E
<!-- PR description-->
In case of init repo - don't match values from config file.
#### Does this PR need a docs update or release note?
- [ ] ⛔ No
#### Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🐛 Bugfix
#### Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>
#### Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
<!-- PR description-->
Use default values from endpoint only if its not present in config file
#### Does this PR need a docs update or release note?
- [ ] ⛔ No
#### Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🐛 Bugfix
#### Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>
#### Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
<!-- PR description-->
Flags for all configs-
Azure cred flags- (azure-tenant-id, azure-client-id, azure-client-secret) present in -
- Backup (create, delete, details and list) and restore of Exchange, Onedrive and Sharepoint command
- S3 repo init and connect command
AWS cred flags - (aws-access-key, aws-secret-access-key, aws-session-token) present in-
- Backup (create, delete, details and list) and restore of Exchange, Onedrive and Sharepoint command
- S3 repo init and connect command
Passphrase flag- (--passphrase) present in-
- Backup (create, delete, details and list) and restore of Exchange, Onedrive and Sharepoint command
- S3 repo init and connect command
S3 flags-
--endpoint, --prefix, --bucket, --disable-tls, --disable-tls-verification - flags is for repo init and connect commands
all the S3 env var will also work only in case of repo init and connect command. For all other commands user first connects to repo. Which will store the config values in config file. And then user can use that config file for other commands.
No cred configs are save in the config file by Corso.
Config file values added-
Azure cred -
- azure_client_id
- azure_secret
- azure_tenantid
AWS cred -
- aws_access_key_id
- aws_secret_access_key
- aws_session_token
Passphrase -
- passphrase
**NOTE:**
- in case of AWS creds all the three values should be provided from same method. Either put all values in env, config file and so on.
- all the S3 env var will also work only in case of repo init and connect command. For all other commands user first connects to repo. Which will store the config values in config file. And then user can use that config file for other commands.
---
#### Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [x] 🕐 Yes, but in a later PR
- [ ] ⛔ No
#### Type of change
<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
#### Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* https://github.com/alcionai/corso/issues/3522
#### Test Plan
<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
now that exchange api has been folded in with the rest of the m365 api, it doesn't make sense to maintain an options file with only exchange functionality. Since all calls in the file were used 1:1 with some api func, those options have been moved into their respective api funcs.
---
#### Does this PR need a docs update or release note?
- [x] ⛔ No
#### Type of change
- [x] 🧹 Tech Debt/Cleanup
#### Issue(s)
* #1996
#### Test Plan
- [x] 💚 E2E
Move code for connecting to a repo into a common
package so that backup and restore CLI code can
both use it
This will also make it easier for maintenance
code in the future as it can reuse the same
helper
There are no logic changes in this PR, only
code movement
---
#### Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
#### Type of change
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup
#### Issue(s)
tangentially related to
* #3077
#### Test Plan
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
While we were reading flags from the cli, they were not being passed onto the restore path.
---
#### Does this PR need a docs update or release note?
- [x] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ] ⛔ No
#### Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup
#### Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>
#### Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
Mostly find/replace on errors.N and errors.W. Also turns all wrapf into wrap, and removes as many errorf calls as possible.
Might follow up with a linter to enforce this change.
---
#### Does this PR need a docs update or release note?
- [x] ⛔ No
#### Type of change
- [x] 🧹 Tech Debt/Cleanup
#### Issue(s)
* #1970
#### Test Plan
- [x] ⚡ Unit test
- [x] 💚 E2E
Doesn't seem like we're using these
---
#### Does this PR need a docs update or release note?
- [x] ⛔ No
#### Type of change
- [x] 🤖 Test
- [x] 🧹 Tech Debt/Cleanup
#### Issue(s)
* #2373
#### Test Plan
- [x] ⚡ Unit test
- [x] 💚 E2E
In order to retrieve all clues structured error data in tests, we need to extract it from the error using the clues library.
This change appends `clues.ToCore(err)` to all
variations of `assert.NoError(t, err)`. The only
other changes are those necessary to preserve
linting, or to produce an error variable for the
ToCore call.
---
#### Does this PR need a docs update or release note?
- [x] ⛔ No
#### Type of change
- [x] 🤖 Test
- [x] 🧹 Tech Debt/Cleanup
#### Issue(s)
* #1970
#### Test Plan
- [x] ⚡ Unit test
- [x] 💚 E2E
## Description
Trigger an event at start of every CLI command.
## Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
## Issue(s)
* https://github.com/alcionai/corso/issues/1761
## Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
---------
Co-authored-by: Keepers <ryanfkeepers@gmail.com>
Co-authored-by: aviator-app[bot] <48659329+aviator-app[bot]@users.noreply.github.com>
## Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
## Type of change
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Test
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup
## Issue(s)
* #2373
## Test Plan
- [x] 💪 Manual
- [ ] ⚡ Unit test
- [ ] 💚 E2E
## Does this PR need a docs update or release note?
- [x] ⛔ No
## Type of change
- [x] 🧹 Tech Debt/Cleanup
## Issue(s)
* #2329
## Test Plan
- [x] 💪 Manual
## Description
Previously the help output looked like:
```
--config-file string config file location (default is $HOME/.corso.toml) (default "$HOME/.corso.toml")
```
## Does this PR need a docs update or release note?
- [ ] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x] ⛔ No
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [x] 🧹 Tech Debt/Cleanup
## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>
## Test Plan
<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ] ⚡ Unit test
- [ ] 💚 E2E
## Description
Introduces config options (`--disable-tls` and `--disable-tls-verification`) to turn off TLS and TLS verification
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor
## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #1415
## Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
## Description
`path` is not cross platform as it hardcodes / as the separator. Also add a lint rule for this.
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor
## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* fixes https://github.com/alcionai/corso/issues/1236
## Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [ ] ⚡ Unit test
- [ ] 💚 E2E
## Description
migrates the client_id, client_secret, and tenant_id
environment variables to versions prepended with
azure_*.
⚠️ Your local environment will need to change in the following ways: ⚠️
1. prepend `AZURE_` to your clientID, clientSecret, and tenantID env vars.
2. update those same env references in any `.corso.env` files used for copying envs to docker containers.
3. in `.corso.toml` (and any variants such as .corso_test.toml), replace `tenantid` with `azure_tenantid`.
## Type of change
- [x] 🐹 Trivial/Minor
## Issue(s)
* #558
## Test Plan
- [x] 💪 Manual
- [x] ⚡ Unit test
- [x] 💚 E2E
## Description
Repo `--prefix` values should be normalized with a trailing `/`
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor
## Issue(s)
* Fixes#1152
## Test Plan
<!-- How will this be tested prior to merging.-->
- [] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
## Description
Ensure s3 bucket name inputs are normalized
in all uses. Currently, the normalized format is
applied when using storage, but not when passing
around --bucket flag values elsewhere.
## Type of change
- [x] 🐛 Bugfix
## Issue(s)
* #975
## Test Plan
- [x] 💪 Manual
- [x] ⚡ Unit test
## Description
Allow users to provide bucket names prefixed
with the s3 scheme: s3://my-bucket
## Type of change
- [x] 🐹 Trivial/Minor
## Issue(s)
* #560
## Test Plan
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
## Description
Use $HOME as the config default.
Allow CORSO_CONFIG_DIR env to
override the default. Allow the flag
--config-file to override either.
## Type of change
- [x] 🐹 Trivial/Minor
## Issue(s)
* #718
## Test Plan
- [x] 💚 E2E
## Description
Swaps the default corso config location from
$HOME to /app/corso in order to align with docs.
## Type of change
- [x] 🐹 Trivial/Minor
## Issue(s)
* #718
## Test Plan
- [x] 💪 Manual
- [ ] ⚡ Unit test
- [ ] 💚 E2E
swaps the corso go module from github.com/
alcionai/corso to github.com/alcionai/corso/src
to align with the location of the go.mod and
go.sum files inside the repo.
All other changes in the repository update the
package imports to the new module path.
## Description
Instead of initializing static AWS credentials, we rely on the
credential provider chain in [Kopia](https://github.com/kopia/kopia/pull/2213) to discover and initialize credentials.
This currently supports the following in this order:
- Static credentials
- Environment variables (what Corso used to implement)
- IAM
Going forward, this will also allow us to support shared credentials (cred file) once that is
added to the credential provider chain.
## Type of change
Please check the type of change your PR introduces:
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 🐹 Trivial/Minor
## Issue(s)
- #384
## Test Plan
<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual (see test output below)
- [ ] ⚡ Unit test
- [x] 💚 E2E
Introduces the log-level flag, defaulting to info. Also does
a minor refactor of how Print is called for backup results,
which moves the backup/details imports out of the cli/print,
and instead has thoses packages call a Print func.
Adds the base CLI integration tests for backup, list, details,
and restore. Also refactors out the global root command
value in favor of a ctx-bound reference so that tests may
control safely overwriting stdout to scrutinize output.
Adds the basic cli-based backup integration test.
Due to discovering some corner cases about panic
conditions and other error handling in bad runtime
state, updates many other packages for safety.
* Enable line width linter
Set to 120 which should be long enough to not be annoying but keep
things from getting "too long." Adding to get rid of the subjectiveness
of what is "too long." Tabs count as a single character.
Makes the necessary changes, including adding helper
funcs, to bring the CLI up to an integration-testable
state. The changes made in this commit should be
sufficient for most other CLI tests. Includes a single
test as verification.
* Add import order linter
Error messages aren't the most clear/helpful, but they will point you to
the lines that need modifying.
* Adjust max errors reported
* Fix lint errors
The alias of testing to ctesting prevented auto-importing
of the testing library. This change (arbitrarily) renames
the package so that it doesn't collide with the core pkg
for "testing".
* Basic golangci-lint config with gofmt and timeout
Remove corresponding sections from other CI config files.
* Enable comment and string spelling linter
* Fix spelling mistakes for linter
Co-authored-by: Danny <danny@alcion.ai>
This allows the user to specify repo configuration via environment variables when a config file is not available.
This is a temporary fix - ideally we would leverage Viper for this but we currently only use Viper if a config file
is available so that needs to be refactored a bit.
[1]
`TENANT_ID` (already supported)
`BUCKET`
`ENDPOINT`
`PREFIX`
* allow connect to create .corso config file
Current bug: if no .corso config file exists, then repo connect
will always fail, even if it has the correct details to build
a new config file. Solution: allow connect to build a .corso
config file when missing, so long as the operation succeeds
otherwise.
In tandem, return an error whenever a user attempts to
call repo connect with details that do not match the existing
.corso config file.
* separate tenantID from m365 creds
Now that account.Account is in place, tenant id needs
to get removed from the credential set (it isn't actually
a secret) and placed in the account configuration instead.
