swaps the corso go module from github.com/
alcionai/corso to github.com/alcionai/corso/src
to align with the location of the go.mod and
go.sum files inside the repo.
All other changes in the repository update the
package imports to the new module path.
## Description
Instead of initializing static AWS credentials, we rely on the
credential provider chain in [Kopia](https://github.com/kopia/kopia/pull/2213) to discover and initialize credentials.
This currently supports the following in this order:
- Static credentials
- Environment variables (what Corso used to implement)
- IAM
Going forward, this will also allow us to support shared credentials (cred file) once that is
added to the credential provider chain.
## Type of change
Please check the type of change your PR introduces:
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 🐹 Trivial/Minor
## Issue(s)
- #384
## Test Plan
<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual (see test output below)
- [ ] ⚡ Unit test
- [x] 💚 E2E
* Fix wsl lint errors in pkg package
* Fix wsl lint errors in most of internal package
Leave some sub-packages out that have higher churn at the moment.
* Basic golangci-lint config with gofmt and timeout
Remove corresponding sections from other CI config files.
* Enable comment and string spelling linter
* Fix spelling mistakes for linter
Co-authored-by: Danny <danny@alcion.ai>
This allows the user to specify repo configuration via environment variables when a config file is not available.
This is a temporary fix - ideally we would leverage Viper for this but we currently only use Viper if a config file
is available so that needs to be refactored a bit.
[1]
`TENANT_ID` (already supported)
`BUCKET`
`ENDPOINT`
`PREFIX`
* separate tenantID from m365 creds
Now that account.Account is in place, tenant id needs
to get removed from the credential set (it isn't actually
a secret) and placed in the account configuration instead.
* move config unions to common code
The configuration union handlers in Storage and Account
overlapped significantly in behavior. Moving those helpers into
a common code folder was requested. Although the behavior
was similar across the files, the types were not, requiring
the addition of generics to solve the need.
* add Account pattern to mimic storage
Introduces /pkg/account, which produces a generic Account
struct that can be configured from specific provider
details (such as m365 credentials), and serialized back to
those details as needed.
Next steps include replacing repository.Account with the
new account. After that, tenantID should get removed from
the m365 credentials handling and placed into the Account
configuration instead.
* use config file for backup create
Now that corso stores its prior connection details in a config
file, that file should get used to retrieve storage details as needed.
credentials requriements surface in many places thorughout corso:
they can be sourced from many locations (envs, files, manually),
and used in many more (cli, repo, kopia). This usage could blossom
into all kinds of duplicate structs sharing similar info. The goal
of this change is to centralize where credentials are declared
and managed, and how they then cascade out to other packages.
* introduce operations (#114)
Adds the /internal/operations package. This is a starting point
which only includes the base operation struct and handling.
Per-process (backup/restore) operations to come at a later time.
* adding comments for clarity in op_progress
* validate required storage props (#85)
Centralizes validation of required storage config properties within
the storage package. Requiremens are checked eagerly at
configuration creation, and lazily at config retrieval.
Additionally, updates /pkg/storage tests to use suites
and assert funcs.
* add validation failure tests to storage
* add common config and encryption passwd
Adds a provider-independent configuration handler, and the
the encryption password config property. The password is used
to encrypt and decrypt the kopia repository properties file.
* fix corso_password in ci.yml
* actually use the corso password in testing
* replace passwd in ci.yml with a secret
* ci.yml secret typo fix
A small collection of changes and code cleanup to successfully run
'corso repo init s3' manually. Adds a default s3 url (might want
this configurable for local testing) and aws session token support.
* introduces storage pkg (#25)
Adds a new package /pkg/storage. Storage is used to
communicate storage provider information throughout the corso
app. In particular, it allows per-provider data to cross interface
boundaries without slicing their details.
* use consts for s3 config keys
