* add backup to operations
Extend the operations set with a backupOperation. New backup
operations are created through an existing repository and, once
created, can be Run synchronously.
Integration tests have a skeleton, but the meat of them is on hold
while dependencies undergo implementation.
* remove operation progress
* adds env-based integration test controls
Some integration tests were running locally without being told
to do so via env variables. This adds some controllers for validating
if and when those tests should run, and for ensuring the env
variables necessary to run the test are available.
* testing helper comments, func name change
* add testing timeOf log, skip gc ci tests
* corrected the test skip in GC_test
* update kopia_test with helpers
* rename /testing files to *_test.go
This will ensure these files do not compile into the release binary.
* Revert "rename /testing files to *_test.go"
This reverts commit 04fd2046cc6d11bf3e3eb98d98a3e84b1bba6366.
POD struct that holds some simple information about a kopia upload. Not
wrapping or returning a handle to a kopia struct to keep upper layers of
corso clear of type dependencies. Struct will help with tests as it can
be returned from public functions in the KopiaWrapper package allowing
more black-box testing. Especially useful since we don't yet have a
restore flow, which means we can't test data == restore(backup(data)).
This commit adds ExchangeData which represents a single item in an ExchangeDataCollection.
Also adds a placeholder for a data collection populator that will be implemented later
* introduce operations (#114)
Adds the /internal/operations package. This is a starting point
which only includes the base operation struct and handling.
Per-process (backup/restore) operations to come at a later time.
* adding comments for clarity in op_progress
This commit does the following:
- Renames DataStream to DataCollection since that describes better the interface
- Introduces DataStream as a streamable item within this collection
* Add function to open kopia and keep handle
* Add kopia handle in Repository
* Have Repository manage lifetime of KopiaWrapper
* Have CLI manage lifetime of Repository
* Issue #68: Updates to dependency files.
* Issue #68: Added GraphConnector module and test suite.
* Issue #68: ErrorList module and test added to package to support GraphConnector.
* Issue #68: Updates made to address comments.
* Issue #68: Added additonal comments to structs and corrected library changes.
* Issue #68: Added additonal comments to structs and corrected library changes.
* Issue #68: Updates made to package.
* Update errorList.go
Fix typo
* Issue #68: ErrorList struct removal required
The structure of the errorList is not required for the first
initialization of the GraphConnector. In the case that this structure or
addtional functionality needs to be added in the future, an issue will
be created.
* Issue #68: Simplification and structure change
The errorList has been changed to []errors. Additionally, the
credentials and authentication have been changed to inline. Provided
early exits for creating NewGraphConnectors. Note, adapter objects can
be created with strings that fail authentication. Reduced the amount of logs.
* Issue #68: Prepend
[errors](https://github.com/alcionai/corso/pull/104#discussion_r887268576)
This fixes the bug that identified in the PR.
* Update src/internal/connector/graphConnector.go
Adjustment for styling.
Co-authored-by: Keepers <104464746+ryanfkeepers@users.noreply.github.com>
* Update src/internal/connector/graphConnector_test.go
Issue #68: PR Resolution
Test environmental value changed to be consistent with the rest of the package.
Co-authored-by: Keepers <104464746+ryanfkeepers@users.noreply.github.com>
* Issue #68: Resolving Open Questions from reviewers
Prior to commit, Issue #116. Additionally, the entire error package was
requested to be removed from the component. GraphConnector tests are
preliminary set-up to investigate the Issue #116. The code will be
redacted after the new functionality is verified. Stashing for merge for
the changes from the reviewers.
* Issue #116: Bug fix. Response verified not to be nil prior to using it in next step.
* Issue #116: Added table structure to verify GraphConnector can handle being given an empty set of parameters.
* Issue #68: PR Changes
List of changes:
- Added check to ensure conversion of user object passed
- setTenant not exported
- TODO added user query during initialization
- Iteration flow adjusted for paging.
Co-authored-by: Keepers <104464746+ryanfkeepers@users.noreply.github.com>
* validate required storage props (#85)
Centralizes validation of required storage config properties within
the storage package. Requiremens are checked eagerly at
configuration creation, and lazily at config retrieval.
Additionally, updates /pkg/storage tests to use suites
and assert funcs.
* add validation failure tests to storage
* add common config and encryption passwd
Adds a provider-independent configuration handler, and the
the encryption password config property. The password is used
to encrypt and decrypt the kopia repository properties file.
* fix corso_password in ci.yml
* actually use the corso password in testing
* replace passwd in ci.yml with a secret
* ci.yml secret typo fix
A small collection of changes and code cleanup to successfully run
'corso repo init s3' manually. Adds a default s3 url (might want
this configurable for local testing) and aws session token support.
* add e2e wiring of cli to kopia
Now that pkg/storage and internal/kopia are in place, we
can wire up the init flow from the cli all the way to kopia. Testing
harness for this functionality still needs investigation afterward.
* factor out awsVars struct for s3Cfg
* adds kopia pkg to handle integration (#25)
internal/kopia will be used to abstract all kopia integration in a
central location.
* defer blob.Storage closure in kopia
blob.Storage objects must get closed at the end of their usage.
This currently isn't getting called.
* introduces storage pkg (#25)
Adds a new package /pkg/storage. Storage is used to
communicate storage provider information throughout the corso
app. In particular, it allows per-provider data to cross interface
boundaries without slicing their details.
* use consts for s3 config keys
Adds the s3 provider set to the repo cli commands. Flag, env
vars, and other configuration is still incomplete and will require
iteration. But this sets us up for quick testing of the commands.
* adds docs/CLI.md
Introduces the cli command documentation.
* extending cli.md with design discussion
* migrate cli.md into docusaurus docs (#11)
Move the CLI.md file into docusaurus so that it appears
as part of the documentation page.
* remove cli version, settings.json (#11)
Remove /.vscode/settings.json and the versioning info from
/docs/cli.md.
* introduce repository api scaffold with s3 (#6)
introduces the public repository package in /pkg, with a
skeleton implementation for generating the s3 provider config
and boilerplate for where that usage gets centralized.
* condense init/connect to single action
A double-pass NewRepo() && Connect() allows Repository structs
to exist independent guarantees that their usage will be successful.
This hides the struct generation within the Connect and Initialize
funcs so that the construction and guarantees can occur as
a single process.
Mac uses low group IDs for the default user, which would cause groupadd
to fail as the group already existed. Only make a new group if it
doesn't already exist.
This does raise other questions though as now we are technically having
the container run with a "services" group.
