From fd221587ec156e15b33f7ddff170f76b84eebecc Mon Sep 17 00:00:00 2001 From: Abhishek Pandey Date: Tue, 19 Sep 2023 17:41:36 +0530 Subject: [PATCH] Create documentation for filesystem repositories (#4287) Website documentation update for local storage. --- #### Does this PR need a docs update or release note? - [x] :white_check_mark: Yes, it's included - [ ] :clock1: Yes, but in a later PR - [ ] :no_entry: No #### Type of change - [ ] :sunflower: Feature - [ ] :bug: Bugfix - [x] :world_map: Documentation - [ ] :robot: Supportability/Tests - [ ] :computer: CI/Deployment - [ ] :broom: Tech Debt/Cleanup #### Issue(s) * https://github.com/alcionai/corso/issues/1416 #### Test Plan - [x] :muscle: Manual - [ ] :zap: Unit test - [ ] :green_heart: E2E --- CHANGELOG.md | 5 ++ website/docs/setup/repos.md | 114 ++++++++++++++++++++++++++++++++---- 2 files changed, 107 insertions(+), 12 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 0fb756904..f81e4e096 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,11 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] (beta) +### Added +- Enables local or network-attached storage for Corso repositories. + +## [v0.13.0] (beta) - 2023-09-18 + ### Added - Groups and Teams service support available as a feature preview! Channel messages and Files are now available for backup and restore in the CLI: `corso backup create groups --group '*'` * The cli commands for "groups" and "teams" can be used interchangably, and will operate on the same backup data. diff --git a/website/docs/setup/repos.md b/website/docs/setup/repos.md index 5615c8c84..bd23d285f 100644 --- a/website/docs/setup/repos.md +++ b/website/docs/setup/repos.md @@ -18,19 +18,19 @@ specified when initializing a repository. Within a repository, Corso uses AES256-GCM-HMAC-SHA256 to encrypt data at rest using keys that are derived from the repository passphrase. -Data in flight to and from the repositiry is encrypted via TLS. +Data in flight to and from the repository is encrypted via TLS. -Repositories are supported on the following object storage systems: +Repositories are supported on the following storage systems:
:::note -Depending on community interest, Corso will add support for other object storage backends in the future. +Depending on community interest, Corso will add support for other storage backends in the future. ::: ## Amazon S3 -### Prerequisites +### S3 Prerequisites Before setting up your Corso S3 repository, the following prerequisites must be met: @@ -73,7 +73,7 @@ The two most commonly-used options are: `AWS_SHARED_CREDENTIALS_FILE`, if not using the default file location. You can learn more about the AWS CLI environment variables [here](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-envvars.html). -### Initialize repository +### Initialize a S3 repository Before first use, you need to initialize a Corso repository with `corso repo init s3`. See the command details [here](../../cli/corso-repo-init-s3). @@ -84,7 +84,7 @@ Before first use, you need to initialize a Corso repository with `corso repo ini ```powershell # Initialize the Corso Repository $Env:CORSO_PASSPHRASE = 'CHANGE-ME-THIS-IS-INSECURE' - .\corso repo init s3 --bucket corso-test + .\corso repo init s3 --bucket corso-repo ``` @@ -93,7 +93,7 @@ Before first use, you need to initialize a Corso repository with `corso repo ini ```bash # Initialize the Corso Repository export CORSO_PASSPHRASE="CHANGE-ME-THIS-IS-INSECURE" - ./corso repo init s3 --bucket corso-test + ./corso repo init s3 --bucket corso-repo ``` @@ -104,13 +104,13 @@ Before first use, you need to initialize a Corso repository with `corso repo ini export CORSO_PASSPHRASE="CHANGE-ME-THIS-IS-INSECURE" docker run --env-file $HOME/.corso/corso.env \\ --volume $HOME/.corso:/app/corso ghcr.io/alcionai/corso:${Version()} \\ - repo init s3 --bucket corso-test` + repo init s3 --bucket corso-repo` } -### Connect to a repository +### Connect to a S3 repository If a repository already exists, you can connect to it with `corso repo connect s3`. See the command details [here](../../cli/corso-repo-connect-s3). @@ -120,7 +120,7 @@ If a repository already exists, you can connect to it with `corso repo connect s ```powershell # Connect to the Corso Repository - .\corso repo connect s3 --bucket corso-test + .\corso repo connect s3 --bucket corso-repo ``` @@ -128,7 +128,7 @@ If a repository already exists, you can connect to it with `corso repo connect s ```bash # Connect to the Corso Repository - ./corso repo connect s3 --bucket corso-test + ./corso repo connect s3 --bucket corso-repo ``` @@ -138,7 +138,7 @@ If a repository already exists, you can connect to it with `corso repo connect s `# Connect to the Corso Repository docker run --env-file $HOME/.corso/corso.env \\ --volume $HOME/.corso:/app/corso ghcr.io/alcionai/corso:${Version()} \\ - repo connect s3 --bucket corso-test` + repo connect s3 --bucket corso-repo` } @@ -159,3 +159,93 @@ need to use the following flag with the initial Corso `repo init` command: Corso also supports the use of object storage systems with no TLS certificate or with self-signed TLS certificates with the `--disable-tls` or `--disable-tls-verification` flags. [These flags](../../cli/corso-repo-init-s3) should only be used for testing. + +## Filesystem Storage + +Corso supports creating a repository on a local or network attached filesystem. + +:::note +Filesystem repositories aren't S3 compatible. Copying a local repository directly to S3-compatible object storage +or the other way around isn't supported. +::: + +### Initialize a filesystem repository + +Before first use, you need to initialize a Corso repository with `corso repo init filesystem`. See the command details +[here](../../cli/corso-repo-init-filesystem). Corso will create the directory structure if necessary, including any +missing parent directories. + + + + + ```powershell + # Initialize the Corso Repository + $Env:CORSO_PASSPHRASE = 'CHANGE-ME-THIS-IS-INSECURE' + .\corso repo init filesystem --path C:\Users\user\corso-repo + ``` + + + + + ```bash + # Initialize the Corso Repository + export CORSO_PASSPHRASE="CHANGE-ME-THIS-IS-INSECURE" + ./corso repo init filesystem --path $HOME/corso-repo + ``` + + + + +{ +`# Initialize the Corso Repository +export CORSO_PASSPHRASE="CHANGE-ME-THIS-IS-INSECURE" +docker run --env-file $HOME/.corso/corso.env \\ + --volume $HOME/.corso:/app/corso ghcr.io/alcionai/corso:${Version()} \\ + --volume /path/on/host/corso-repo:/path/in/container/corso-repo \\ + repo init filesystem --path /path/in/container/corso-repo` +} + + + + +### Connect to a filesystem repository + +If a repository already exists, you can connect to it with `corso repo connect filesystem`. See the command details +[here](../../cli/corso-repo-connect-filesystem). + + + + + ```powershell + # Connect to the Corso Repository + .\corso repo connect filesystem --path C:\Users\user\corso-repo + ``` + + + + + ```bash + # Connect to the Corso Repository + ./corso repo connect filesystem --path $HOME/corso-repo + ``` + + + + +{ +`# Connect to the Corso Repository +docker run --env-file $HOME/.corso/corso.env \\ + --volume $HOME/.corso:/app/corso ghcr.io/alcionai/corso:${Version()} \\ + --volume /path/on/host/corso-repo:/path/in/container/corso-repo \\ + repo connect filesystem --path /path/in/container/corso-repo` +} + + + + +### Remarks + +* Filesystem repositories can be quickly set up and are well-suited for smaller scale projects or +testing environments. However, they don't offer the same level of interoperability as an S3 compatible object storage. +* `--prefix` options aren't supported. +* Repository directories are created with `0700` permission mode. Files are created with `0600`.