782 Commits

Author SHA1 Message Date
Abin Simon
6846ebf964
Skip mdgen build in case of valid mdgen in Makefile (#1214)
## Description

`_validatemdgen` resulted in the build for mdgen to always happen. This fixes that.

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [x] 🗺️ Documentation
- [ ] 🤖 Test
- [x] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

## Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2022-10-20 00:19:41 +00:00
ashmrtn
afa8734268
Fix OneDrive details list flag validation (#1232)
## Description

Call the OneDrive flag validation function to ensure flags that take time strings are formatted properly. Also add tests to ensure there's no future regressions for this

First commit has important code changes. Remainder of commits have test data/setup

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

* closes #1231 
* #913 

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-10-19 23:03:31 +00:00
Keepers
d98ba729b8
Issue 902 fails (#1227)
## Notes

All users in the test set have been ensured to have a
complete set of populated data, relevant to the per-
category-per-user assertions in the test.

## Type of change

- [x] 🐛 Bugfix
- [x] 🤖 Test

## Issue(s)

* #902

## Test Plan

- [x] 💪 Manual
- [x] 💚 E2E
2022-10-19 21:58:48 +00:00
Keepers
4cc60be091
add azure_tenant_id envvar --env entry (#1219)
## Type of change

- [x] 🐛 Bugfix

## Issue(s)

* #1211

## Test Plan

- [x] 💪 Manual
2022-10-19 20:43:26 +00:00
Niraj Tolia
fc218457d7
Clean up "install" section (#1213)
## Description

This is the first of many changes to improve the Corso documentation. This page
will see more edits with the possibility of further consolidaiton.

The issues this and subsequent comments are trying to fix include:

- Complexity
- Too many clicks
- Information spread out across too many pages
- No single-page getting started guide
- Incorrect or stale information

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🗺️ Documentation

## Test Plan

- [x] 💪 Manual
2022-10-19 20:17:48 +00:00
Keepers
2a51cc6975
investigate users not appearing in test assertions (#1188)
## Description

Some minor bugs.  Further assurance that users will appear
in the test results as expected will require manual curation
or addition of data in each user's account.

## Type of change

- [x] 🐛 Bugfix
- [x] 🤖 Test

## Issue(s)

* #902

## Test Plan

- [x] 💪 Manual
- [x] 💚 E2E
2022-10-19 19:39:00 +00:00
ashmrtn
bc316df6d9
Remove flakey test (#1223)
## Description

There's enough other tests in the codebase that check the functionality of getting multiple folders that it should be fine without this one.

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

* closes #1222 

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-10-19 18:31:06 +00:00
Keepers
e57aa124ac
prevent logger flag from overriding --help (#1217)
## Type of change

- [x] 🐛 Bugfix

## Issue(s)

* #763

## Test Plan

- [x] 💪 Manual
2022-10-19 18:02:05 +00:00
ashmrtn
86740f1457
Lint rule for making new contexts in tests (#1199)
## Description
Create a lint rule that warns if `context.TODO` or `context.Background` are called in tests. This helps standardize the use of `tester.NewContext` and ensures log messages are properly flushed during tests.

Output from the linter is similar to (sadly can't get rid of the `\\` in there)
```
pkg/services/m365/m365_test.go:49:22: use of `context.Background` forbidden because "tests should use tester\\.NewContext" (forbidigo) 
        ids, err := UserIDs(context.Background(), acct)
```

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [x] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

* closes #1198 

## Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2022-10-19 17:30:28 +00:00
Niraj Tolia
43a3dcbc84
Minor tweak env help text (#1212)
## Description

Minor tweak env help text

## Type of change

- [x] 🗺️ Documentation
- [x] 🐹 Trivial/Minor
2022-10-19 16:31:32 +00:00
Danny
5cd2583466
GC: Purge: Updates to use graph.ContainerResolver (#1190)
## Description
purge.go to use the ContainerResolvers to collect containers from the Graph Connector.

NOTE: CollectFolders in service_query.go unchanged. `CollectFolderrs()` is stubbed for another PR. This will reduce the amount of line changes to be used in the future. 
<!-- Insert PR description-->

## Type of change

- [x] 🐛 Bugfix

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
*closes  #1189<issue>

## Test Plan

- [x]  Unit test
2022-10-19 14:08:03 +00:00
Danny
5ff890b760
GC: Backup: OneDrive: Call Backup on User without OneDrive Access FIX (#1197)
## Description
Corso should be able to have `Backup` called on a user that does not have OneDrive or does not have OneDrive files without producing an error. This feature branch creates OneDriveIntegration testing to be able to fetch OneDrive data for a particular user. 

Logic changed within `drive.go` 

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [x] 🐛 Bugfix


## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
**closes**  #966

## Test Plan
- [x]  Unit test
2022-10-19 13:41:22 +00:00
Abin Simon
17221f23d0
Drop option to build without Docker in Makefile (#1204)
## Description

Having both option could possibly lead to issues with caching.

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [x] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

## Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2022-10-19 06:52:06 +00:00
Niraj Tolia
64c796ff18
Add git shortref to docs build (#1205)
## Description

Add git version to docs build

This will help us verify what version of docs has been published. This should be replaced by the git tag for production builds.

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature

## Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
2022-10-19 06:15:55 +00:00
Niraj Tolia
7dd3b07db4
Corso docs container should be in the corso namespace (#1208)
## Description

Removes an alcion reference from the docs container

## Type of change

- [x] 🐛 Bugfix

## Test Plan

- [x] 💪 Manual
2022-10-19 06:00:43 +00:00
Abin Simon
5e5ccc1dec
Drop search in docs (#1207)
## Description

It was causing issues with building docs locally via make.
This reverts commit 4be8e825ada0b6e4d0e4372446a0f097c128896f.

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2022-10-19 05:58:25 +00:00
Keepers
5d2bc1eb4e
add event production to the factory script (#1200)
## Description

Adds the capacity to programmatically generate events in the factory script.  Status is currently functional in code completion, but experiencing unexpected results.

## Type of change

- [x] 🤖 Test

## Issue(s)

* #902

## Test Plan

- [x] 💪 Manual
- [x] 💚 E2E
2022-10-18 23:54:37 +00:00
ashmrtn
28c3bbcf7a
Remove (old) unused code for fetching messages (#1168)
## Description

Just code cleanup

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [x] 🐹 Trivial/Minor

## Issue(s)

* #1156

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-10-18 23:09:58 +00:00
ashmrtn
0a91a3e097
Store pointer to data instead of serialized form (#1196)
## Description

Remove JSON [de]serialization from mock model store and just use assignments. Model Store tests check that all fields are properly filled out so using JSON here is not required.

Unblocks #1089 as mock model store no longer relies on JSON serialization (and tags associated with models)

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [x] 🐹 Trivial/Minor

## Issue(s)

* #1085 

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-10-18 20:48:08 +00:00
ashmrtn
4800904b04
Add secondary test user (#1186)
## Description

Wire up code to fetch secondary user from config/env and also provide it to tests.

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [x] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

* closes #1185 

merge after to ensure automatic cleanup:
* #1193 

## Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2022-10-18 18:19:53 +00:00
ashmrtn
b752d1da8d
Some minor test cleanup (#1194)
## Description

* switch up email restore/backup test slightly to cover multiple folders
* remove multi-folder mail backup, now overlaps with the above
* remove stale TODO
* remove unused struct field
* factor out common struct

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Test
- [ ] 💻 CI/Deployment
- [x] 🐹 Trivial/Minor

## Issue(s)

* #913 

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-10-18 17:24:07 +00:00
ashmrtn
3e3c21ab1b
Update purge CI task (#1193)
## Description

Add a call to the purge job for the secondary CI test user. Requires an update to the workflow secrets to populate the secondary user name.

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [x] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

* #1185 

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2022-10-18 17:01:48 +00:00
Abin Simon
a9acbc28f5
Rework build.sh script (#1167)
## Description

While trying to work on https://github.com/alcionai/corso/issues/1166, I ended up reworking parts of the build script ~, but I'm running into some issues with building it on CI. Pushing it here just for reference.~

The new one combines both `build.sh` and `build-container.sh` into a single script where you can specify what to build. Also, inorder setup a proper multi arch, build system locally, we will have to properly setup buildx which is explained in https://docs.docker.com/build/building/multi-platform/ or https://stackoverflow.com/a/70837025/2724649 . I'll add this instructions to docs.

The new build script looks something like this:

```
Usage: build.sh <binary|image> [--platforms ...] [--tag ...]

OPTIONS
 -p|--platforms  Platforms to build for (default: linux/amd64)
                 Specify multiple platforms using ',' (eg: linux/amd64,darwin/arm)
 -t|--tag        Tag for container image (default: alcionai/corso)
```

---
I've made sure the image and binary has the proper architecure and that the amd64 one runs properly in my system. It would be helpful if someone who has access to arm system can validate the arm image. You can use https://github.com/alcionai/corso/pkgs/container/corso/45878348?tag=84fc9d4 image to verify.

```
$ cat check-image.sh
imgid="$(docker create "$1")"
docker cp "$imgid:corso" /tmp/corso

echo Image: "$(docker inspect "$1" | jq '.[0].Architecture')"
echo Binary: "$(file /tmp/corso)"

$ ./check-image.sh ghcr.io/alcionai/corso:84fc9d4@sha256:2278a2b4f108e5dd2ae545f53da1d151b77171f969ffd9718e4bb9886e332ee2
WARNING: The requested image's platform (linux/arm64) does not match the detected host platform (linux/amd64) and no specific platform was requested
Image: "arm64"
Binary: /tmp/corso: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-musl-aarch64.so.1, Go BuildID=7iTBXW0reyfIOS-b-ciS/a5K0Q1IjuA0m9DJxmuNk/Ju1lI6bUZeKn6M_xqon6/KNXwYSnL7e5RVtjAKW9A, not stripped

$ ./check-image.sh ghcr.io/alcionai/corso:84fc9d4@sha256:6320b95470014ca07b9cf1db98b73f5672870c2c53c22c3d13223d88fa621ee0
Image: "amd64"
Binary: /tmp/corso: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-musl-x86_64.so.1, Go BuildID=sBziMHPFI9K-G0et0WjJ/Pe9A2Vy8_xpV3FEDJUMo/p4UeMEzgheASvylZ1N3j/fnwmDeVif4rhneou-S6O, not stripped

$ docker run -it --rm ghcr.io/alcionai/corso:84fc9d4@sha256:6320b95470014ca07b9cf1db98b73f5672870c2c53c22c3d13223d88fa621ee0
[...help message...]
```

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [x] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* https://github.com/alcionai/corso/issues/1166

## Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2022-10-18 05:28:44 +00:00
Abin Simon
4be8e825ad
Add local search in docs (#1165)
## Description

Adds a simple local search in docs based on https://github.com/easyops-cn/docusaurus-search-local . I always found myself looking for a search button and thought it would be a good idea to add it even thought it might not be able to do some fancy searches.
![2022-10-14-11-30-13](https://user-images.githubusercontent.com/14259816/195773364-7448592d-ce6e-47b5-8a84-96ce71b9af95.png)

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [x] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

## Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2022-10-18 04:30:36 +00:00
Danny
e79051318c
GC: Deprecated functions removed. (#1179)
Removal of deprecated functions from code base. 
-  GetRestoreContainer no longer used for restore workflow
- GetContainerID no longer used to find the M365ID of container objects. 
## Description

<!-- Insert PR description-->

## Type of change

<!--- Please check the type of change your PR introduces: --->

- [x] 🐹 Trivial/Minor

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-10-18 01:17:13 +00:00
Keepers
48cb751ee0
lookup contact folder as fallback (#1174)
## Description

In the event that a user only has a primary contact
folder, and no subfolders, the contact folder legacy
iter needs to fall back to checking for the contacts
default folder in an isolated query, because that
folder isn't provided as part of the contacts folders
get request.

## Type of change

- [x] 🐛 Bugfix

## Issue(s)

* #1113

## Test Plan

- [x] 💪 Manual
- [x] 💚 E2E
2022-10-18 00:17:13 +00:00
ashmrtn
0963bbe364
Reenable email tests (#1171)
## Description

Helper code for getting output destination has been merged so it's possible to reenable these tests now.

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

* closes #1169

merge after:
* #1184 

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-10-17 21:36:03 +00:00
ashmrtn
8c495f8866
Fix email recipient comparison (#1184)
## Description

M365 overrides recipient names if the email is known so don't compare that while checking.

Similar updates will likely be needed for other fields like [B]CC in the future.

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

* closes #1183 

## Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2022-10-17 21:13:11 +00:00
Keepers
a32f86e40d
migrate naming of azure env vars (#1176)
## Description

migrates the client_id, client_secret, and tenant_id
environment variables to versions prepended with
azure_*.

⚠️  Your local environment will need to change in the following ways:  ⚠️ 
1. prepend `AZURE_` to your clientID, clientSecret, and tenantID env vars.
2. update those same env references in any `.corso.env` files used for copying envs to docker containers.
3. in `.corso.toml` (and any variants such as .corso_test.toml), replace `tenantid` with `azure_tenantid`.

## Type of change

- [x] 🐹 Trivial/Minor

## Issue(s)

* #558

## Test Plan

- [x] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2022-10-17 20:47:04 +00:00
Eno Thereska
fb46f5c7f3
Export call to get user ids (#1069)
## Description

Exposing user IDs in addition to user emails.

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [X] 🐹 Trivial/Minor

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [X]  Unit test
- [ ] 💚 E2E
2022-10-17 18:17:39 +00:00
ashmrtn
72a3c7ab3b
Trim trailing '/' from CLI input for restore/details (#1175)
## Description

Use paths package to trim unescaped trailing '/' characters from input for Exchange mail and OneDrive folder names. Add tests for Exchange showing that the trimming also works properly if the folder name ends with '/'.

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

* closes #1147 

## Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-10-17 17:54:36 +00:00
Keepers
7d72cd12a4
unskip event restores (#1019)
## Type of change

- [x] 🤖 Test

## Issue(s)

* #1018

## Test Plan

- [x] 💚 E2E
2022-10-14 20:38:33 +00:00
Keepers
f69e27b6fd
add progress bar to onedrive restore (#1170)
## Type of change

- [x] 🐹 Trivial/Minor

## Issue(s)

* #903

## Test Plan

- [x] 💪 Manual
2022-10-14 19:51:58 +00:00
Keepers
d2d5cd7f42
load test all data for all users (#1172)
## Description

Watch the minimum-complete load test over the weekend.
Backup only, all tenant users, all data types, no restrictions.

## Type of change

- [x] 🤖 Test

## Issue(s)

* #902

## Test Plan

- [x] 💚 E2E
2022-10-14 19:11:46 +00:00
Keepers
f737f58a0f
generate emails in factory script (#1162)
## Description

Adds scripted production of mock emails for building out
large/rolling datasets for load testing.

## Type of change

- [x] 🌻 Feature
- [x] 🤖 Test

## Issue(s)

* #902

## Test Plan

- [x] 💪 Manual
2022-10-14 18:48:00 +00:00
Keepers
7facdf1cc3
Issue 902 onedriveindividual (#1140)
## Description

Added an additional set of test suites to load_test
to focus on the backup of a single user who will
have their account prepared with a significant amount
of data (many emails, many events, large files, etc).

## Type of change

- [x] 🤖 Test

## Issue(s)

* #902 

## Test Plan

- [x] 💪 Manual
- [x] 💚 E2E
2022-10-14 17:45:06 +00:00
Keepers
39f9c7f50f
add script boilerplate for mock data generation (#1161)
## Description

Introduces a `factory` script to help devs generate
fake m365 data in bulk for testing purposes.  Only
boilerplate at this time.  Any TODO's or not-yet-
implemented statuses are present as placeholders
alone, without official deadlines.

## Type of change

- [x] 🤖 Test

## Issue(s)

* #902

## Test Plan

- [x] 💪 Manual
2022-10-14 17:19:40 +00:00
Keepers
65d6780906
check for flag population instead of empty string (#1075)
Adds a processor that confirms whether user has
added a value for a flag in the cmd, or if it is the
default value.  This map of valued flags is added
to the service opts structs to for validation.

Also migrates many service flags to utils as
consts so that these values can be maintained
as consistent across packages.
2022-10-14 11:01:27 -06:00
Danny
4a29d22216
GC: Restore: Directory Hierarchy Feature for Exchange (#1053)
## Description
Feature to add the folder hierarchy for folders when restored.
This required an overhaul of the `graph.ContainerResolver` interfaces:
- MailFolderCache
- ContactFolderCache
- ~EventFolderCache (placed in a separate PR)~ https://github.com/alcionai/corso/pull/1101

Restore Pipeline changed to separate the caching / container creation process from the rest of the restore pipeline.


## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature

## Issue(s)

* closes #1046 
* #1004 
* closes #1091
* closes #1098 
* closes #1097 
* closes #1096
* closes #1095
* closes #991
* closes #895
* closes #798 

## Test Plan
- [x]  Unit test
2022-10-14 13:14:33 +00:00
Abin Simon
69a6fd1593
Add corso version using ldflags (#1104)
* Make corso respect --version

* Add proper version information when building

* Use unified version string

$(git describe --exact-match --tags $(git rev-parse HEAD) 2>/dev/null || echo unreleased)-$(git rev-parse --short HEAD)

With tag:    v0.0.0-ceaf04c9
Without tag: unreleased-ceaf04c9

* Add a simple build command to build using Makefile
2022-10-14 12:20:30 +05:30
Abin Simon
2d30f4a999
Set Metrics configuration using ldlfags (#1088)
## Description

This sets up the metrics configs using `ldflags`. Keeping it in draft as I wanted to wait till https://github.com/alcionai/corso/pull/1052 is merged as that will affect how we use it in the CI. I have currently set the base branch to `release-ci` on GH, the branch for https://github.com/alcionai/corso/pull/1052 instead of `main` as the diff would make more sense that way.

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [x] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #1067

## Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2022-10-14 03:37:37 +00:00
Abin Simon
4b94c4f012
Option to not use docker for Makefile in docs/website (#1087)
## Description

This makes it so that we have an option to locally run all Makefile targets without having to go through Docker. That said, it retains Docker as the default way to run it, but now we can set an evn variable `CORSO_USE_DOCKER=-1` to skip running through docker. I understand if this looks a bit hacky and don't want to add it in, but thought I wold propose this anyways. While not major, I was able to get a good amount of decrease in build times.

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [x] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

## Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2022-10-14 03:18:50 +00:00
ashmrtn
29015df8ef
Test helper functions for selector creation (#1163)
## Description

Mostly path manipulations here to allow creating selectors for a backup and paths for the output of a backup given some information about the input. Capable of working for OneDrive and Exchange.

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

* #913 

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-10-14 01:16:29 +00:00
Keepers
42b9b133af
support multi-line progress display (#1138)
## Description

Extends progress bar display with multi-line support.

## Type of change

- [x] 🌻 Feature

## Issue(s)

* #1112

## Test Plan

- [x] 💪 Manual
- [x]  Unit test
2022-10-14 00:46:05 +00:00
Keepers
d4bf3f872f
scrutinize load test results for all users (#1118)
## Description

Adds a check that reviews the load test details.
Details entries to make sure that all users were
involved in the operation, and that for each user,
the test involved all item categories for that app.

## Type of change

- [x] 🤖 Test

## Issue(s)

* #902

## Test Plan

- [x] 💪 Manual
- [x] 💚 E2E
2022-10-14 00:22:38 +00:00
Abin Simon
a6f4f65dba
Update test setup docs and few typo fixes (#1150)
## Description

Found a few misses in the docs regarding test env setup.

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [x] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

## Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2022-10-13 23:12:16 +00:00
Abin Simon
1c5f3d631d
Update builds docs (#1151)
## Description

Adds more info in the build docs for building corso binary locally or using containers. Also improved docs on building corso container image.

@gmatev let me know if I should add anything else specifically to the docs.

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [x] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* fixes #684

## Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2022-10-13 23:07:59 +00:00
Keepers
e5243404a8
add prefix option to scopes (#1141)
## Description

adds extensible options to folder-level scopes that allows the caller to specify whether they want a
prefix-comparison matcher or a contains-comparison matcher.

Also corrects the behavior of the prefix filter so that it accurately follows the "target is prefix of input" specification, rather than the reverse.

## Type of change

- [x] 🌻 Feature

## Issue(s)

* #1133

## Test Plan

- [x]  Unit test
- [x] 💚 E2E
2022-10-13 22:58:18 +00:00
Keepers
5ee4ceb5a6
add --backup flag to list (#1117)
## Description

Allows listing individual backups, instead of always listing the entire set.  Could be expanded to accept multiple backup IDs in the future.

## Type of change

- [x] 🌻 Feature

## Issue(s)

* #1077

## Test Plan

- [x] 💪 Manual
- [x] 💚 E2E
2022-10-13 20:38:35 +00:00
Vaibhav Kamra
1cbb34e403
Scope OneDrive backup - for testing (#1109)
## Description

<!-- Insert PR description-->

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

* closes #1108 
* closes #1137 

## Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-10-13 17:24:42 +00:00