79 Commits

Author SHA1 Message Date
Keepers
2c913f9ef7
connector data coll refactor (#1507)
A quick code movement before adding the
sharepoint datacollection production so that
we minimize graph_conn file bloat.
2022-11-17 14:04:16 -07:00
Keepers
2d66dcbd7a
replace golang purge script with powershell script (#1407)
## Description

Three primary changes:
1. move the list of load test users into a secret in the github test domain.
2. purge all data that was generated for users in the load test
3. fix all `m356` typos.

## Type of change

- [x] 🤖 Test
- [x] 💻 CI/Deployment

## Issue(s)

* #1266

## Test Plan

- [x] 💪 Manual
- [x] 💚 E2E
2022-11-04 15:41:36 +00:00
Keepers
1bc3210c97
add rolling data production to load test (#1339)
## Description

Adds production of 10 exchange items of each type, only for the load_test user, every time the load tests succeed.  Also migrates that user ID out to a secret value stored in github.

## Type of change

- [x] 🤖 Test

## Issue(s)

* #902

## Test Plan

- [x] 💚 E2E
2022-10-26 21:56:32 +00:00
Vaibhav Kamra
6047805d6c
Switch test user accounts (#1291)
## Description

Switches the test accounts used in CI since the previously used account was hitting slowness due to
items in Deleted Folders.

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [x] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #1191 

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [ ]  Unit test
- [x] 💚 E2E
2022-10-21 21:33:10 +00:00
Abin Simon
1cad17bfc7
Use filepath instead of path (#1247)
## Description

`path` is not cross platform as it hardcodes / as the separator. Also add a lint rule for this.

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* fixes https://github.com/alcionai/corso/issues/1236

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2022-10-20 22:57:30 +00:00
ashmrtn
4800904b04
Add secondary test user (#1186)
## Description

Wire up code to fetch secondary user from config/env and also provide it to tests.

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [x] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

* closes #1185 

merge after to ensure automatic cleanup:
* #1193 

## Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2022-10-18 18:19:53 +00:00
Keepers
a32f86e40d
migrate naming of azure env vars (#1176)
## Description

migrates the client_id, client_secret, and tenant_id
environment variables to versions prepended with
azure_*.

⚠️  Your local environment will need to change in the following ways:  ⚠️ 
1. prepend `AZURE_` to your clientID, clientSecret, and tenantID env vars.
2. update those same env references in any `.corso.env` files used for copying envs to docker containers.
3. in `.corso.toml` (and any variants such as .corso_test.toml), replace `tenantid` with `azure_tenantid`.

## Type of change

- [x] 🐹 Trivial/Minor

## Issue(s)

* #558

## Test Plan

- [x] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2022-10-17 20:47:04 +00:00
Keepers
0b65b20200
attach logger to test ctx, use flusher (#1083)
## Description

Replace all cases of context.Background() and
context.TODO() with the testing ctx, which seeds
a logging instance into the context.  The seed is
less important than the flush action, since errant log.Ctx(ctx) calls will automatically use the log
singleton.  But in order for the logs to show up
within the test, they need to get flushed.

See my comments for focus on non-chaff changes.

## Type of change

- [x] 🤖 Test

## Issue(s)

* #902

## Test Plan

- [x] 💪 Manual
- [x]  Unit test
2022-10-10 21:14:10 +00:00
ashmrtn
dd7710ca5b
OneDrive get and delete folder functions (#1093)
## Description

Useful helper functions for the purge script

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

* #1090 

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-10-08 01:34:15 +00:00
Keepers
89668ed164
describe time value standards, validate time flags (#1066)
## Description

Adds validation of cli inputs for time-based flags.
All time values that can be parsed by the time
handling in common count as supported, even
though official support includes a smaller set.

Also adds some clean-up to time.go and adds a
design document describing standard time value
formats, and the maintenance thereof, in corso.

## Type of change

- [x] 🌻 Feature
- [x] 🗺️ Documentation
- [x] 🤖 Test

## Issue(s)

* #943
* #924

## Test Plan

- [x]  Unit test
- [x] 💚 E2E
2022-10-06 17:46:21 +00:00
ashmrtn
3f0951dea2
Higher resolution on test restore folder names (#1040)
## Description

\~Only affects folders created by tests\~
Create and use helper function that sets the restore destination folders to have microsecond granularity. This is to avoid naming collisions with second granularity (of which I've observed at least once)

Switches all tests for all services/categories to use OneDrive DateTime format+microseconds

Manually verified that a manually created folder with the given timestamp format was removed by the CI purge script

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

* closes #1036

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-10-04 23:52:13 +00:00
Keepers
d633be9eef
nightly load test footprint and workflow (#928)
## Description

Adds the footprint file for running nightly load
tests out of the repository package.  The tests
don't do anything at this stage, but that's okay.
Also introduces a workflow to kick off the tests
every day at 2am UTC.

## Type of change

- [X] 🤖 Test
- [X] 💻 CI/Deployment

## Issue(s)

* #902

## Test Plan

- [X] 💚 E2E
2022-09-22 15:16:25 +00:00
Keepers
6cdc691c6f
correct the corso module (#749)
swaps the corso go module from github.com/
alcionai/corso to github.com/alcionai/corso/src
to align with the location of the go.mod and
go.sum files inside the repo.

All other changes in the repository update the
package imports to the new module path.
2022-09-07 15:50:54 -06:00
Vaibhav Kamra
b1cd472483
Remove AWS credentials initialization (#610)
## Description

Instead of initializing static AWS credentials, we rely on the 
credential provider chain in [Kopia](https://github.com/kopia/kopia/pull/2213) to discover and initialize credentials.

This currently supports the following in this order:
- Static credentials
- Environment variables (what Corso used to implement)
- IAM

Going forward, this will also allow us to support shared credentials (cred file) once that is
added to the credential provider chain.

## Type of change

Please check the type of change your PR introduces:
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 🐹 Trivial/Minor

## Issue(s)
- #384 

## Test Plan

<!-- How will this be tested prior to merging.-->

- [x] 💪 Manual (see test output below)
- [ ]  Unit test
- [x] 💚 E2E
2022-09-02 18:38:01 +00:00
ashmrtn
09cc2769d9
Fix most remaining wsl lint errors (#656)
* Fix wsl lint errors in pkg package

* Fix wsl lint errors in most of internal package

Leave some sub-packages out that have higher churn at the moment.
2022-08-26 17:58:58 +00:00
Keepers
99908b914d
propagate test cache dir in cli integration tests (#634) 2022-08-23 17:13:07 +00:00
Keepers
854635ac24
allow tests to set a specific kopia config dir (#607) 2022-08-19 00:24:32 +00:00
Keepers
649255f112
fix suite in restore cli integration tests (#603) 2022-08-18 22:52:33 +00:00
Danny
ce087a2fd2
Fixes the resource-not-found-fix (#587)
Bug fix-> UserID replaces the use of a TenantID on query.
2022-08-17 18:17:34 -04:00
Keepers
192c69c68f
add backup cli integration test (#517)
Adds the basic cli-based backup integration test.
Due to discovering some corner cases about panic
conditions and other error handling in bad runtime
state, updates many other packages for safety.
2022-08-12 20:04:44 +00:00
ashmrtn
e76860fd80
Set line length to 120 characters (#506)
* Enable line width linter

Set to 120 which should be long enough to not be annoying but keep
things from getting "too long." Adding to get rid of the subjectiveness
of what is "too long." Tabs count as a single character.
2022-08-12 16:05:46 +00:00
Vaibhav Kamra
88bcd053e5
Simplify test utilities (#532) 2022-08-12 02:49:22 +00:00
Keepers
0cb11e0cc5
clean up env var tester func for multiple slices (#508) 2022-08-12 00:08:00 +00:00
Keepers
5070296e18
add connect cli integration tests (#493) 2022-08-05 12:05:45 -07:00
ashmrtn
9e2e88f5f3
Enable some linters in revive (#486)
* Turn on revive linter, ignore only a few things

* Fix lint errors

Ignore shadowing of 'suite' in tests for now. Also move some constants
that had the same value to tester.
2022-08-04 14:42:51 -07:00
Keepers
342dd2e9f9
set up CLI for integration testing (#478)
Makes the necessary changes, including adding helper
funcs, to bring the CLI up to an integration-testable
state.  The changes made in this commit should be
sufficient for most other CLI tests.  Includes a single
test as verification.
2022-08-04 17:15:13 +00:00
ashmrtn
d920589507
Add linter for import groups and group order (#477)
* Add import order linter

Error messages aren't the most clear/helpful, but they will point you to
the lines that need modifying.

* Adjust max errors reported

* Fix lint errors
2022-08-03 16:21:38 -07:00
Keepers
640dfcf967
derive test userID from test config (#465)
In order to both maintain control of test env data, as
well as ensure open-source users are able to run the
integration tests on personal account data, the test
user must be derived from the test configuration, rather
than hardcoded within the tests.
2022-08-02 19:13:23 +00:00
Keepers
d7fe518007
rename /internal/testing to /tester (#463)
The alias of testing to ctesting prevented auto-importing
of the testing library.  This change (arbitrarily) renames
the package so that it doesn't collide with the core pkg
for "testing".
2022-08-02 12:21:14 -06:00