298 Commits

Author SHA1 Message Date
Keepers
854635ac24
allow tests to set a specific kopia config dir (#607) 2022-08-19 00:24:32 +00:00
Vaibhav Kamra
c3c361282d
Onedrive selectors (#594)
## Description

Introduces OneDrive selectors - currently only users are supported.

## Type of change

Please check the type of change your PR introduces:
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 🐹 Trivial/Minor

## Issue(s)
- #388  

## Test Plan

<!-- How will this be tested prior to merging.-->

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-08-19 00:00:28 +00:00
Vaibhav Kamra
8fd867745f
Onedrive item reader (#582)
## Description

This contains the following changes:
- Support functions to enumerate a users drives and the files within it.
- `driveItemReader` method that that reads a drive item
- Integration tests for the above

## Type of change

Please check the type of change your PR introduces:
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 🐹 Trivial/Minor

## Issue(s)
- #388 

## Test Plan

<!-- How will this be tested prior to merging.-->

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-08-18 23:38:23 +00:00
Keepers
649255f112
fix suite in restore cli integration tests (#603) 2022-08-18 22:52:33 +00:00
Keepers
42df5cef58
push doc autogeneration changes to new branch (#531) 2022-08-18 22:34:03 +00:00
Vaibhav Kamra
350b148155
Instructions on running tests (#588)
* Add testing instructions

* Update link
2022-08-18 20:09:40 +00:00
Keepers
91e5c3d90c
add the corso env command for env var guidance (#591) 2022-08-18 19:09:25 +00:00
Keepers
3ac05acf64
introduce log-level control (#589)
Introduces the log-level flag, defaulting to info.  Also does
a minor refactor of how Print is called for backup results,
which moves the backup/details imports out of the cli/print,
and instead has thoses packages call a Print func.
2022-08-18 18:30:15 +00:00
Keepers
c13ef798eb
use scope type instead of map (#597) 2022-08-18 18:06:22 +00:00
Keepers
e198aa9ae7
add scopes.go (#579)
Introduces selectors/scopes.go, which is the base file for
managing service-agnostic scope logic.  Funcs like "isAny",
"contains", "getValue" and etc are expected to be housed
here, so that service instances only need to provide thin
wrappers, mostly typecasting, around their behavior.
2022-08-18 11:21:14 -06:00
Vaibhav Kamra
f8bfe93261
Scope restore CLI test (#592) 2022-08-18 14:19:08 +00:00
Georgi Matev
0ebf4d33e3
Start building out README and basic structure (#590) 2022-08-18 13:42:26 +00:00
Danny
ce087a2fd2
Fixes the resource-not-found-fix (#587)
Bug fix-> UserID replaces the use of a TenantID on query.
2022-08-17 18:17:34 -04:00
Vaibhav Kamra
c4e5915983
OneDrive Collection (#580)
## Description

Introduces a OneDrive data collection.

Follow up PRs will implement the `collection.driveItemReader()` method that uses the Graph API

## Type of change

Please check the type of change your PR introduces:
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 🐹 Trivial/Minor

## Issue(s)
- #387 

## Test Plan

<!-- How will this be tested prior to merging.-->

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-08-17 19:38:05 +00:00
Keepers
a1b9f876ae
add final basic cli tests (#578)
Adds the base CLI integration tests for backup, list, details,
and restore.  Also refactors out the global root command
value in favor of a ctx-bound reference so that tests may
control safely overwriting stdout to scrutinize output.
2022-08-17 18:13:55 +00:00
Danny
9049f3c2bf
552 gc mockdata events (#583)
Addition of mock event data structure and testing suite.
2022-08-17 11:22:39 -06:00
Danny
90be4200ca
GC: Events Support added. Not connected within ExchangeDataCollection (#516)
m365 events added for querying and iteration. Test suite added. Mock data is in a different PR.
2022-08-17 16:29:37 +00:00
Danny
ad2c17876f
ContactInfo added to exchange package (#574)
ContactInfo created for models.Contactable. ContactInfo test suite created.
2022-08-17 13:47:27 +00:00
Danny
41f22ad9a4
Added error trace reading for query calls. (#575)
Added error tracing that gives more details for failed GraphQuery reading for query calls.
2022-08-17 13:07:53 +00:00
ashmrtn
24b40cf601
Persist and enable kopia compression options (#577)
* Skip updating policy if no changes needed

* Set default compressor on repo connection and initialization

* Factor out common code for connecting to a repo

* New unit test ensuring compressor is set properly

Test compressor is set both through Initialize and Connect because
client code doesn't have to call Connect if it just did Initialize.
2022-08-16 23:19:57 +00:00
Vaibhav Kamra
f870667856
Fix docker build command (#521) 2022-08-15 23:16:43 +00:00
ashmrtn
9fc5cbb9d2
Allow configuring kopia compression (#544)
* Function that allows adjusting global compression

Takes the name of a kopia compressor and sets the default compressor to
be that in the repo.

* Basic unit tests for setting compression

Ensures that the policy is actually applied to future snapshots that
kopia will take.

* Integration test for changing compression types

Make sure that data is still available even if the global compression
policy changes from what it was when the backup was made.
2022-08-15 15:56:21 -07:00
Danny
2f7d8420ae
551 mockdata contacts (#553)
Creation of mock data for Contact M365 objects.
CreateContactFromBytes function added to support package
2022-08-15 16:00:58 -04:00
Danny
be7b778769
GC Contacts requires additional tests placed in Exchange Services (#475)
Test coverage extended to GraphQuery and GraphIterateFuncs for exchange.Mail and exchange.Contact use cases.
2022-08-15 17:50:06 +00:00
Keepers
9c62134cb2
change command folding in cleanup yml (#546) 2022-08-15 10:36:07 -06:00
Keepers
84fae9e215
fix action useage of date command (#545) 2022-08-12 18:25:23 -06:00
Keepers
6c5d6d167c
skip build cache in golangci-lint (#540) 2022-08-12 15:24:39 -07:00
Keepers
a8960129c8
fix bad substitution in yml env declaration (#538) 2022-08-12 22:12:12 +00:00
ashmrtn
a43a98e8ad
Document installing/running/interpreting linters (#541)
Also minor documentation for import formatting expectations.
2022-08-12 14:20:53 -07:00
ashmrtn
ea1cdcf119
Update to current HEAD of kopia's main branch (#534)
Pull in recent patches allowing the use of AWS IAM credentials
instead of tokens.
2022-08-12 13:13:57 -07:00
Keepers
192c69c68f
add backup cli integration test (#517)
Adds the basic cli-based backup integration test.
Due to discovering some corner cases about panic
conditions and other error handling in bad runtime
state, updates many other packages for safety.
2022-08-12 20:04:44 +00:00
Keepers
6464a66b46
use standard user env var in graphconnector tests (#537) 2022-08-12 18:40:32 +00:00
Keepers
c052536094
proper env value retrieval in test cleanup yml (#535) 2022-08-12 12:17:17 -06:00
Keepers
ceec4dfb45
prevent panics in failed operation runs (#524) 2022-08-12 17:34:49 +00:00
Keepers
9f1c8aa64c
do not auto-generate date updates in cli markdown (#526) 2022-08-12 16:43:19 +00:00
ashmrtn
e76860fd80
Set line length to 120 characters (#506)
* Enable line width linter

Set to 120 which should be long enough to not be annoying but keep
things from getting "too long." Adding to get rid of the subjectiveness
of what is "too long." Tabs count as a single character.
2022-08-12 16:05:46 +00:00
ashmrtn
53a0207870
Make rules to check for linter and run it (#519)
Explicitly check for the linter version as different versions can return
different results. Currently the desired version is hard-coded, but may
be sourced from the github workflow file in the future.
2022-08-12 08:46:15 -07:00
Vaibhav Kamra
88bcd053e5
Simplify test utilities (#532) 2022-08-12 02:49:22 +00:00
Keepers
79e7be71bf
add steps to ci_test_cleanup (#522) 2022-08-12 00:27:48 +00:00
Keepers
0cb11e0cc5
clean up env var tester func for multiple slices (#508) 2022-08-12 00:08:00 +00:00
Vaibhav Kamra
af86107c86
Disable creating docs commit (#530) 2022-08-11 23:36:07 +00:00
ashmrtn
da66cc4c2f
Enable more rigorous version of gofmt linting (#488)
* Enable a stricter linter

* Fix new lint errors
2022-08-05 13:33:20 -07:00
Danny
eff95b7702
GC: Contact Framework added (#473)
`exchange`.Contacts added to Collection retrieval option base through the use of a selector. 

The ExchangeDataCollection Function to be updated to make the function `Live` to be called externally
2022-08-05 16:19:48 -04:00
Keepers
5070296e18
add connect cli integration tests (#493) 2022-08-05 12:05:45 -07:00
Keepers
20aa3d3a28
purge test user folders every half hour (#491) 2022-08-05 18:24:58 +00:00
ashmrtn
195d5efccc
Allow kopia directories to have streamed items and subdirectories (#505)
* Remove artificial limit on kopia directories

Originally did not allow a directory to have both child directories and
items. Remove that limit and move logic to execute callbacks on static
items to the iteration function.

* Update tests for new kopia directory structure
2022-08-05 11:18:07 -07:00
ashmrtn
133314ebaa
Minor code maintenance for ModelStore (#492) 2022-08-05 08:49:44 -07:00
Danny
97113aa80b
Folder Granularity scope Added to Collections Retrieval (#482)
Backup function `DataExchangeCollections` supports the ability to backup a single file using the `selectors` package. Support files and documentation updated accordingly
2022-08-05 11:20:36 -04:00
Keepers
73942fe952
fix up logger specs (#489) 2022-08-04 23:28:26 +00:00
Keepers
2d6c765ee8
add contains helpers to exchange scope (#487) 2022-08-04 23:17:52 +00:00