115 Commits

Author SHA1 Message Date
Keepers
c73ca6a4e5
correct paths in selectors (#789)
The path package changed the standard format
of fullPath and repoRef design.  This should have
failed tests before being pushed to main, but was
able to slip in falsely while github actions were
configured to pass all tests until failed.
2022-09-07 14:14:51 -06:00
ashmrtn
12f61dd71c
Switch to using path categories in graph connector (#734)
Will help make later PRs easier as the category will already be known.

## Description

<!-- Insert PR description-->

## Type of change

Please check the type of change your PR introduces:
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [x] 🐹 Trivial/Minor

## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
on path to:
* #456 

## Test Plan

<!-- How will this be tested prior to merging.-->

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-09-02 20:12:22 +00:00
Vaibhav Kamra
b1cd472483
Remove AWS credentials initialization (#610)
## Description

Instead of initializing static AWS credentials, we rely on the 
credential provider chain in [Kopia](https://github.com/kopia/kopia/pull/2213) to discover and initialize credentials.

This currently supports the following in this order:
- Static credentials
- Environment variables (what Corso used to implement)
- IAM

Going forward, this will also allow us to support shared credentials (cred file) once that is
added to the credential provider chain.

## Type of change

Please check the type of change your PR introduces:
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 🐹 Trivial/Minor

## Issue(s)
- #384 

## Test Plan

<!-- How will this be tested prior to merging.-->

- [x] 💪 Manual (see test output below)
- [ ]  Unit test
- [x] 💚 E2E
2022-09-02 18:38:01 +00:00
Keepers
784f006da5
don't split json output by type (#725)
## Description

Prevents output printing of detail entries from splitting into
multiple json arrays when the entries contains multiple item
type.s

## Type of change

Please check the type of change your PR introduces:
- [x] 🐛 Bugfix

## Issue(s)

#501 

## Test Plan

- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2022-09-01 22:06:35 +00:00
Keepers
da6288cb8b
minor value fixes in mail consts/types (#722)
- [x] 🐹 Trivial/Minor
- [x] 🐛 Bug

closes #726
2022-09-01 21:34:57 +00:00
Keepers
a043304e0b
add eventOrganizer selector (#720)
## Description
Adds the eventOrganizer exchange filter, wraps
up the addition of eventRecurs filtering, and fixes
a couple improperly designed filter constructors.

## Type of change

Please check the type of change your PR introduces:
- [x] 🌻 Feature

## Issue(s)
#501

## Test Plan

- [x] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-09-01 19:56:17 +00:00
Keepers
c29d93e655
adds calendars as a folder structure to events (#714)
## Description

Exchange events were amended to use calendars
as a folder structure.  This updates the selector
to treat events as having folders similar to mail and
contacts.

## Type of change

Please check the type of change your PR introduces:
- [x] 🌻 Feature

## Issue(s)

#501

## Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-09-01 15:39:14 +00:00
Keepers
5c59522fc2
print separate tables per entry type (#709)
## Description

Since a backup can encapsulate multiple data
types (ex: mail, contacts, and events), it doesn't
make sense to print one table with all disjoint
values.  This change splits up the print output
so that each data type in the details gets its
own table.

## Type of change

Please check the type of change your PR introduces:
- [x] 🌻 Feature

## Issue(s)

#501 

## Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-08-31 21:10:10 +00:00
Keepers
6de326a2c4
add event selector info filters (#703)
## Description

Adds filtering support for exchnage events in
the exchange selector.

## Type of change

Please check the type of change your PR introduces:
- [x] 🌻 Feature

## Issue(s)

#501

## Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-08-31 20:44:03 +00:00
Keepers
cfda6ff643
add contactName filter in ExchangeSelector (#697) 2022-08-31 19:31:53 +00:00
Keepers
7a8c6eaf78
clean up/simplify filter packagee (#695) 2022-08-31 16:43:54 +00:00
Danny
068f0560e4
Issue #501: Event support (#704)
Added EventRecurs as option for backup details.
2022-08-31 10:04:13 -06:00
Keepers
6f04321a60
replace scope values with filters (#674)
The filters package allows callers to specify both a target
to match on, and behavior of the comparison.  While data-
type scopes always equate to "equals", the control over
different comparison behavior is useful for info-type
filters.  This change integrates filters into scopes for
built-in control of those comparisons.
2022-08-30 21:29:24 +00:00
Keepers
56c7e0f85f
swap selector category types for ints (#661)
Though enumerated as ints, the selector service
categories are stored and managed as strings. The
only time that we use the int is when passing their
iota const to and from functions.  Using a string
type instead of an int allows us to get rid of the
string builders and AtoI funcs (though not the
stringer requirement in the Iface, sadly).
2022-08-29 19:46:08 +00:00
ashmrtn
09cc2769d9
Fix most remaining wsl lint errors (#656)
* Fix wsl lint errors in pkg package

* Fix wsl lint errors in most of internal package

Leave some sub-packages out that have higher churn at the moment.
2022-08-26 17:58:58 +00:00
Keepers
db2c1ec8e2
delete backups in modelStore and snapshot (#640)
Introduces manual deletion of existing backups.  The delete
includes: the modelStore backup, modelStore details, and
the kopia snapshot of the backup itself.
2022-08-24 16:30:27 +00:00
Keepers
15b12e634d
name cleanup in selectors and onedrive (#643) 2022-08-24 15:13:04 +00:00
Keepers
2d88e59cd0
Issue-556 introduce discreteScopes in selector handling (#635)
Graph may have an easier time parsing scopes if the root
user is discretely identified, as opposed to being a wildcard.
DiscreteScopes() automatically handles replacement of
scopes matching Any() user with the set of user provided
discrete IDs.
2022-08-23 21:56:24 +00:00
Keepers
b524053e96
resolve lint errors (#638) 2022-08-23 20:22:25 +00:00
Keepers
8d2a437f1d
clean up scope production, cat comparators (#630)
Scope production was still using service-type specific
factories instead of a generic factory set.  This has been
centralized so that all service instances share the same
scope production concerns.

Additionally, category comparator funcs now use generic
comparators as well, which allows for the removal of the
isType() and includesType() comparator funcs.
2022-08-22 18:02:54 -06:00
Keepers
e7b863c444
onedrive selector interface compliance (#626)
Scope filtering is currently hardcoded to the exchange
use case.  In order for future work to rely on boilerplate
rather than re-writing the full filtering logic on each new
type, as much of that code as is possible has been moved
into a generic toolset.
2022-08-22 21:35:46 +00:00
Keepers
12dbfce6d6
refactor scope filtering (#555)
Scope filtering is currently hardcoded to the exchange
use case.  In order for future work to rely on boilerplate
rather than re-writing the full filtering logic on each new
type, as much of that code as is possible has been moved
into a generic toolset.
2022-08-22 19:08:14 +00:00
Keepers
b483db0228
make exchangeScope comply with scoper (#619)
Centralizes as many of the exchange scope funcs as
possible into scopes.go.  Ensures exchangeScopes comply
with the scoper interface.  Reshuffles some test helper
code in selectors to a centralized file.
2022-08-19 22:46:55 +00:00
Danny
4f5bb2b995
Exchange event info (#569)
Exchange Info feature added and verified with `mockdata` to ensure correctness
2022-08-19 13:43:26 -06:00
Keepers
daa939f5f8
fullfill categorizer for exchange selectors (#602) 2022-08-19 17:43:53 +00:00
Keepers
854635ac24
allow tests to set a specific kopia config dir (#607) 2022-08-19 00:24:32 +00:00
Vaibhav Kamra
c3c361282d
Onedrive selectors (#594)
## Description

Introduces OneDrive selectors - currently only users are supported.

## Type of change

Please check the type of change your PR introduces:
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 🐹 Trivial/Minor

## Issue(s)
- #388  

## Test Plan

<!-- How will this be tested prior to merging.-->

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-08-19 00:00:28 +00:00
Keepers
3ac05acf64
introduce log-level control (#589)
Introduces the log-level flag, defaulting to info.  Also does
a minor refactor of how Print is called for backup results,
which moves the backup/details imports out of the cli/print,
and instead has thoses packages call a Print func.
2022-08-18 18:30:15 +00:00
Keepers
c13ef798eb
use scope type instead of map (#597) 2022-08-18 18:06:22 +00:00
Keepers
e198aa9ae7
add scopes.go (#579)
Introduces selectors/scopes.go, which is the base file for
managing service-agnostic scope logic.  Funcs like "isAny",
"contains", "getValue" and etc are expected to be housed
here, so that service instances only need to provide thin
wrappers, mostly typecasting, around their behavior.
2022-08-18 11:21:14 -06:00
Vaibhav Kamra
c4e5915983
OneDrive Collection (#580)
## Description

Introduces a OneDrive data collection.

Follow up PRs will implement the `collection.driveItemReader()` method that uses the Graph API

## Type of change

Please check the type of change your PR introduces:
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 🐹 Trivial/Minor

## Issue(s)
- #387 

## Test Plan

<!-- How will this be tested prior to merging.-->

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-08-17 19:38:05 +00:00
Danny
ad2c17876f
ContactInfo added to exchange package (#574)
ContactInfo created for models.Contactable. ContactInfo test suite created.
2022-08-17 13:47:27 +00:00
ashmrtn
e76860fd80
Set line length to 120 characters (#506)
* Enable line width linter

Set to 120 which should be long enough to not be annoying but keep
things from getting "too long." Adding to get rid of the subjectiveness
of what is "too long." Tabs count as a single character.
2022-08-12 16:05:46 +00:00
Vaibhav Kamra
88bcd053e5
Simplify test utilities (#532) 2022-08-12 02:49:22 +00:00
Keepers
0cb11e0cc5
clean up env var tester func for multiple slices (#508) 2022-08-12 00:08:00 +00:00
ashmrtn
da66cc4c2f
Enable more rigorous version of gofmt linting (#488)
* Enable a stricter linter

* Fix new lint errors
2022-08-05 13:33:20 -07:00
Keepers
73942fe952
fix up logger specs (#489) 2022-08-04 23:28:26 +00:00
Keepers
2d6c765ee8
add contains helpers to exchange scope (#487) 2022-08-04 23:17:52 +00:00
ashmrtn
9e2e88f5f3
Enable some linters in revive (#486)
* Turn on revive linter, ignore only a few things

* Fix lint errors

Ignore shadowing of 'suite' in tests for now. Also move some constants
that had the same value to tester.
2022-08-04 14:42:51 -07:00
ashmrtn
d920589507
Add linter for import groups and group order (#477)
* Add import order linter

Error messages aren't the most clear/helpful, but they will point you to
the lines that need modifying.

* Adjust max errors reported

* Fix lint errors
2022-08-03 16:21:38 -07:00
Danny
186c2d9dc4
Restore into Custom Folder (#391)
Custom folder added for restore process workflow
2022-08-03 12:14:45 -04:00
Keepers
d7fe518007
rename /internal/testing to /tester (#463)
The alias of testing to ctesting prevented auto-importing
of the testing library.  This change (arbitrarily) renames
the package so that it doesn't collide with the core pkg
for "testing".
2022-08-02 12:21:14 -06:00
Keepers
7e60ec5073
move operation.Options into /pkg/control (#452)
In order for sdk users to build an operation using a repo,
without importing /inernal packages, the operation
options needs to be available via a /pkg package.
2022-07-29 12:52:19 -06:00
Keepers
9ea0409125
print backup details at the end of backup create (#439) 2022-07-28 17:53:49 -06:00
Keepers
aca3d856af
add printable backup struct and tabular formatting (#435) 2022-07-28 19:25:16 +00:00
Keepers
99a210b6da
add jsonDebug, MinimumPrintable to interface (#430)
* add jsonDebug, MinimumPrintable to interface

We want to optionally print print either debugging
(containing all internal data) or user-friendly json
outputs.  This requires a new flag: json-debug, and
an extension to the Print interface: minimumPrintable.
2022-07-28 16:26:30 +00:00
Keepers
e35acb79ca
rename stableID to ID (#425)
embedding a baseModel means that objects like backup
have a backup.StableID, but no backup.ID nor backup.
backupID.  This could be confusing for users.  This change
swaps the value and type names, so that baseModels
include an ID value of type StableID.
2022-07-27 20:54:58 +00:00
Keepers
cc810fa3ab
introduce printable version of selectors (#424) 2022-07-27 20:14:53 +00:00
Keepers
587c239dd1
store backup operation results in the backup manifest (#410)
* store backup operation results in the backup manifest

Adds backup operation metadata like the outcome
statistics and selector definitions to the backup manifest
entry.  These additional details will appear when users
call `corso backup list`.
2022-07-26 19:35:14 +00:00
Keepers
1929b2307f
reduce filter selector flags to single strings (#393)
* reduce filter selector flags to single strings

Since it doesn't make sense for many of the filter-type
selector flags to receive multiple input values (ex: what
would --received-after date1,date2 result in?), the filter
flags are getting reduced to single-string values.
2022-07-26 10:31:34 -06:00