999 Commits

Author SHA1 Message Date
ashmrtn
9e2e88f5f3
Enable some linters in revive (#486)
* Turn on revive linter, ignore only a few things

* Fix lint errors

Ignore shadowing of 'suite' in tests for now. Also move some constants
that had the same value to tester.
2022-08-04 14:42:51 -07:00
Keepers
a586512b42
add command to purge user's folders by prefix/date (#480) 2022-08-04 15:28:28 -06:00
Keepers
342dd2e9f9
set up CLI for integration testing (#478)
Makes the necessary changes, including adding helper
funcs, to bring the CLI up to an integration-testable
state.  The changes made in this commit should be
sufficient for most other CLI tests.  Includes a single
test as verification.
2022-08-04 17:15:13 +00:00
ashmrtn
d920589507
Add linter for import groups and group order (#477)
* Add import order linter

Error messages aren't the most clear/helpful, but they will point you to
the lines that need modifying.

* Adjust max errors reported

* Fix lint errors
2022-08-03 16:21:38 -07:00
Danny
bb7531d95c
exchange.Collection to fill in key information from selectors (#457)
Selectors and scope information are used to create the correct populate methods. Populate methods changed to fit type abstractions definitions for iteration, querying, and collection population.
2022-08-03 19:07:26 -04:00
Danny
d656bc181c
Regression test added for serialization and (de)serialization process… (#461)
* Regression test added for serialization and (de)serialization process for graphConnector.
2022-08-03 17:53:19 -04:00
ashmrtn
04b4f3935e
Use different method to get kopia write session (#474)
Kopia maintainer mentioned this was a better way to deal with it than
the old version. Mostly superficial changes to logic.

Always safe to flush the write session during an upload. That will
ensure write sessions are closed. Validity of the uploaded data will
be via reachability of the data in the repo.

Drawback is if there's a problem flushing the write session it will
return that error instead of the original error (if there was one)
2022-08-03 19:05:35 +00:00
Danny
2b95553355
Gc serialize to collection (#467)
Abstractions: Exchange selector function introduced, GraphIterateFuncs and service functions extended
2022-08-03 14:26:06 -04:00
Danny
186c2d9dc4
Restore into Custom Folder (#391)
Custom folder added for restore process workflow
2022-08-03 12:14:45 -04:00
Keepers
640dfcf967
derive test userID from test config (#465)
In order to both maintain control of test env data, as
well as ensure open-source users are able to run the
integration tests on personal account data, the test
user must be derived from the test configuration, rather
than hardcoded within the tests.
2022-08-02 19:13:23 +00:00
Keepers
d7fe518007
rename /internal/testing to /tester (#463)
The alias of testing to ctesting prevented auto-importing
of the testing library.  This change (arbitrarily) renames
the package so that it doesn't collide with the core pkg
for "testing".
2022-08-02 12:21:14 -06:00
Danny
87427a52aa
Gc iterate abstraction (#459)
Abstraction functions and iterate functions created for the `serializeMessages` move to createCollections. 
Graph Connector refactor still underway.
2022-08-02 13:31:53 -04:00
Danny
a1b8868600
Removal of internal Contains methods (#462)
Removal of internal  `contains`methods
2022-08-02 12:49:21 -04:00
Danny
3eaebaed8f
Incremental change prior to selector (#460)
Use of abstract variable for creating page iterator.
2022-08-02 12:21:35 -04:00
Danny
bcbf0b1fb8
GetMailFolderID feature added (#455)
GetMaillFolderID add to `exchange` package. Test Suite Added.
2022-08-01 15:12:02 -04:00
Danny
890c8f2b7f
Exchange service: Abstractions (#450)
- Creation of exchange service-based query file. Test package expanded for support
2022-08-01 13:27:30 -04:00
Danny
bd559f690e
Exchange service functions (#454)
`exchange`.service_actions.go created. Create/Delete Mail folder functions inherited. Test suite updated accordingly.
2022-08-01 16:29:03 +00:00
Danny
2f56a849c9
Exchange.Collection.Items() to initiate populate method (#446)
`Items()` starts populate method for collection for the `exchange.Collection` package. Test packages expanded to verify the functionality of this feature.
2022-07-31 17:57:14 +00:00
Keepers
babf8b8d95
replace operation Successful with Completed (#426) 2022-07-29 21:25:02 +00:00
Keepers
7c62939c85
update GC's restore to use AwaitStatus (#451)
GC's restore funcs update their status field differently than
the backup funcs.  This brings the behavior in line.
2022-07-29 21:09:15 +00:00
Danny
da9a22271d
Mock Connector expansion (#437)
Mock Connector data collection is a Messageable item and has been changed to be utilized in other portions of the corso repo.
2022-07-29 15:08:36 -04:00
Keepers
7e60ec5073
move operation.Options into /pkg/control (#452)
In order for sdk users to build an operation using a repo,
without importing /inernal packages, the operation
options needs to be available via a /pkg package.
2022-07-29 12:52:19 -06:00
Danny
5a68108006
Gc separate tests (#448)
Merge places online and offline tests into different files.
2022-07-29 11:33:15 -06:00
Danny
e6f5b4515d
GC: exchange.Collection PopulateFunction refactor without TaskList (#441)
exchange.PopulateFunc executed without a tasklist.
2022-07-29 12:35:27 -04:00
Danny
9eb6a5dd0c
GC: Tasklist removed from packages (#440)
Tasklist struct removed from code base
2022-07-29 12:32:58 +00:00
Keepers
9ea0409125
print backup details at the end of backup create (#439) 2022-07-28 17:53:49 -06:00
Danny
992504ed36
exchange.Collection FinishPopulation function removed (#436)
* removal of FinishPopulation() method.
* graph_connector disconnected from exchange collection internal functions.
2022-07-28 14:08:35 -04:00
Keepers
4862d5b800
return errors if selectors reduce all entries (#427)
* return errors if selectors reduce all entries

During a `backup details` or `restore` command, if the
selectors produce 0 results, return an error rather than
running a no-op process.
2022-07-28 11:03:30 -06:00
Danny
a4d99a74ed
Adapter creation moved to graph package (#432)
CreateAdapter function exported to the `graph` package. This will be the same area that authentication and token refreshment will eventually go.
2022-07-28 16:06:33 +00:00
Danny
987980510c
exchange.Collection addition of job field to struct (#434)
`exchange.Collection` field **jobs** `type: [] string` added to struct
2022-07-28 15:50:55 +00:00
Danny
402b3aae06
Exported fields removed (#433)
Exported fields removed.
2022-07-28 15:22:28 +00:00
ashmrtn
dac8c03139
Add tests for path.ExchangeMail (#428)
Tests for path.ExchangeMail were not originally included because
the functionality for them was not implemented yet
2022-07-28 15:17:51 +00:00
Danny
8bfff3c88f
Population Function Moved to Exchange Package (#429)
Functions and interfaces moved to interact properly with connector package. Fields are no longer exported until the required functions are moved to the exchange package.
2022-07-27 19:18:20 -04:00
Danny
665fa21b13
ExchangeDataCollection moved to exchange.ObjectCollection (#423)
Exchange DataCollection moved to the exchange package. Unused fields removed at this time.
2022-07-27 21:22:27 +00:00
Keepers
e35acb79ca
rename stableID to ID (#425)
embedding a baseModel means that objects like backup
have a backup.StableID, but no backup.ID nor backup.
backupID.  This could be confusing for users.  This change
swaps the value and type names, so that baseModels
include an ID value of type StableID.
2022-07-27 20:54:58 +00:00
ashmrtn
e50728c0d6
Add function to get unescaped elements for a path segment (#422)
* Function and tests for returned unescaped elements

* Regression test and fix for empty segment bug

If a raw segment had no elements that had length > 0 or just didn't have
any elements it would still create a segment, throwing everything else
off. Explicitly test for that now.
2022-07-27 19:53:15 +00:00
Danny
34a7a1a80c
Data Collection --> Collection refactor (#415)
DataCollection changed to Collection in the repository. All associated imports changed to reflect the change.
2022-07-27 12:04:31 -04:00
ashmrtn
9b28d71705
fill in more path functions (#401)
Segment-based ctor and tests
2022-07-27 15:24:37 +00:00
Danny
3e792e69eb
Move DataCollection to data package (#414)
DataCollection moved to `/src/internal/data` directory and imports changed to recognize the move.
2022-07-27 08:04:35 -04:00
Keepers
587c239dd1
store backup operation results in the backup manifest (#410)
* store backup operation results in the backup manifest

Adds backup operation metadata like the outcome
statistics and selector definitions to the backup manifest
entry.  These additional details will appear when users
call `corso backup list`.
2022-07-26 19:35:14 +00:00
Keepers
350d27dcb4
Issue-271-mustfilter: add must-pass filters to selectors (#392)
* add must-pass filters to selectors

Extends the selectors scope set to include Filters.  This
allows users to define all-pass matchers (filters),
separate from any-pass matchers (includes), and global
exclusions.
2022-07-25 22:41:49 +00:00
ashmrtn
ea3c9c035e
Implement and test one path constructor and join (#398)
Implement and test one path constructor and join
2022-07-25 20:31:09 +00:00
Keepers
5a532e808e
fix errant conditional on GC restore success counter (#407) 2022-07-25 13:14:13 -06:00
Sidhartha Mani
6a9ccf2a51
add --succeed-on-exists flag to make s3 repo init idempotent (#324)
* [Minor #323] add --success-on-exists flag to make  idempotent

Co-authored-by: ryanfkeepers <ryanfkeepers@gmail.com>
Co-authored-by: Keepers <104464746+ryanfkeepers@users.noreply.github.com>
2022-07-25 18:42:24 +00:00
Danny
25c781eec6
Gc edc extension (#402)
* Extension of ExchangeDataCollection for async population.
2022-07-25 12:46:17 -04:00
Keepers
dcbc149aa7
Build binary test separately (#399)
* refactor ci caching to ensure go test runs without downloading packages
2022-07-25 09:05:40 -06:00
Keepers
0739ea7e09
add info-based selector flags to backup details (#380)
* filter backup details by flags

`backup details` should have its output filtered by the flags provided by
the user.  In addition, the selector's FilterDetails should maintain
information (esp service info) about the entries, rather than slicing them
down to only the path reference.
2022-07-22 13:23:16 -06:00
ashmrtn
395c7c8525
Add spell check linter and run gofmt in golangci-lint (#396)
* Basic golangci-lint config with gofmt and timeout

Remove corresponding sections from other CI config files.

* Enable comment and string spelling linter

* Fix spelling mistakes for linter

Co-authored-by: Danny <danny@alcion.ai>
2022-07-22 18:28:25 +00:00
ashmrtn
da8ff3c267
Define struct and interface for corso paths (#339)
* Define struct and interface for corso paths
* Add wrapper for ExchangeMail as an example
2022-07-22 10:25:55 -07:00
Keepers
532922f662
introduce exchange info selector support (#379)
* introduce exchange info selector support

Adds support in selectors/exchange for queries based on
backup.ExchangeInfo entries.  This allows the declaration
of selectors based on non-identifier details such as sender,
subject, or receivedAt time.

Changes Exclude scope matching from being an Any-
match comparator to an All-match.  This keeps exclude
and include behavior identical, hopefully making less
confusion for users.
2022-07-21 15:08:03 -06:00