403 Commits

Author SHA1 Message Date
ashmrtn
d1bf2b90a6
Re-add logic for getting Exchange mail paths (#648)
* Re-add logic for getting Exchange mail paths

Paths can either be for an item (email) or a folder. Returned struct
provides safe access to underlying information in the path once it is
created.

* Refine tests, add struct comment

Add test to make sure Item() and Folder() work properly for a resource
that has no parent folder, just the prefix stuff.

* Rework resource-specific paths based on comments

* use a single type to represent resource specific paths
* use service/category enums to denote the resource the path represents
* update tests to check for service/category of the path
* rename exchange_path*.go -> resource_path*.go
2022-08-30 22:45:47 +00:00
Danny
67bc038c55
GC: Add WSL Linting Formatting src/internal/connector (#694)
Lint remaining of `internal/connector` package for wsl
2022-08-30 22:12:38 +00:00
Keepers
6f04321a60
replace scope values with filters (#674)
The filters package allows callers to specify both a target
to match on, and behavior of the comparison.  While data-
type scopes always equate to "equals", the control over
different comparison behavior is useful for info-type
filters.  This change integrates filters into scopes for
built-in control of those comparisons.
2022-08-30 21:29:24 +00:00
ashmrtn
e3abc281d6
Enable wsl for most of internal/connector package (#680)
* Lint part of internal/connector package with wsl

* Cleanup some wsl lint errors in connector package
2022-08-30 13:13:00 -07:00
ashmrtn
ee57c7cb6c
Ignore multiple trailing '/' in path elements (#690)
Multiple unescaped trailing path separators should be trimmed from input elements so there are no empty elements or trailing unescaped separators at the end of the path or between elements
2022-08-30 19:30:28 +00:00
Danny
0b7fe0fdb9
GC: Small GC Refactor of Restore workflow (#682)
Removal of 'unused' if-clause and category variable being updated.
2022-08-30 18:50:31 +00:00
Danny
0b3f345727
GC: Restore Contact Interface (#642)
Functionality added to restore `exchange.Contact` to M365 back store. Unit tests added to support proper testing of new feature.
2022-08-30 17:46:47 +00:00
ashmrtn
ae06b36e7a
Enable wsl linter for most of codebase (#679)
Turn on now to keep from cycle of fixing most code, waiting for some PRs
in other packages, and then needing to do more fixups. Will slowly
enable on various files in connector package until we get full coverage.
2022-08-29 23:14:11 +00:00
ashmrtn
c3e89db0df
Lint doc for wsl (#678)
Some tips and examples on coexisting with the wsl linter
2022-08-29 22:46:57 +00:00
ashmrtn
68a1d8e868
Revisions left out from #541 (#677)
Revisions left out from #541
2022-08-29 22:12:26 +00:00
Danny
b2d3e62536
Contact Restore: Create / Delete Folders
Contact folder creation and deletion enabled by the PR. Test suites expanded
2022-08-29 21:23:34 +00:00
Keepers
3f6f6604a2
remove extraneous debug logs from cli (#675) 2022-08-29 20:48:23 +00:00
Keepers
56c7e0f85f
swap selector category types for ints (#661)
Though enumerated as ints, the selector service
categories are stored and managed as strings. The
only time that we use the int is when passing their
iota const to and from functions.  Using a string
type instead of an int allows us to get rid of the
string builders and AtoI funcs (though not the
stringer requirement in the Iface, sadly).
2022-08-29 19:46:08 +00:00
Vaibhav Kamra
81cb76dd55
Pull request template (#673) 2022-08-29 12:16:45 -07:00
Niraj Tolia
4abc4edfcc
Add am image zoom plugin for Docusaurus (#666) 2022-08-29 17:13:42 +00:00
Danny
d55ee98d7d
GC: Iterate "Select All Messages/Contacts" Consolidation with new Interface (#663)
`Descendable` interface used to move "IterateSelectAll" functions into a singular function.
Test suites are modified to support changes.
2022-08-29 16:29:15 +00:00
Georgi Matev
9f6b1bc1b8
Docs structure and initial content (#658)
* Clean up tree structure and add proper logo

* Proper logo files

* Initial docs

* Pick up new page after merging main

* Apply suggestions from code review

Some nits

Co-authored-by: Niraj Tolia <ntolia@users.noreply.github.com>

* Quash a bunch of Vale prose linter errors, warnings, and suggestions.

* PR Review feedback
* Simplified mount folders
* Cleaned up windows container
* Hid `latest` release tag

* Review feedback: Link to repo page

* Better arrow symbol

Co-authored-by: Niraj Tolia <ntolia@users.noreply.github.com>

* Clarify sentence

* Update README.md

* Reframing focus to M365

* More linting

* Use only MD files

Co-authored-by: Niraj Tolia <ntolia@users.noreply.github.com>
2022-08-26 20:45:50 -07:00
Georgi Matev
802001513f
Some quality of life docs tooling improvements (#664) 2022-08-26 19:03:36 -07:00
Danny
d3bced28f9
Bug fix: (#662)
Fixes bug introduced by merge.
2022-08-26 16:28:13 -06:00
Danny
ab92358ea6
GC: Backup all users flag support for exchange (#631)
All-users flag support enabled for Graph-Connector
2022-08-26 15:05:14 -06:00
Danny
2ceafd8d62
Contact / Mail folder interfaces (2) (#659)
Added 2 interfaces for future abstractions.
2022-08-26 16:15:55 -04:00
ashmrtn
09cc2769d9
Fix most remaining wsl lint errors (#656)
* Fix wsl lint errors in pkg package

* Fix wsl lint errors in most of internal package

Leave some sub-packages out that have higher churn at the moment.
2022-08-26 17:58:58 +00:00
Danny
02c2b0b4d6
GC: Iterate-Functions moved to separate file (#657)
* Issue #654: Prepwork: `service_iterate.go` contains iterate functions for the package.
2022-08-26 11:47:49 -06:00
Danny
a2a5260d3f
Gc option extract (#653)
Content Separation `query_options.go` created for `exchange package.
2022-08-26 12:47:45 -04:00
ashmrtn
88a318a7c0
Fix wsl errors in cmd and cli packages (#651) 2022-08-26 09:13:39 -07:00
ashmrtn
f24ad6ccbd
Simplify path struct (#647)
* Rewrite basic path logic to be simpler

Make basic path logic deal only with path elements instead of elements
and segments. Upper-layer logic can deal with elements.

Base path logic does not require a complete resource path as would be
seen by kopia, it just manages splitting/joining/escaping path elements.
Will have transformers to go from a basic path to a resource specific
path in a follow up.

Remove upper-layer logic for now to reduce load while reviewing as it
also changed slightly. Will be re-added in a follow up
2022-08-26 08:43:46 -07:00
ashmrtn
c4e9046870
Fix wsl lint errors in kopia package (#650) 2022-08-26 08:13:25 -07:00
Keepers
f1370b36e4
add cli commands to delete backups (#641) 2022-08-24 18:26:14 +00:00
Keepers
db2c1ec8e2
delete backups in modelStore and snapshot (#640)
Introduces manual deletion of existing backups.  The delete
includes: the modelStore backup, modelStore details, and
the kopia snapshot of the backup itself.
2022-08-24 16:30:27 +00:00
Keepers
15b12e634d
name cleanup in selectors and onedrive (#643) 2022-08-24 15:13:04 +00:00
Keepers
2d88e59cd0
Issue-556 introduce discreteScopes in selector handling (#635)
Graph may have an easier time parsing scopes if the root
user is discretely identified, as opposed to being a wildcard.
DiscreteScopes() automatically handles replacement of
scopes matching Any() user with the set of user provided
discrete IDs.
2022-08-23 21:56:24 +00:00
Danny
3b9c8e284c
GC: Reduce the amount of folders upon restore (#637)
* Issue #595: All collections with the same `fullPath` name are to be stored within the same folder.
2022-08-23 20:46:59 +00:00
Danny
6e7335cc4d
Gc restore multi framework (#639)
* Abstraction extended to restore. Application logic moved to service_functions.go
2022-08-23 20:30:46 +00:00
Keepers
b524053e96
resolve lint errors (#638) 2022-08-23 20:22:25 +00:00
Danny
a1f79862f1
GC: Event collections to be referenced by calendarID instead of folder (#636)
`data.Collection` of type event  `fullPath()` contain the calendar id
2022-08-23 17:38:30 +00:00
Keepers
99908b914d
propagate test cache dir in cli integration tests (#634) 2022-08-23 17:13:07 +00:00
ashmrtn
e8e4bf0914
WIP 316 kopia upload errors (#330)
* Update to kopia with required callback

* Support structs for materializing backup details

Kopia will not allow us to pass data to it that should be passed back to
us in the `FinishedFile` callback. To work around this, create a small
thread-safe support struct that handles information about files kopia is
currently processing. Entries are removed from the set when kopia is
done with them and if no error occurred, the item's info will be added
to the BackupDetails.

* Switch to best attempt for iterating through files

Defaulting to "best-attempt" error handling where all data that didn't
result in an error is handed to kopia and then all errors encountered
are returned at the end.

* Test for uploads that have an error

Simple error reading a file. BackupDetails should not contain
information about the file that had the error (needs update to kopia
code and this code to pass). All other files should be present in kopia
and in BackupDetails.

Co-authored-by: Danny <danny@alcion.ai>
2022-08-23 08:18:29 -07:00
Danny
eca16e4a69
CLI: Contact and Event Flags active (#633)
Contact and Event flags restored
2022-08-23 12:29:19 +00:00
Keepers
8d2a437f1d
clean up scope production, cat comparators (#630)
Scope production was still using service-type specific
factories instead of a generic factory set.  This has been
centralized so that all service instances share the same
scope production concerns.

Additionally, category comparator funcs now use generic
comparators as well, which allows for the removal of the
isType() and includesType() comparator funcs.
2022-08-22 18:02:54 -06:00
Keepers
e7b863c444
onedrive selector interface compliance (#626)
Scope filtering is currently hardcoded to the exchange
use case.  In order for future work to rely on boilerplate
rather than re-writing the full filtering logic on each new
type, as much of that code as is possible has been moved
into a generic toolset.
2022-08-22 21:35:46 +00:00
Danny
d7abed1406
GC: setTenants not checking *users.GetMail() (#629)
User query updated from `*user.GetMail()` -> `*user.GetPrincipalName()`. Guard checks added and added GetUsers as one of the GraphQueryFunctions
2022-08-22 14:07:57 -07:00
Danny
b573882746
GC: Accept selectors.BackupExchange exchange.Event and exchange.Contacts (#605)
GraphConnector suite expanded to support backup of Mail, Contacts, and Events. e2e testing suite expanded for all 3 applications.
2022-08-22 20:52:01 +00:00
Keepers
d954c68216
optional arm docker image build (#624)
adds the `--arm` flag to /corso/build/build-container.sh
2022-08-22 19:22:09 +00:00
Keepers
12dbfce6d6
refactor scope filtering (#555)
Scope filtering is currently hardcoded to the exchange
use case.  In order for future work to rely on boilerplate
rather than re-writing the full filtering logic on each new
type, as much of that code as is possible has been moved
into a generic toolset.
2022-08-22 19:08:14 +00:00
ashmrtn
7d057dd2ac
Update to kopia with StreamingFile caching (#617) 2022-08-22 15:31:12 +00:00
Keepers
b483db0228
make exchangeScope comply with scoper (#619)
Centralizes as many of the exchange scope funcs as
possible into scopes.go.  Ensures exchangeScopes comply
with the scoper interface.  Reshuffles some test helper
code in selectors to a centralized file.
2022-08-19 22:46:55 +00:00
Keepers
ade400126d
final cleanup for new PR (#609) 2022-08-19 20:40:56 +00:00
ashmrtn
35713acffb
Regression test for kopia sharing local config (#620)
Ensure that kopia is not sharing the local config and that different
instances of kopia reconnect to the proper remote repo.
2022-08-19 20:33:07 +00:00
Danny
4f5bb2b995
Exchange event info (#569)
Exchange Info feature added and verified with `mockdata` to ensure correctness
2022-08-19 13:43:26 -06:00
Keepers
daa939f5f8
fullfill categorizer for exchange selectors (#602) 2022-08-19 17:43:53 +00:00