11 Commits

Author SHA1 Message Date
Keepers
5980c4dca0
log observed messages (#2073)
## Description

add logging to the observe package, assume that
every instance where a message is observed, it
also gets logged.

Merger may want to wait until logging to a file is the standard behavior, else the terminal might get messy/buggy.

## Does this PR need a docs update or release note?

- [x]  No 

## Type of change

- [x] 🧹 Tech Debt/Cleanup

## Issue(s)

* closes #2061

## Test Plan

- [x] 💪 Manual
2023-01-13 19:34:01 +00:00
Keepers
aacb013b60
add doNotMerge func to collections (#1919)
## Description

Adds a new func to the data.Collection iface:
DoNotMergeItems() tells kopia to skip the
retention of items from prior snapshots when
generating the new snapshot for this collection.
A primary use case for this flag is when a delta
token expires, preventing an incremental lookup
and forcing gc to re-discover all items in the
container.

## Does this PR need a docs update or release note?

- [x]  No 

## Type of change

- [x] 🌻 Feature

## Issue(s)

* #1914

## Test Plan

- [x]  Unit test
2022-12-22 21:27:11 +00:00
Danny
fe27fea2ae
GC: Backup: SharePoint List Integration with Tests (#1788)
## Description
Framework for the SharePoint backup workflow. 

### Special Instructions for Review
Ensure that the information used to build `path.Path` are in line with future PRs (@ashmrtn )
## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature

## Issue(s)

* closes #1403
* closes #1474<issue>
* closes #1795

## Test Plan
- SharePoint Integration checked:
  - `src/internal/connector/data_collections_test.go`
  - `TestSharePointDataCollection()`
- SharePoint List basics checked:  
  - `src/internal/connector/sharepoint/collection_test.go`
- SharePoint Operational Backup:
  - `src/internal/operations/backup_test.go`
  - `TestBackup_Run_sharePoint() ` verified as operational during testing
- [x]  Unit test
2022-12-20 21:30:20 +00:00
Keepers
c8ad2e03ce
produce coll state and item deletion (#1815)
## Description

Fulfills the data collection interfaces to provide item deleted
flags and collection state enums.

A note on the additional data consumption: I'm making an assumption
that the @removed property will appear in that map.  This is awaiting
testing for verification, which I'll get into as a follow-up step.

## Does this PR need a docs update or release note?

- [x]  No 

## Type of change

- [x] 🌻 Feature

## Issue(s)

* #1727

## Test Plan

- [x]  In a follow-up PR
2022-12-19 20:16:51 +00:00
Keepers
fc5f42545f
rename graph.Service to Servicer (#1787)
## Description

`graph.Service -> graph.Servicer`,  no other changes.

More compliant with golang naming standards,
and will allow us to eventually migrate the
Service struct out of connector and into graph.

## Type of change

- [x] 🐹 Trivial/Minor

## Issue(s)

* #1725
2022-12-13 17:37:03 +00:00
ashmrtn
09c48c1ec9
Expand Stream and Collection interfaces for delta token-based incrementals (#1710)
## Description

Add functions to Collection and Stream interfaces that allow for getting
more information about the difference between the previous backup and
the currently in-progress one. These will allow delta token-based
incremental backups to determine how the state has evolved.

Current code does not use these functions and return values for them
are "default" values that should result in full backups even if
KopiaWrapper is updated to start checking the values and GraphConnector
still pulls all items

These functions are not used during restore and can return "default"
values

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

* #1700 

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-12-07 22:20:59 +00:00
ashmrtn
8fb55ec886
Implement ModTime interface in service streams (#1670)
## Description

Add ModTime to Exchange, OneDrive and SharePoint list stream items. This enables kopia-assisted incrementals for those items. Backup details still contains a complete set of information for all items in the backup regardless of if kopia uploaded data for the item or not.

Kopia-assisted incrementals does come with some caveats though. If changes are made to an item in M365 and that change does not cause the modified time reported by M365 to update, then the change will not be backed up. Currently, only marking an email as read/unread is known to hit this edge case.

This patch does not lazily fetch data from Graph API. This means that kopia may upload less data, but the same amount of data will still be pulled from Graph

## Type of change

- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

* closes #622 

## Test Plan

<!-- How will this be tested prior to merging.-->
- [x] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-12-05 18:01:38 +00:00
Danny
e767bb0b77
Backup: SharePoint: List content retrieval (#1633)
## Description
Logic for List retrieval from M365 back store. Keeping separate from Collection creation logic as there are a lot of calls to retrieve all supporting relationship data from the backstore.  
Adds collection to 
## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature


## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
*closes  #1475<issue>

## Test Plan

- [x]  Unit test
2022-12-02 19:31:20 +00:00
Keepers
11371f6e94
add sharepoint to path (#1465)
Adds the sharepoint service to /pkg/path.  Currently uses the
"Files" category for its category type, which is just a placeholder
for kicking off development.

Additionally, uncomments selector tests that were dependent
upon the path service declaration.
2022-11-14 15:35:57 -07:00
Danny
765fd6222b
GC: Backup: Sharepoint: Collection Implementation (#1477)
## Description
Creates test package for SharePoint List Collection. Verifies Item and ListInfo for collection

<!-- Insert PR description-->

## Type of change

- [x] 🌻 Feature

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* related to #1474<issue>

## Test Plan

- [x]  Unit testt: E2E
2022-11-14 17:51:44 +00:00
Danny
c4632cb152
GC SharePoint Collection Framework (#1444)
## Description
Boilerplate for SharePoint collection 

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
2022-11-04 14:11:10 +00:00