## Description
The MS Graph's adapter contains a serialization writer within it. Create a framework to use this writer rather than creating the object within serialization workflows. The writer library was used separately when the package still had many bugs and the most current version was required. This does not change any of the writer instances within the package. Will change out the writer instances in follow-up PRs.
<!-- Insert PR description-->
## Does this PR need a docs update or release note?
- [x] ⛔ No
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
*related to #2031 <issue>
## Test Plan
- [x] ⚡ Unit test
## Description
Fulfills the data collection interfaces to provide item deleted
flags and collection state enums.
A note on the additional data consumption: I'm making an assumption
that the @removed property will appear in that map. This is awaiting
testing for verification, which I'll get into as a follow-up step.
## Does this PR need a docs update or release note?
- [x] ⛔ No
## Type of change
- [x] 🌻 Feature
## Issue(s)
* #1727
## Test Plan
- [x] ❓ In a follow-up PR
## Description
These two structures are effectively identical.
This refactor deferrs to the graph version since
services may still wrap that for more specific
behavior as needed.
## Does this PR need a docs update or release note?
- [x] ⛔ No
## Type of change
- [x] 🐹 Trivial/Minor
## Issue(s)
* #1727
## Test Plan
- [x] ⚡ Unit test
- [x] 💚 E2E
## Description
Adds an additional metadata collection: a folder
id to path string mapping. This collection is
created on backup, and retrieved along with
the delta metadata on the next backup, but is
not yet parsed or utilzed downstream.
## Type of change
- [x] 🌻 Feature
## Issue(s)
* #1726
## Test Plan
- [x] ⚡ Unit test
- [x] 💚 E2E
## Description
Relocates the graphService struct to graph as
the Service struct. Replaces GC's embedded
graphService with a graph.Servicer reference.
## Type of change
- [x] 🐹 Trivial/Minor
## Issue(s)
* #1725
## Test Plan
- [x] ⚡ Unit test
- [x] 💚 E2E
## Description
Configuration and attenion to the graphService
failFast is haphazard and has shared ownership.
This change removes that property from the
service, along with the ErrPolicy func, in favor of passing around a control.Options struct.
## Type of change
- [x] 🐹 Trivial/Minor
## Issue(s)
* #1725
* #302
## Test Plan
- [x] ⚡ Unit test
## Description
`graph.Service -> graph.Servicer`, no other changes.
More compliant with golang naming standards,
and will allow us to eventually migrate the
Service struct out of connector and into graph.
## Type of change
- [x] 🐹 Trivial/Minor
## Issue(s)
* #1725
## Description
In a backup operation, begins the operation by
retrieving all backup manifests and metadata
from prior operations.
## Type of change
- [x] 🌻 Feature
## Issue(s)
* #1725
## Test Plan
- [x] ⚡ Unit test
- [x] 💚 E2E
## Description
Begin persisting Exchange delta tokens for data categories that support delta queries. Tokens are stored in a single file as a `map[M365 container ID]token` where both the container ID and token are of type `string`. The file is located in the kopia snapshot that has all the other backup data at the path `tenant-id/{service}Metadata/user/category/delta`. No information about the delta token file is stored in backup details.
## Type of change
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor
## Issue(s)
* closes#1685
## Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
## Description
Since scopes are service specific, we cannot
easily house them within the graph QueryParam
struct, unless we bloat the struct with all types.
Alternatively, we could add a generic "scope"
with parsers, much like the Selector itself. But
really, the most simple solution is to only pass
the scope within the tree of service funcs that
use it.
## Type of change
- [x] 🐹 Refactor
## Issue(s)
* #1506
## Test Plan
- [x] ⚡ Unit test
## Description
`graph.ContainerResolver` has the capacity to keep the directory structure of the m365 objects that are helpful within a user's account. Leveraging this abstraction allows for a better flow of data from M365 into storage.
## Type of change
- [x] ⚡ : Optimization
## Issue(s)
* closes #1125<issue>
* closes#1122
## Test Plan
- [x] ⚡ Unit test
## Description
Feature to add the folder hierarchy for folders when restored.
This required an overhaul of the `graph.ContainerResolver` interfaces:
- MailFolderCache
- ContactFolderCache
- ~EventFolderCache (placed in a separate PR)~ https://github.com/alcionai/corso/pull/1101
Restore Pipeline changed to separate the caching / container creation process from the rest of the restore pipeline.
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
## Issue(s)
* closes#1046
* #1004
* closes#1091
* closes#1098
* closes#1097
* closes#1096
* closes#1095
* closes#991
* closes#895
* closes#798
## Test Plan
- [x] ⚡ Unit test
* Move interfaces to common location
Upcoming PRs are using these interfaces across packages. Move them to a
common location so multiple packages can use them without import cycles
etc.
* Allow adding newly created folders to the cache (#1131)
* New function to add folders to cache
Allow adding new folders to the cache. Automatically cache the paths for
the new folders. Also add the new function to the interface.
* Reuse the AddToCache function during population
* Wire up ability to back up a single subfolder of mail (#1132)
* Expand cache to return items in it
Required to allow matching an item's path to a selector as the selector
will not provide which paths it matches on easily.
* Function to get collections from cached folders
Returned collections match any matchers given for the folders
* Thread resolver through iterator functions
Allow the folder resolver to be used in all iterator functions. The
resolver will be tied to the current category and user.
* Choose between using resolver and making queries
Allow either using the resolver to get folders with matching names or
using queries to get them.
* Wire up resolver at entry point
Create a resolver instance for each user/category of data being
backedup.
* Preparation for changing how mail enumeration is done (#1157)
* Step towards redoing mail fetching
Pull out old way to get data into a new function and setup some helper
functions etc.
* Switch to pulling mail items folder by folder (#1158)
* Function to pull mail items given collections
Given a set of collections and IDs for those collections pull the mail
items for each collection.
* Create helper function to fetch mail
New helper function to fetch mail items. This goes through each folder
and gets the items for them individually.
* Wire up new way to fetch mail
Leaves fetch logic for other data types undisturbed.
* Tests for new mail fetching logic
Remove tests that were previously in iterators_test.go and move them to
graph_connector_test.go. These tests only had to do with mail logic.
Tests that handled all data types in iterators_test.go have been updated
to skip mail now.
## Description
Code changed to support caching on the default folder of `exchange.Mail` as well as an independent node within the Inbox.
<!-- Insert PR description-->
## Type of change
- [x] 🌻 Feature
## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* related to Issue #1004<issue>
## Test Plan
- [x] ⚡ Unit test
## Description
Moves the `path` package to the `pkg` package so other code outside of Corso can use it if they need it
## Type of change
<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [x] 🐹 Trivial/Minor
## Issue(s)
* closes#908
## Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [x] 💚 E2E
swaps the corso go module from github.com/
alcionai/corso to github.com/alcionai/corso/src
to align with the location of the go.mod and
go.sum files inside the repo.
All other changes in the repository update the
package imports to the new module path.
## Description
Creates and uses a `ContainerResolver` interface to fetch container paths for items of different categories (when other resolvers are implemented). If the resolver is not available or fails to resolve a folder, defaults to the old implementation of using the folder's ID as its path.
## Type of change
Please check the type of change your PR introduces:
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 🐹 Trivial/Minor
## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #456
## Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [ ] 💚 E2E
## Description
Group like parameters to graph functions so SonarCloud is happier
## Type of change
Please check the type of change your PR introduces:
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [x] 🐹 Trivial/Minor
## Issue(s)
<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
on the path to:
* #456
## Test Plan
<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [x] 💚 E2E
Functions and interfaces moved to interact properly with connector package. Fields are no longer exported until the required functions are moved to the exchange package.
