15 Commits

Author SHA1 Message Date
ashmrtn
99edf7d5b0
Add and populate mod time for BaseModel (#4065)
Get the last time a model was modified and return it in BaseModel. This will help with discovering what items can be garbage collected during incomplete backup cleanup as we don't want to accidentally delete in-flight backups.

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #3217

#### Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2023-08-21 23:48:57 +00:00
Keepers
9abd9d4f96
remove all uses of iota (#4046)
I've needed to catch gotchas that arise from contributors adding a value in the middle of an iota list, not to mention have dealt with prior bugs that happened the same way, now too many times to feel safe about its usage.

This PR removes the use of iota from all const declarations. The intent is to not allow the use of iota within the codebase.

---

#### Does this PR need a docs update or release note?

- [x]  No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #3993

#### Test Plan

- [x]  Unit test
- [x] 💚 E2E
2023-08-18 22:45:21 +00:00
ashmrtn
4ebb2d3bfb
Common structs/mocks/consts for assist backups (#3985)
Pull into separate PR so we can start merging things separately.

`BackupBases` sourced from branch `backup_bases_changes`
`model.go` sourced from branch `partial_bup`
mock backup bases sourced from branch `backup-details-merge`

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Test Plan

- [ ] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-08-08 18:06:15 +00:00
Keepers
62daf10213
migrate onedrive using prefix collection (#3122)
#### Does this PR need a docs update or release note?

- [x] 🕐 Yes, but in a later PR

#### Type of change

- [x] 🌻 Feature

#### Issue(s)

* #2825

#### Test Plan

- [x] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2023-04-24 17:36:10 +00:00
ashmrtn
1aa315412a
Add more logging to middleware and incremental backup operation logic (#2899)
Ensures:
* context clues are added to logged messages
* kv-pairs added with `log.With` are added to logged messages
* add log messages for how many details items were sourced from a base snapshot during merging
* ensure log messages for base selection have all the info they should

---

#### Does this PR need a docs update or release note?

- [ ]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [x]  No

#### Type of change

- [ ] 🌻 Feature
- [x] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Test Plan

- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2023-03-21 22:33:19 +00:00
Keepers
565c33af4e
generate a unique repository id on init (#1716)
## Description

generates a unique repository ID on repo init, and retrieves it again on repo connect.  This allows
the event bus (and any other tracker) to id a repo independent of other context.

I've been unable to find any good way to retrieve a unique repository ID from kopia.  This seems like
the next quickest solution.

## Type of change

- [x] 🐛 Bugfix

## Issue(s)

* #1602

## Test Plan

- [x] 💪 Manual
2022-12-12 22:14:37 +00:00
Vaibhav Kamra
93a7ff27e3
Store backup details in a separate snapshot (#1755)
## Description

This PR uses a separate snapshot to store backup details instead of a Corso model (i.e. a kopia manifest).

Introduces a `StreamStore` that can be leveraged to store larger metadata objects. We can also leverage this
for incrementals or restartable backups going forward.

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #1735 

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2022-12-10 00:47:49 +00:00
ashmrtn
2524523298
Add a comment for updating Schema constants (#1759)
## Description

Note what things need to stay the same and what things can change when updating schema constant values. This is required because some information about the constants are persisted in the model store.

## Type of change

- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [x] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [x] 🐹 Trivial/Minor

## Issue(s)

* closes #1757 

## Test Plan

- [x] 💪 Manual
- [ ]  Unit test
- [ ] 💚 E2E
2022-12-09 18:00:45 +00:00
Keepers
8ddbc16077
Add GetID() to model.BaseModel (#1336)
## Type of change

- [x] 🐹 Trivial/Minor

## Issue(s)

* #1334

## Test Plan

- [x]  Unit test
2022-10-25 23:46:49 +00:00
Keepers
286a74e819
persist service tag in backup model, list by service (#1233)
## Description

Adds a tag describing the service within the backup model and the backup details model.  Adds a filter builder pattern to the store wrapper for filtering results according to certain tags.  Finally, adds a filter builder to specify the service type when listing backups.

## Type of change

- [x] 🐛 Bugfix

## Issue(s)

* #1226

## Test Plan

- [x] 💪 Manual
- [x]  Unit test
2022-10-21 01:04:54 +00:00
ashmrtn
b9e33901f4
Add versions to models (#1064)
## Description

Currently only the ModelStore populates and checks versions.

Some high-level points:
* fail any sort of get (just metadata or full model) if there's a version mismatch
* update operations automatically set things to the current version
* versions are stored as tags so there's some int->string (and vice versa) munging
* versions stored as tags so they can be repopulated even if only the metadata (BaseModel) for a model is pulled. This is done mostly for consistency

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [ ] 🐹 Trivial/Minor

## Issue(s)

* closes #284 

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x]  Unit test
- [ ] 💚 E2E
2022-10-06 00:46:49 +00:00
Keepers
e35acb79ca
rename stableID to ID (#425)
embedding a baseModel means that objects like backup
have a backup.StableID, but no backup.ID nor backup.
backupID.  This could be confusing for users.  This change
swaps the value and type names, so that baseModels
include an ID value of type StableID.
2022-07-27 20:54:58 +00:00
Keepers
6c22d5c0ce
adds store package for wrapping model_store (#346)
* adds store package for wrapping model_store

Introduces the pkg/store package, which contains funcs
for wrapping the model_store with common requests.
This package choice was made for its combination
of being in an accessible place, centralizing functionality
and not introducing circular dependencies.
2022-07-19 11:54:53 -06:00
ashmrtn
5ec6d4f286
Base model revision (#263)
* Update model interface and base struct

* Update code and tests for new Model interface
2022-07-01 14:17:15 -07:00
ashmrtn
99691f46d5
ModelStore Put/Get implmentations (#261)
* Simple Get and Put implementations

Get implementation is currently the one that uses the kopia ID of the
model/manifest.

* Basic tests for ModelStore Get/Put
2022-07-01 11:07:01 -07:00