ashmrtn bcde15689f
Smarter item load ordering during ProduceRestoreCollections (#3294)
Optimize loading items for restore a little bit by first grouping items
by directory, loading the directory once, and then loading all items
from the loaded directory. This brings item loading on my local machine
(communicating with remote S3) down to ~1.5min/1k items

Future improvements could lazily load items as they're returned in the
Items() call of each collection but doing so would change the semantics
of ProduceRestoreCollections() (specifically item not found errors would
be returned during Items() instead of during
ProduceRestoreCollections())

The kopia data collection has also been updated to hold onto a
reference to the folder it corresponds to. This folder reference is
used to service Fetch() calls

---

#### Does this PR need a docs update or release note?

- [x]  Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ]  No

#### Type of change

- [x] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Supportability/Tests
- [ ] 💻 CI/Deployment
- [ ] 🧹 Tech Debt/Cleanup

#### Issue(s)

* #3293

#### Test Plan

- [ ] 💪 Manual
- [x]  Unit test
- [x] 💚 E2E
2023-05-04 21:02:34 +00:00
..