446ccfe491
OneNote files >2GB cannot be downloaded via the graph API. This change will gracefully skip them and prevent the backup from erroring out. A OneNote file is represented by a Package(folder) which contains two file within it, `<file>.one` and `<file>.onetoc2`. From what I can tell the `onetoc2` file is the metadata file and the `one` file is the one that contains data and can get big. **The current implementation has a limitation(or feature depending on how you see it) in that we will end up doing a partial backup with just the `onetoc2` file but avoiding the big `one` file if that is big.** I did not try to change the behaviour as even currently if for somehow we ended up with an error like 404 when trying to backup one of these files, we do skip it. I've added a comment in for now, let me know if we should go back and revisit the behaviour. *The main issue is the fact that we only hit this problem once it reaches kopia and removing it at that point is harder.* --- #### Does this PR need a docs update or release note? - [x] ✅ Yes, it's included - [ ] 🕐 Yes, but in a later PR - [ ] ⛔ No #### Type of change <!--- Please check the type of change your PR introduces: ---> - [ ] 🌻 Feature - [x] 🐛 Bugfix - [ ] 🗺️ Documentation - [ ] 🤖 Supportability/Tests - [ ] 💻 CI/Deployment - [ ] 🧹 Tech Debt/Cleanup #### Issue(s) <!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. --> * fixes https://github.com/alcionai/corso/issues/2910 #### Test Plan <!-- How will this be tested prior to merging.--> - [ ] 💪 Manual - [x] ⚡ Unit test - [ ] 💚 E2E
38 lines
1.0 KiB
Go
38 lines
1.0 KiB
Go
package stats
|
|
|
|
import (
|
|
"sync/atomic"
|
|
"time"
|
|
)
|
|
|
|
// ReadWrites tracks the total count of reads and writes. ItemsRead
|
|
// and ItemsWritten counts are assumed to be successful reads.
|
|
type ReadWrites struct {
|
|
BytesRead int64 `json:"bytesRead,omitempty"`
|
|
BytesUploaded int64 `json:"bytesUploaded,omitempty"`
|
|
ItemsRead int `json:"itemsRead,omitempty"`
|
|
ItemsWritten int `json:"itemsWritten,omitempty"`
|
|
ResourceOwners int `json:"resourceOwners,omitempty"`
|
|
}
|
|
|
|
// StartAndEndTime tracks a paired starting time and ending time.
|
|
type StartAndEndTime struct {
|
|
StartedAt time.Time `json:"startedAt"`
|
|
CompletedAt time.Time `json:"completedAt"`
|
|
}
|
|
|
|
type ByteCounter struct {
|
|
NumBytes int64
|
|
}
|
|
|
|
func (bc *ByteCounter) Count(i int64) {
|
|
atomic.AddInt64(&bc.NumBytes, i)
|
|
}
|
|
|
|
type SkippedCounts struct {
|
|
TotalSkippedItems int `json:"totalSkippedItems"`
|
|
SkippedMalware int `json:"skippedMalware"`
|
|
SkippedNotFound int `json:"skippedNotFound"`
|
|
SkippedInvalidOneNoteFile int `json:"skippedInvalidOneNoteFile"`
|
|
}
|