larryh/corso
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
corso/CHANGELOG.md
Abin Simon a4147f5498
Speedup OneDrive backup by pulling urls in delta API (#1842) 
## Description

This brings in another huge improvement in the backup speed dropping from ~15m to ~1.5m for an account with ~5000 files. This one also drastically reduces the number of requests we have to make for the same account from 5505 to just ~55. This also means that we don't get throttled anymore and we can easily run multiple backup jobs in parallel before we hit the 1024 limit.

~The code works as of now, but I have to double check the numbers as well as fix an issue with us opening too many files and causing program to crash with 'too many open files' when we bump up the numbers. With the current numbers, it works, but I wanna double check and optimize them. Plus some code cleanup.~

## Does this PR need a docs update or release note?

- [x] ✅ Yes, it's included
- [ ] 🕐 Yes, but in a later PR
- [ ] ⛔ No 

## Type of change

<!--- Please check the type of change your PR introduces: --->
- [ ] 🌻 Feature
- [ ] 🐛 Bugfix
- [ ] 🗺️ Documentation
- [ ] 🤖 Test
- [ ] 💻 CI/Deployment
- [x] 🐹 Trivial/Minor

## Issue(s)

<!-- Can reference multiple issues. Use one of the following "magic words" - "closes, fixes" to auto-close the Github issue. -->
* #<issue>

## Test Plan

<!-- How will this be tested prior to merging.-->
- [ ] 💪 Manual
- [x] ⚡ Unit test
- [x] 💚 E2E
2022-12-20 09:34:28 +00:00

3.7 KiB
Raw Blame History

Changelog

All notable changes to this project will be documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

Unreleased

Changed

  • Breaking Change: Changed how backup details are stored in the repository to improve memory usage (#1735) from vkamra
  • Improve OneDrive backup speed (#1842) from meain

[v0.0.3] (alpha) - 2022-12-05

Added

  • Display backup size in backup list command (#1648) from meain
  • Improve OneDrive backup performance (#1607) from meain
  • Improve Exchange backup performance (#1608) from meain
  • Add flag to retain all progress bars (#1582) from ryanfkeepers
  • Fix resource owner display on backup list (#1580) from ryanfkeepers

Changed

  • Improve logging (#1642) from ryanfkeepers
  • Generate separate backup for each resource owner (#1609) from ashmrtn
  • Print version info to stdout instead of stderr (#1503) from meain

v0.0.2 (alpha) - 2022-11-14

Added

  • Added AWS X-Ray support for better observability (#1111) from ryanfkeepers
  • Allow disabling TLS and TLS verification (#1415) from vkamra
  • Add filtering based on path prefix/contains (#1224) from ryanfkeepers
  • Add info about doc owner for OneDrive files (#1366) from meain
  • Add end time for Exchange events from (#1366) meain

Changed

  • Export RepoAlreadyExists error for sdk users (#1136)from ryanfkeepers
  • RudderStack logger now respects corso logger settings (#1324) from ryanfkeepers

v0.0.1 (alpha) - 2022-10-24

New features

  • Supported M365 Services

    • Exchange - email, events, contacts (RM-8)
    • OneDrive - files (RM-12)

  • Backup workflows

    • Create a full backup (RM-19)
    • Create a backup for a specific service and all or some data types (RM-19)
    • Create a backup for all or a specific user (RM-20)
    • Delete a backup manually (RM-24)

  • Restore workflows

    • List, filter, and view backup content details (RM-23)
    • Restore one or more items or folders from backup (RM-28, RM-29)
    • Non-destructive restore to a new folder/calendar in the same account (RM-30)

  • Backup storage

    • Zero knowledge encrypted backups with user conrolled passphrase (RM-6)
    • Initialize and connect to an S3-compliant backup repository (RM-5)

  • Miscellaneous

    • Optional usage statistics reporting (RM-35)