larryh/corso
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

move default rate limit to per-second (#3637)

Browse Source 
The current rate limit default maximizes the number of calls we can make in the first 10 minutes, but the larger initial bucket size means we have to drop the per-second token burst.  As a result, any process longer than 10 minutes is going to see unnecessary rate limiting by ~100 calls per minute.

This change swaps the large initial bucket for a smaller one, and a larger token burst to increase throughput across long runtimes.

---

#### Does this PR need a docs update or release note?

- [x] ⛔ No

#### Type of change

- [x] 🧹 Tech Debt/Cleanup

#### Test Plan

- [x] ⚡ Unit test
- [x] 💚 E2E
This commit is contained in:
Keepers 2023-06-22 18:25:16 -06:00 committed by GitHub
parent b11d80599b
commit 1bfa195b33
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 6 additions and 10 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

16
src/internal/m365/graph/concurrency_middleware.go
View File

@ -73,17 +73,13 @@ func (cl *concurrencyLimiter) Intercept(
const ( const (
// Default goal is to keep calls below the 10k-per-10-minute threshold. // Default goal is to keep calls below the 10k-per-10-minute threshold.
// 14 tokens every second nets 840 per minute. That's 8400 every 10 minutes, // 16 tokens every second nets 960 per minute. That's 9600 every 10 minutes,
// which is a bit below the mark. // which is a bit below the mark.
// But suppose we have a minute-long dry spell followed by a 10 minute tsunami. // If the bucket is full, we can push out 200 calls immediately, which brings
// We'll have built up 750 tokens in reserve, so the first 750 calls go through // the total in the first 10 minutes to 9800. We can toe that line if we want,
// immediately. Over the next 10 minutes, we'll partition out the other calls // but doing so risks timeouts. It's better to give the limits breathing room.
// at a rate of 840-per-minute, ending at a total of 9150. Theoretically, if defaultPerSecond = 16 // 16 * 60 * 10 = 9600
// the volume keeps up after that, we'll always stay between 8400 and 9150 out defaultMaxCap = 200 // real cap is 10k-per-10-minutes
// of 10k. Worst case scenario, we have an extra minute of padding to allow
// up to 9990.
defaultPerSecond = 14 // 14 * 60 = 840
defaultMaxCap = 750 // real cap is 10k-per-10-minutes
// since drive runs on a per-minute, rather than per-10-minute bucket, we have // since drive runs on a per-minute, rather than per-10-minute bucket, we have
// to keep the max cap equal to the per-second cap. A large maxCap pool (say, // to keep the max cap equal to the per-second cap. A large maxCap pool (say,
// 1200, similar to the per-minute cap) would allow us to make a flood of 2400 // 1200, similar to the per-minute cap) would allow us to make a flood of 2400