Fix rate limiter holding permits during tool execution (#47494)
The rate limiter's semaphore guard was being held for the entire duration of a turn, including during tool execution. This caused deadlocks when subagents tried to acquire permits while parent requests were waiting for them to complete. ## The Problem In `run_turn_internal`, the stream (which contains the `RateLimitGuard` holding the semaphore permit) was kept alive throughout the entire loop iteration - including during **tool execution**: 1. Parent request acquires permit 2. Parent starts streaming, consumes response 3. Parent starts executing tools (subagents) 4. **Stream/guard still held** while tools execute 5. Subagents try to acquire permits → blocked because parent still holds permit 6. Deadlock if all permits are held by parents waiting for subagent children ## The Fix Two changes were made: 1. **Drop the stream early**: Added an explicit `drop(events)` after the stream is fully consumed but before tool execution begins. This releases the rate limit permit so subagents can acquire it. 2. **Removed the `bypass_rate_limit` workaround**: Since the root cause is now fixed, the bypass mechanism is no longer needed. Note: no release notes because subagents are still feature-flagged, and this rate limiting change isn't actually observable without them. Release Notes: - N/A
This commit is contained in:
1 parent
9ef825d753
commit
29cf14ed2f
24 files changed
+279
-603
No files matched your search
@@ -544,7 +544,6 @@ impl CodegenAlternative {
|
||||
temperature,
|
||||
messages,
|
||||
thinking_allowed: false,
|
||||
bypass_rate_limit: false,
|
||||
}
|
||||
}))
|
||||
}
|
||||
@@ -623,7 +622,6 @@ impl CodegenAlternative {
|
||||
temperature,
|
||||
messages: vec![request_message],
|
||||
thinking_allowed: false,
|
||||
bypass_rate_limit: false,
|
||||
}
|
||||
}))
|
||||
}
|
||||
|
||||
Reference in new issue
Block a user