Files
zed/crates
Oleksiy Syvokon 3ae0ba85b5 Pass routing headers to ChatGPT subscription API (#62556)
## Objective

Improve cache hit rate of LLM requests sent to ChatGPT subscription API.

## Solution

Set headers that help route request to the correct servers. This is
ChatGPT specific; plain OpenAI API doesn't require it.

## Testing

I run a benchmark which imitates a thread that makes 20 sequential tool
calls. I tried different cache route strategies. Results are below:


| Configuration                                   | Hit rate |
|-------------------------------------------------|---------:|
| No identity headers, no cache key               | 22.2%    |
| `thread-id` only, no cache key                  | 44.4%    |
| `thread-id` + matching cache key                | 42.1%    |
| Cache key only                                  | 53.6%    |
| Turn state + cache key                          | 47.4%    |
| `session-id` + matching cache key               | 100.0%   |
| `session-id` + `thread-id` + matching cache key | 97.4%    |



---

Release Notes:

- Improved ChatGPT subscription caching
2026-08-13 01:04:16 +00:00
..
2025-10-17 18:58:14 +00:00
2026-08-09 22:29:52 +00:00
2026-08-09 22:29:52 +00:00
2026-08-09 22:29:52 +00:00
2026-08-09 22:29:52 +00:00
2026-04-29 10:27:47 +00:00
2026-08-09 22:29:52 +00:00
2026-08-09 22:29:52 +00:00
2026-08-09 22:29:52 +00:00
2026-05-04 06:30:47 +00:00
2025-10-17 18:58:14 +00:00
2026-08-09 22:29:52 +00:00
2026-08-09 22:29:52 +00:00
2026-08-07 18:51:36 +00:00
2026-08-09 22:29:52 +00:00
2026-07-21 07:58:16 +00:00
2026-08-09 22:29:52 +00:00
2026-08-09 22:29:52 +00:00
2026-05-27 10:31:53 +00:00
2026-08-09 22:29:52 +00:00
2026-08-09 22:29:52 +00:00
2026-04-29 10:27:47 +00:00
2026-08-09 22:29:52 +00:00
2026-08-09 22:29:52 +00:00
2025-11-07 11:36:05 +00:00
2026-05-27 10:31:53 +00:00
2025-10-17 18:58:14 +00:00
2026-08-09 22:29:52 +00:00
2026-08-12 14:59:40 +00:00
2026-08-09 22:29:52 +00:00