This PR adds a lifetime spending limit on LLM usage.
Exceeding this limit will prevent further use of the Zed LLM provider.
Currently the cap is $1,000.
Release Notes:
- N/A
This PR makes it so users are upserted by their `github_user_id` instead
of by their `github_login`.
The `github_user_id` is a stable identifier that does not change, while
the `github_login` can change.
In practice we were already using
`get_or_create_user_by_github_account`, which already checks for an
existing user with a `github_user_id` first, so this change doesn't
result in a change in behavior.
This change is primarily for correctness in the event that `create_user`
is called directly, as we want to be upserting by the stable identifier.
Release Notes:
- N/A
This PR makes the `github_user_id` column on the `users` table required
and replaces the index with a unique index.
I have gone through and ensured that all users have a unique
`github_user_id` in the staging and production databases.
Release Notes:
- N/A
This PR adds additional reporting of the active user counts as separate
logs.
We were already reporting these on individual rate limit events/logs,
but it seems like something that would be good to report on independent
of user activity.
Release Notes:
- N/A
This PR fixes an issue where the active user count spanned individual
models.
We now track the active user counts on a per-model basis.
Release Notes:
- N/A
This PR adds traces for when users hit LLM rate limits.
We were already emitting telemetry events for these to Clickhouse, but
it will be handy to have them available in Axiom as well.
Release Notes:
- N/A
This PR adds two Postgrest containers—one for the app database and one
for the LLM database—to the Docker Compose cluster.
Also fixed an issue where `postgres_app.conf` and `postgres_llm.conf`
had been switched.
Release Notes:
- N/A
This PR adds the `is_staff` field to the `upstream rate limit` spans.
Since we use different API keys for staff vs non-staff, it will be
useful to break down the rate limits accordingly.
Release Notes:
- N/A
This PR fixes the writing of LLM rate limit events to Clickhouse.
We had a table in the table name: `llm_rate_limits` instead of
`llm_rate_limit_events`.
I also extracted a helper function to write to Clickhouse so we can use
it anywhere we need to.
Release Notes:
- N/A
This PR reworks how we do checks for model names in the LLM service.
We now normalize the model names using the models defined in the
database.
Release Notes:
- N/A
This PR updates the LLM service to include the GitHub login on its
spans.
We need to pass this information through on the LLM token, so it will
temporarily be `None` until this change is deployed and new tokens have
been issued.
Release Notes:
- N/A
- db deadlock in GetLlmToken for non-staff users
- typo in allowed model name for non-staff users
Release Notes:
- N/A
---------
Co-authored-by: Marshall <marshall@zed.dev>
Co-authored-by: Joseph <joseph@zed.dev>
This PR adds the ability to revoke access tokens for the LLM service.
There is a new `revoked_access_tokens` table that contains the
identifiers (`jti`) of revoked access tokens.
To revoke an access token, insert a record into this table:
```sql
insert into revoked_access_tokens (jti) values ('1e887b9e-37f5-49e8-8feb-3274e5a86b67');
```
We now attach the `jti` as `authn.jti` to the tracing spans so that we
can associate an access token with a given request to the LLM service.
Release Notes:
- N/A
This PR adds feature-flagged access to the LLM service.
We've repurposed the `language-models` feature flag to be used for
providing access to Claude 3.5 Sonnet through the Zed provider.
The remaining RPC endpoints that were previously behind the
`language-models` feature flag are now behind a staff check.
We also put some Zed Pro related messaging behind a feature flag.
Release Notes:
- N/A
---------
Co-authored-by: Max <max@zed.dev>
This PR restricts usage of the LLM service to accounts older than 30
days.
We now store the GitHub user's `created_at` timestamp to check the
GitHub account age. If this is not set—which it won't be for existing
users—then we use the `created_at` timestamp in the Zed database.
Release Notes:
- N/A
---------
Co-authored-by: Max <max@zed.dev>
Now, when an anthropic request is invalid or anthropic's API is down,
we'll expose that to the user instead of just returning a generic 500.
Release Notes:
- N/A
Co-authored-by: Marshall <marshall@zed.dev>
This PR makes it so Zed staff can use a separate Anthropic API key for
the LLM service.
We also added an `is_staff` column to the `usages` table so that we can
exclude staff usage from the "active users" metrics that influence the
rate limits.
Release Notes:
- N/A
---------
Co-authored-by: Max <max@zed.dev>
This PR adds a check to the LLM API token issuance to ensure that we
only issue tokens to users that have accepted the terms of service.
Release Notes:
- N/A
This PR makes it so hitting upstream rate limits from Anthropic result
in an HTTP 429 response instead of an HTTP 500.
To do this we need to surface structured errors out of the `anthropic`
crate.
Release Notes:
- N/A
This adds the requirement for users to accept the terms of service the
first time they send a message with the Cloud provider.
Once this is out and in a nightly, we need to add the check to the
server side too, to authenticate access to the models.
Demo:
https://github.com/user-attachments/assets/0edebf74-8120-4fa2-b801-bb76f04e8a17
Release Notes:
- N/A
This PR makes it so staff members will be exempt from rate limiting by
the LLM service.
This is just a temporary measure until we can tweak the rate-limiting
heuristics.
Staff members are still subject to upstream LLM provider rate limits.
Release Notes:
- N/A
When Anthropic releases a new version of their models, Zed AI users
should always get access to the new version even when using an old
version of zed.
Co-Authored-By: Thorsten <thorsten@zed.dev>
Release Notes:
- N/A
Co-authored-by: Thorsten <thorsten@zed.dev>
This PR adjusts how we display the "mode" collab is running in on the
root endpoint.
It's minor, but it does make things a bit cleaner.
Release Notes:
- N/A
Marshall Bowers