This PR makes it so we always pass up an organization ID when creating
an LLM token.
We should have an organization ID in all cases.
Release Notes:
- N/A
Code had been assuming (erroneously, but understandably) that
LlmApiToken::acquire would give them a valid token.
This is not true, as those tokens expire and you must call refresh
explicitly.
Add some helpers to do the retry for you, and rename acquire to cached
to be
clearer about the intent.
Closes #ISSUE
Release Notes:
- Fixed some rare cases where API requests would fail with Unauthorized
- `language_model` no longer depends on provider-specific crates such as
`anthropic` and `open_ai` (inverted dependency)
- `language_model_core` was extracted from `language_model` which
contains the types for the provider-specific crates to convert to/from.
- `gpui::SharedString` has been extracted into its own crate (still
exposed by `gpui`), so `language_model_core` and provider API crates
don't have to depend on `gpui`.
- Removes some unnecessary `&'static str` | `SharedString` -> `String`
-> `SharedString` conversions across the codebase.
- Extracts the core logic of the cloud `LanguageModelProvider` into its
own crate with simpler dependencies.
Release Notes:
- N/A
---------
Co-authored-by: John Tur <john-tur@outlook.com>
This PR decouples `language_model`'s dependence on Zed-specific
implementation details. In particular
* `credentials_provider` is split into a generic `credentials_provider`
crate that provides a trait, and `zed_credentials_provider` that
implements the said trait for Zed-specific providers and has functions
that can populate a global state with them
* `zed_env_vars` is split into a generic `env_var` crate that provides
generic tooling for managing env vars, and `zed_env_vars` that contains
Zed-specific statics
* `client` is now dependent on `language_model` and not vice versa
Release Notes:
- N/A
The edit prediction, web search and completions endpoints in Cloud all
use tokens called LlmApiToken. These were independently created, cached,
and refreshed in three places: the cloud language model provider, the
edit prediction store, and the cloud web search provider. Each held its
own LlmApiToken instance, meaning three separate requests to get these
tokens at startup / login and three redundant refreshes whenever the
server signaled a token update was needed.
We already had a global singleton reacting to the refresh signals:
RefreshLlmTokenListener. It now holds a single LlmApiToken that all
three services use, performs the refresh itself, and emits
RefreshLlmTokenEvent only after the token is fresh. That event is used
by the language model provider to re-fetch models after a refresh. The
singleton is accessed only through `LlmApiToken::global()`.
I have tested this manually, and it token acquisition and usage appear
to be working fine.
Edit: I've tested it with a long running session, and refresh seems to
be working fine too.
Release Notes:
- N/A
---------
Co-authored-by: Marshall Bowers <git@maxdeviant.com>
This is already expected on the cloud side. This lets us know under
which organization the user is logged in when requesting an llm_api
token.
Closes CLO-337
Release Notes:
- N/A
This PR adds a new `needs_llm_token_refresh` helper method for checking
if the LLM token needs to be refreshed.
We were duplicating the check for the `x-zed-expired-token` header in a
number of spots, and it will be gaining an additional case soon.
Release Notes:
- N/A
We've been considering removing workspace-hack for a couple reasons:
- Lukas ran into a situation where its build script seemed to be causing
spurious rebuilds. This seems more likely to be a cargo bug than an
issue with workspace-hack itself (given that it has an empty build
script), but we don't necessarily want to take the time to hunt that
down right now.
- Marshall mentioned hakari interacts poorly with automated crate
updates (in our case provided by rennovate) because you'd need to have
`cargo hakari generate && cargo hakari manage-deps` after their changes
and we prefer to not have actions that make commits.
Currently removing workspace-hack causes our workspace to grow from
~1700 to ~2000 crates being built (depending on platform), which is
mainly a problem when you're building the whole workspace or running
tests across the the normal and remote binaries (which is where
feature-unification nets us the most sharing). It doesn't impact
incremental times noticeably when you're just iterating on `-p zed`, and
we'll hopefully get these savings back in the future when
rust-lang/cargo#14774 (which re-implements the functionality of hakari)
is finished.
Release Notes:
- N/A
This PR makes it so all LLM traffic is routed through `cloud.zed.dev`.
We're already routing `llm.zed.dev` to `cloud.zed.dev` on the server,
but we want to standardize on `cloud.zed.dev` moving forward.
Release Notes:
- N/A
This PR adds a new `zed-cloud` feature flag that can be used to send
traffic to `cloud.zed.dev` instead of `llm.zed.dev`.
This is just so Zed staff can test the new infrastructure. When we're
ready for prime-time we'll reroute traffic on the server.
Release Notes:
- N/A
* Updates to `zed_llm_client-0.8.5` which adds support for `retry_after`
when anthropic provides it.
* Distinguishes upstream provider errors and rate limits from errors
that originate from zed's servers
* Moves `LanguageModelCompletionError::BadInputJson` to
`LanguageModelCompletionEvent::ToolUseJsonParseError`. While arguably
this is an error case, the logic in thread is cleaner with this move.
There is also precedent for inclusion of errors in the event type -
`CompletionRequestStatus::Failed` is how cloud errors arrive.
* Updates `PROVIDER_ID` / `PROVIDER_NAME` constants to use proper types
instead of `&str`, since they can be constructed in a const fashion.
* Removes use of `CLIENT_SUPPORTS_EXA_WEB_SEARCH_PROVIDER_HEADER_NAME`
as the server no longer reads this header and just defaults to that
behavior.
Release notes for this is covered by #33275
Release Notes:
- N/A
---------
Co-authored-by: Richard Feldman <oss@rtfeldman.com>
Co-authored-by: Richard <richard@zed.dev>
https://github.com/zed-industries/zed/issues/30972 brought up another
case where our context is not enough to track the actual source of the
issue: we get a general top-level error without inner error.
The reason for this was `.ok_or_else(|| anyhow!("failed to read HEAD
SHA"))?; ` on the top level.
The PR finally reworks the way we use anyhow to reduce such issues (or
at least make it simpler to bubble them up later in a fix).
On top of that, uses a few more anyhow methods for better readability.
* `.ok_or_else(|| anyhow!("..."))`, `map_err` and other similar error
conversion/option reporting cases are replaced with `context` and
`with_context` calls
* in addition to that, various `anyhow!("failed to do ...")` are
stripped with `.context("Doing ...")` messages instead to remove the
parasitic `failed to` text
* `anyhow::ensure!` is used instead of `if ... { return Err(...); }`
calls
* `anyhow::bail!` is used instead of `return Err(anyhow!(...));`
Release Notes:
- N/A
This PR removes the individual URL overrides for the LLM service.
We initially had `ZED_PREDICT_EDITS_URL` to allow for directing traffic
to the LLM Worker back when there was still the split of the
Collab-based LLM Service and the Cloudflare-based LLM Worker.
But now that all of the LLM functionality has been moved into the
Worker, we can just direct all traffic there.
Release Notes:
- N/A
Some changes in the LanguageModelRegistry caused the web search tool not
to show up, because the `DefaultModelChanged` event is not emitted at
startup anymore.
Release Notes:
- agent: Fixed an issue where the web search tool would not be available
after starting Zed (only when using zed.dev as a provider).
Staff only for now. We'll work on making this usable for non zed.dev
users later
Release Notes:
- N/A
---------
Co-authored-by: Antonio Scandurra <me@as-cii.com>
Co-authored-by: Danilo Leal <daniloleal09@gmail.com>
Co-authored-by: Marshall Bowers <git@maxdeviant.com>
Marshall Bowers