Refactors our query loading to instead have an enum of supported queries
and load the query files themselves in parallel, where possible.
We will use this to warn extension authors when unsupported queries are
present, as we no longer want to ship those as part of the extension.
It also now enforces one file per supported query - only one extension
currently utilizes this feature and was undocumented previously, so I
decided in favor of removing it here, since this will help with
enforcing extensions to just ship query files we actually support in the
future. The extension has also been migrated so that it does not break
with this effort
Side-effect of this change is that we now load both embedded and
external languages up to at least 10%/5% faster, since the files are now
read in parallel as opposed to sequentially one by one.
Release Notes:
- N/A
Bumps the treesitter version to include
https://github.com/tree-sitter/tree-sitter/pull/5851
Also makes some changes to `cx.spawn_dedicated` to make it work on web:
- remove the blocking `recv` on the main thread
- adds a new variant `TaskState::Rendezvous` to allow this code to
synchronously return a task
`TaskState::Rendezvous` is needed because of how
`spawn_dedicated_thread` works:
- the caller provides a callback that produces a non-`Send` `Future` on
the dedicated thread
- the callback is sent to the dedicated thread, executed, and the
resulting `Task` is sent back to the main thread
- this all happens synchronously, so that `spawn_dedicated_thread`
returns a synchronous `Task`
However, the blocking `recv` on the main thread traps on the main worker
in the browser.
This PR replaces that with a call to `recv_async`, but this requires a
new `Rendezvous` variant on `TaskState` which represents "a task that it
still waiting to receive the underlying handle from the dedicated
thread".
It also enables the `flume/spin` feature on the web, which is needed to
avoid a synchronous lock acquisition in `send` (it's replaced with a
spinlock). Note that `send_async` wouldn't work here because it acquires
the same lock, and it's unnecessary because `send` only blocks when the
channel is full, but it's an unbounded channel.
---
Release Notes:
- N/A or Added/Fixed/Improved ...
# Objective
I noticed that in workspace symbol search, the function's symbol kind
has become `Trait`.
## Solution
Add `to_proto` and a macro to define the mapping instead of `as i32`.
## Testing
Updated the test.
## Self-Review Checklist:
- [x] I've reviewed my own diff for quality, security, and reliability
- [x] Unsafe blocks (if any) have justifying comments
- [x] The content adheres to Zed's UI standards
([UX/UI](https://github.com/zed-industries/zed/blob/main/CONTRIBUTING.md#uiux-checklist)
and
[icon](https://github.com/zed-industries/zed/blob/main/crates/icons/README.md)
guidelines)
- [x] Tests cover the new/changed behavior
- [x] Performance impact has been considered and is acceptable
---
Release Notes:
- Fixed `SymbolKind` mapping to LSP protocol values
This adds a dedicated `AvailableLanguages` struct in preparation for a
more cabable language matching based on a given language config. No
functional changes, just shuffling some code around for this round.
Also made the LanguageMatcher non-cloneable in favor of wrapping it in
an Arc, since cloning is rather expensive for this and having a
reference is sufficient in all cases right now.
Release Notes:
- N/A
---------
Co-authored-by: Ben Brandt <benjamin.j.brandt@gmail.com>
Closes#59620
## Problem
A language whose name contains a `/`, such as a custom `PL/X` extension
(`lsp_id` → `pl/x`), broke snippets end to end:
- The `snippets: configure snippets` action wrote the file to
`~/.config/zed/snippets/pl/x.json`, i.e. inside a `pl/` subdirectory.
- The snippet scanner reads `snippets/` non-recursively and skips
directories, so that file was never loaded and the snippet could never
be used.
- The completion lookup keyed off the raw `lsp_id` (`pl/x`), which
wouldn't have matched the file-stem key even if the file had been
scanned.
So Zed's own UI created a snippet file it could never read back.
## Fix
Add `LanguageName::snippet_scope_id()` (the `lsp_id` with `/` and `\`
removed) and use it everywhere a language maps to its snippet file name
or lookup key:
- the Configure Snippets writer and its "already configured" label, and
- the two completion lookups in `editor`.
`PL/X` now maps to a flat `plx.json`, as suggested in the issue. The
`editor::InsertSnippet` action is intentionally left unchanged: its
`language` field is documented to be the snippet file name stem, which
is already separator-free.
Note: files created under the old behavior (nested `foo/bar.json`)
aren't migrated; re-running Configure Snippets writes the corrected flat
file.
## Testing
- Added a `language_core` unit test asserting `snippet_scope_id()`
strips `/` and `\` (e.g. `PL/X` → `plx`).
- `cargo test -p language_core` passes.
- `./script/clippy -p language_core -p language` passes.
- Docs Prettier passes.
Release Notes:
- Fixed snippets being unusable for languages whose name contains a `/`
character
This is necessary to remove some `util` dependencies from crates, as
well as better sharing for our projects. This also includes the WIP
AbsPath abstraction as well as some bug fixes from internal tooling.
Release Notes:
- N/A or Added/Fixed/Improved ...
Sets up infra for fixing #46881. Follow-up to the approach discussed in
#36802 (comment).
The initial problem: Go table tests weren't being detected reliably in
larger files. The old approach created one tree-sitter match per table
row, which hit the cursor match limit (64) and caused rows to be
silently dropped.
The suggested fix was to use query repetition to capture all rows in a
single match, then post-process in Rust.
But this hit another problem: some captured rows aren't valid runnables
(e.g., when a struct has multiple string fields and the query can't tell
which one is the test name). Tree-sitter predicates can't validate
subsets of captures within a match, that logic needs to happen in Rust.
This PR adds a `RunnableResolver` trait that lets languages post-process
multi-capture matches. When a query uses `@_run_item` to mark item
boundaries, we split the captures into per-item groups and pass them to
the resolver. The resolver can then:
- Pick which `@run` to use: e.g., compare field names against
`t.Run(tc.name, ...)` to find the right string field.
- Return per-item extras: only the captures that correspond to the
chosen `@run`.
Release Notes:
- N/A
We had an internal report of soft wrap not working in git panel's commit
editor. Given the following settings:
```json
{
"languages": {
"Git Commit": {
"preferred_line_length": 80,
"soft_wrap": "preferred_line_length",
},
},
}
```
We would not soft-wrap in narrow viewports. As it turned out, the
problem was that we were always prefering a `preferred_line_length` as
our soft wrap boundary over the actual width of the editor.
Self-Review Checklist:
- [x] I've reviewed my own diff for quality, security, and reliability
- [x] Unsafe blocks (if any) have justifying comments
- [ ] The content is consistent with the [UI/UX
checklist](https://github.com/zed-industries/zed/blob/main/CONTRIBUTING.md#uiux-checklist)
- [ ] Tests cover the new/changed behavior
- [x] Performance impact has been considered and is acceptable
Closes #ISSUE
Release Notes:
- Fixed git commits editor not respecting soft wrap boundaries.
- settings: Removed `"soft_wrap": "preferred_line_length"` in favour of
`"soft_wrap": "bounded"`. Soft wrap now always respects editor width
when it's enabled.
- `language_model` no longer depends on provider-specific crates such as
`anthropic` and `open_ai` (inverted dependency)
- `language_model_core` was extracted from `language_model` which
contains the types for the provider-specific crates to convert to/from.
- `gpui::SharedString` has been extracted into its own crate (still
exposed by `gpui`), so `language_model_core` and provider API crates
don't have to depend on `gpui`.
- Removes some unnecessary `&'static str` | `SharedString` -> `String`
-> `SharedString` conversions across the codebase.
- Extracts the core logic of the cloud `LanguageModelProvider` into its
own crate with simpler dependencies.
Release Notes:
- N/A
---------
Co-authored-by: John Tur <john-tur@outlook.com>
Self-Review Checklist:
- [X] I've reviewed my own diff for quality, security, and reliability
- [ ] Unsafe blocks (if any) have justifying comments
- [ ] The content is consistent with the [UI/UX
checklist](https://github.com/zed-industries/zed/blob/main/CONTRIBUTING.md#uiux-checklist)
- [ ] Tests cover the new/changed behavior
- [ ] Performance impact has been considered and is acceptable
Release Notes:
- Added the option for highlights from languages to specify fallbacks.
That means that if you have a pattern with the captures `@second.capture
@first.capture`, Zed will first try resolving a highlight from your
theme for the code fragment using the first capture, then look for the
second capture if no match for the first capture could be found.
---------
Co-authored-by: Kirill Bulatov <kirill@zed.dev>
This PR removes the imports query and all surrounding support from the
language_core crate.
The imports query was experimented with during the addition of Zeta 2.
However, it has been unused for a few months now and the direction does
not seemt to be pursued anymore.
Thus, removing the support here.
Release Notes:
- N/A
This extracts a `language_core` crate from the existing `language`
crate, and creates a `grammars` data crate. The goal is to separate
tree-sitter grammar infrastructure, language configuration, and LSP
adapter types from the heavier buffer/editor integration layer in
`language`.
## Motivation
The `language` crate pulls in `text`, `theme`, `settings`, `rpc`,
`task`, `fs`, `clock`, `sum_tree`, and `fuzzy` — all of which are needed
for buffer integration (`Buffer`, `SyntaxMap`, `Outline`,
`DiagnosticSet`) but not for grammar parsing or language configuration.
Extracting the core types lets downstream consumers depend on
`language_core` without pulling in the full integration stack.
## Dependency graph after extraction
```
language_core ← gpui, lsp, tree-sitter, util, collections
grammars ← language_core, rust_embed, tree-sitter-{rust,python,...}
language ← language_core, text, theme, settings, rpc, task, fs, ...
languages ← language, grammars
```
## What moved to `language_core`
- `Grammar`, `GrammarId`, and all query config/builder types
- `LanguageConfig`, `LanguageMatcher`, bracket/comment/indent config
types
- `HighlightMap`, `HighlightId` (theme-dependent free functions
`highlight_style` and `highlight_name` stay in `language`)
- `LanguageName`, `LanguageId`
- `LanguageQueries`, `QUERY_FILENAME_PREFIXES`
- `CodeLabel`, `CodeLabelBuilder`, `Symbol`
- `Diagnostic`, `DiagnosticSourceKind`
- `Toolchain`, `ToolchainScope`, `ToolchainList`, `ToolchainMetadata`
- `ManifestName`
- `SoftWrap`
- LSP data types: `BinaryStatus`, `ServerHealth`,
`LanguageServerStatusUpdate`, `PromptResponseContext`, `ToLspPosition`
## What stays in `language`
- `Buffer`, `BufferSnapshot`, `SyntaxMap`, `Outline`, `DiagnosticSet`,
`LanguageScope`
- `LspAdapter`, `CachedLspAdapter`, `LspAdapterDelegate` (reference
`Arc<Language>` and `WorktreeId`)
- `ToolchainLister`, `LanguageToolchainStore` (reference `task` and
`settings` types)
- `ManifestQuery`, `ManifestProvider`, `ManifestDelegate` (reference
`WorktreeId`)
- Parser/query cursor pools, `PLAIN_TEXT`, point conversion functions
## What the `grammars` crate provides
- Embedded `.scm` query files and `config.toml` files for all built-in
languages (via `rust_embed`)
- `load_queries(name)`, `load_config(name)`,
`load_config_for_feature(name, grammars_loaded)`, and `get_file(path)`
functions
- `native_grammars()` for tree-sitter grammar registration (behind
`load-grammars` feature)
## Pre-cleanup (also in this PR)
- Removed unused `Option<&Buffer>` from
`LspAdapter::process_diagnostics`
- Removed unused `&App` from `LspAdapter::retain_old_diagnostic`
- Removed `fs: &dyn Fs` from `ToolchainLister` trait methods
(`PythonToolchainProvider` captures `fs` at construction time instead)
- Moved `Diagnostic`/`DiagnosticSourceKind` out of `buffer.rs` into
their own module
## Backward compatibility
The `language` crate re-exports everything from `language_core`, so
existing `use language::Grammar` (etc.) continues to work unchanged. The
only downstream change required is importing `CodeLabelExt` where
`.fallback_for_completion()` is called on the now-foreign `CodeLabel`
type.
Release Notes:
- N/A
---------
Co-authored-by: Agus Zubiaga <agus@zed.dev>
Co-authored-by: Tom Houlé <tom@tomhoule.com>
Finn Evers