Also updates CI to always run this - that way we ensure that disabling
or enabling tests does not become stale at some point and thus can
safely add this.
Release Notes:
- N/A
- [GPUI] N/A
Closes https://github.com/zed-industries/zed/issues/63730
Related to #52340 and #43460
Based on https://github.com/zed-industries/zed/pull/63138 and needs it
to be merged first.
`BufferSnapshot::chunks` re-ran the tree-sitter highlight query over the
requested range on every call, so scrolling and cursor movement paid the
full query cost per frame.
Highlight runs are now computed once per 50-row chunk (the same
`RowChunks` that back bracket colorization), resolved against the
current `HighlightMap` at compute time, merged into `(Range<usize>,
HighlightId)` runs, and kept in a cost-budgeted per-buffer LRU (10MB).
Entries reuse the `ResolvedHighlights` representation from #63138: each
stores the grammars and maps it was resolved against and is served only
while every map is pointer-identical (`is_current`), so theme changes
and grammar reloads invalidate structurally, with no invalidation hooks
and no global state.
Edits and reparses drop the cache through the existing `TreeSitterData`
swap.
Chunks spanning more than 64KB bypass the cache and fall back to direct
querying, so files with giant lines behave as before.
## Benchmarks
`editor_render_highlighted` and `editor_render_highlighted_minimap`
benches (`cargo bench -p benchmarks --bench editor_render`) move the
cursor through a generated 10K-line Rust file with real tree-sitter
highlighting, with and without the minimap.
The benches are new in this PR, so the baseline is the same binary with
`ZED_DISABLE_HIGHLIGHT_CACHE=1`, which routes rendering through the
pre-existing direct-query path.
Run 1 — baseline, cache disabled:
```
editor_render_highlighted time: [1.7164 ms 1.7377 ms 1.7604 ms]
editor_render_highlighted_minimap time: [12.273 ms 12.407 ms 12.549 ms]
```
Run 2 — cache enabled:
```
editor_render_highlighted time: [1.4572 ms 1.4597 ms 1.4627 ms]
change: [-16.378% -14.944% -13.526%] (p = 0.00 < 0.05)
editor_render_highlighted_minimap time: [9.2750 ms 9.3213 ms 9.3744 ms]
change: [-25.800% -24.872% -23.930%] (p = 0.00 < 0.05)
```
Over the same runs, GPUI reports window draw mean 0.81 ms → 0.68 ms and
p99 1.06 ms → 0.76 ms without the minimap, and mean 5.14 ms → 4.12 ms,
p99 6.12 ms → 4.75 ms with it.
The relative deltas held across two independent sessions (-14.1…-14.9%
and -20.7…-24.9%), while absolute times drift a few percent with machine
state.
Release Notes:
- Improved editor rendering performance in syntax-highlighted files,
especially with the minimap enabled
Part of https://github.com/zed-industries/zed/issues/57349
`MarkdownElement` re-highlights every code block with tree-sitter on
every frame.
Each visible code block pays a full parse per frame, which makes agent
panel streaming and hover popovers janky once the content grows.
This PR computes code block highlights once, in the background parse
task, instead of at render time:
- Highlights are resolved into `(Range<usize>, HighlightId)` runs at
parse time; each entry stores the grammar and `HighlightMap` it was
resolved against and is served only while the grammar still holds the
pointer-identical map (`ResolvedHighlights::is_current`), so theme
changes need no invalidation hooks, and a block whose entry went stale
falls back to direct fragment highlighting at render until the next
parse.
- `Language::highlight_text_resolved` memoizes results in a
content-keyed LRU (4MB budget, 512KB per-entry cap), so the re-parses
triggered by streaming appends reuse the highlights of unchanged blocks
instead of re-running tree-sitter on them.
- Blocks larger than the per-entry cap bypass the cache and are
highlighted directly, as before.
`CostBudgetedLru` is generic rather than specialized because a follow-up
PR reuses it for per-buffer highlight caching in
`BufferSnapshot::chunks`; the capture flattening keyed by grammar index
(`HighlightCaptureRef`) is shared with that follow-up as well.
## Benchmarks
`cargo bench -p benchmarks --bench markdown_renderer` renders generated
documents that mix paragraphs, headings, tables, and Rust code blocks.
**Run 1 — baseline, `main` (`ee2366a395`):**
```
Markdown render/min_bytes/5000 time: [1.2956 ms 1.2974 ms 1.3016 ms]
Markdown render/min_bytes/10000 time: [1.7919 ms 1.7955 ms 1.8026 ms]
Markdown render/min_bytes/50000 time: [6.5510 ms 6.6241 ms 6.6671 ms]
Markdown render/min_bytes/250000 time: [37.644 µs 2.4819 ms 4.3791 ms] <- bimodal already on main
```
**Run 2 — this branch:**
```
Markdown render/min_bytes/5000 time: [940.11 µs 945.28 µs 948.79 µs]
change: [-29.317% -28.385% -27.607%] (p = 0.00 < 0.05)
Markdown render/min_bytes/10000 time: [1.1162 ms 1.1331 ms 1.1407 ms]
change: [-40.859% -39.151% -37.719%] (p = 0.00 < 0.05)
Markdown render/min_bytes/50000 time: [2.4943 ms 2.9225 ms 3.1051 ms]
change: [-80.088% -67.885% -58.058%] (p = 0.00 < 0.05)
Markdown render/min_bytes/250000 time: [43.442 µs 44.661 µs 45.514 µs]
change: [-98.210% -95.806% +5.5167%] (p = 0.17 > 0.05)
```
Over the same runs, GPUI reports window draw mean 2.22 ms → 1.26 ms, p99
31.2 ms → 16.0 ms, and the share of frames that blow the frame budget
drops from 3.9% to 2.0%.
Note: at the 250,000-byte case root blocks render lazily, which makes
the measurement bimodal and inconclusive in both configurations.
Release Notes:
- Improved rendering performance of markdown code blocks (agent panel,
hover popovers, markdown preview)
Release Notes:
- Fixed injected language highlighting for strings containing
interpolations
current | updated
:-------------------------:|:-------------------------:
|
<details>
<summary> test snippet </summary>
```python
my_regex = r"[0-9]"
spark.sql( # sql
f"""
SELECT
col1,
CASE WHEN col2 RLIKE '{my_regex}' THEN '1' ELSE '2' END col4,
ROW_NUMBER() OVER(PARTITION BY col1, col2 ORDER BY col3) as rn
FROM tbl
"""
)
```
</details>
## Summary
Strings with interpolations like `f"SELECT col1 FROM tbl WHERE col2 =
'{my_var}'"` in Python produce multiple `string_content` nodes. Without
grouping, each fragment gets its own injection layer which breaks syntax
highlighting.
This PR adds an `@injection.host` capture to group `@injection.content`
ranges by their host node. This was implemented by extending the
existing `combined_injection_ranges` hash map with a node key per host.
Added tests for this behavior covering simple strings, f-strings with
single/multiple interpolations, and multi-string isolation.
---------
Co-authored-by: Piotr Osiewicz <24362066+osiewicz@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
I think this is the final attempt of mine to do anything about the
binary size, as those fixes become relatively intricate and I like them
less and less.
Each commit has its own improvement stats and mainly reduce generic-code
duplication through shared non-generic helpers and `#[inline(never)]`
boundaries.
Total reduction stats:
With sccache disabled, `cargo clean` and `cargo build --release -p zed`:
Before: zed binary 396716472 bytes
After: zed binary 359416504 bytes (-9.40%, -37.29MB)
Release Notes:
- N/A
---------
Co-authored-by: zed-zippy[bot] <234243425+zed-zippy[bot]@users.noreply.github.com>
Refactors our query loading to instead have an enum of supported queries
and load the query files themselves in parallel, where possible.
We will use this to warn extension authors when unsupported queries are
present, as we no longer want to ship those as part of the extension.
It also now enforces one file per supported query - only one extension
currently utilizes this feature and was undocumented previously, so I
decided in favor of removing it here, since this will help with
enforcing extensions to just ship query files we actually support in the
future. The extension has also been migrated so that it does not break
with this effort
Side-effect of this change is that we now load both embedded and
external languages up to at least 10%/5% faster, since the files are now
read in parallel as opposed to sequentially one by one.
Release Notes:
- N/A
Bumps the treesitter version to include
https://github.com/tree-sitter/tree-sitter/pull/5851
Also makes some changes to `cx.spawn_dedicated` to make it work on web:
- remove the blocking `recv` on the main thread
- adds a new variant `TaskState::Rendezvous` to allow this code to
synchronously return a task
`TaskState::Rendezvous` is needed because of how
`spawn_dedicated_thread` works:
- the caller provides a callback that produces a non-`Send` `Future` on
the dedicated thread
- the callback is sent to the dedicated thread, executed, and the
resulting `Task` is sent back to the main thread
- this all happens synchronously, so that `spawn_dedicated_thread`
returns a synchronous `Task`
However, the blocking `recv` on the main thread traps on the main worker
in the browser.
This PR replaces that with a call to `recv_async`, but this requires a
new `Rendezvous` variant on `TaskState` which represents "a task that it
still waiting to receive the underlying handle from the dedicated
thread".
It also enables the `flume/spin` feature on the web, which is needed to
avoid a synchronous lock acquisition in `send` (it's replaced with a
spinlock). Note that `send_async` wouldn't work here because it acquires
the same lock, and it's unnecessary because `send` only blocks when the
channel is full, but it's an unbounded channel.
---
Release Notes:
- N/A or Added/Fixed/Improved ...
# Objective
I noticed that in workspace symbol search, the function's symbol kind
has become `Trait`.
## Solution
Add `to_proto` and a macro to define the mapping instead of `as i32`.
## Testing
Updated the test.
## Self-Review Checklist:
- [x] I've reviewed my own diff for quality, security, and reliability
- [x] Unsafe blocks (if any) have justifying comments
- [x] The content adheres to Zed's UI standards
([UX/UI](https://github.com/zed-industries/zed/blob/main/CONTRIBUTING.md#uiux-checklist)
and
[icon](https://github.com/zed-industries/zed/blob/main/crates/icons/README.md)
guidelines)
- [x] Tests cover the new/changed behavior
- [x] Performance impact has been considered and is acceptable
---
Release Notes:
- Fixed `SymbolKind` mapping to LSP protocol values
This adds a dedicated `AvailableLanguages` struct in preparation for a
more cabable language matching based on a given language config. No
functional changes, just shuffling some code around for this round.
Also made the LanguageMatcher non-cloneable in favor of wrapping it in
an Arc, since cloning is rather expensive for this and having a
reference is sufficient in all cases right now.
Release Notes:
- N/A
---------
Co-authored-by: Ben Brandt <benjamin.j.brandt@gmail.com>
Closes#59620
## Problem
A language whose name contains a `/`, such as a custom `PL/X` extension
(`lsp_id` → `pl/x`), broke snippets end to end:
- The `snippets: configure snippets` action wrote the file to
`~/.config/zed/snippets/pl/x.json`, i.e. inside a `pl/` subdirectory.
- The snippet scanner reads `snippets/` non-recursively and skips
directories, so that file was never loaded and the snippet could never
be used.
- The completion lookup keyed off the raw `lsp_id` (`pl/x`), which
wouldn't have matched the file-stem key even if the file had been
scanned.
So Zed's own UI created a snippet file it could never read back.
## Fix
Add `LanguageName::snippet_scope_id()` (the `lsp_id` with `/` and `\`
removed) and use it everywhere a language maps to its snippet file name
or lookup key:
- the Configure Snippets writer and its "already configured" label, and
- the two completion lookups in `editor`.
`PL/X` now maps to a flat `plx.json`, as suggested in the issue. The
`editor::InsertSnippet` action is intentionally left unchanged: its
`language` field is documented to be the snippet file name stem, which
is already separator-free.
Note: files created under the old behavior (nested `foo/bar.json`)
aren't migrated; re-running Configure Snippets writes the corrected flat
file.
## Testing
- Added a `language_core` unit test asserting `snippet_scope_id()`
strips `/` and `\` (e.g. `PL/X` → `plx`).
- `cargo test -p language_core` passes.
- `./script/clippy -p language_core -p language` passes.
- Docs Prettier passes.
Release Notes:
- Fixed snippets being unusable for languages whose name contains a `/`
character
This is necessary to remove some `util` dependencies from crates, as
well as better sharing for our projects. This also includes the WIP
AbsPath abstraction as well as some bug fixes from internal tooling.
Release Notes:
- N/A or Added/Fixed/Improved ...
Sets up infra for fixing #46881. Follow-up to the approach discussed in
#36802 (comment).
The initial problem: Go table tests weren't being detected reliably in
larger files. The old approach created one tree-sitter match per table
row, which hit the cursor match limit (64) and caused rows to be
silently dropped.
The suggested fix was to use query repetition to capture all rows in a
single match, then post-process in Rust.
But this hit another problem: some captured rows aren't valid runnables
(e.g., when a struct has multiple string fields and the query can't tell
which one is the test name). Tree-sitter predicates can't validate
subsets of captures within a match, that logic needs to happen in Rust.
This PR adds a `RunnableResolver` trait that lets languages post-process
multi-capture matches. When a query uses `@_run_item` to mark item
boundaries, we split the captures into per-item groups and pass them to
the resolver. The resolver can then:
- Pick which `@run` to use: e.g., compare field names against
`t.Run(tc.name, ...)` to find the right string field.
- Return per-item extras: only the captures that correspond to the
chosen `@run`.
Release Notes:
- N/A
We had an internal report of soft wrap not working in git panel's commit
editor. Given the following settings:
```json
{
"languages": {
"Git Commit": {
"preferred_line_length": 80,
"soft_wrap": "preferred_line_length",
},
},
}
```
We would not soft-wrap in narrow viewports. As it turned out, the
problem was that we were always prefering a `preferred_line_length` as
our soft wrap boundary over the actual width of the editor.
Self-Review Checklist:
- [x] I've reviewed my own diff for quality, security, and reliability
- [x] Unsafe blocks (if any) have justifying comments
- [ ] The content is consistent with the [UI/UX
checklist](https://github.com/zed-industries/zed/blob/main/CONTRIBUTING.md#uiux-checklist)
- [ ] Tests cover the new/changed behavior
- [x] Performance impact has been considered and is acceptable
Closes #ISSUE
Release Notes:
- Fixed git commits editor not respecting soft wrap boundaries.
- settings: Removed `"soft_wrap": "preferred_line_length"` in favour of
`"soft_wrap": "bounded"`. Soft wrap now always respects editor width
when it's enabled.
- `language_model` no longer depends on provider-specific crates such as
`anthropic` and `open_ai` (inverted dependency)
- `language_model_core` was extracted from `language_model` which
contains the types for the provider-specific crates to convert to/from.
- `gpui::SharedString` has been extracted into its own crate (still
exposed by `gpui`), so `language_model_core` and provider API crates
don't have to depend on `gpui`.
- Removes some unnecessary `&'static str` | `SharedString` -> `String`
-> `SharedString` conversions across the codebase.
- Extracts the core logic of the cloud `LanguageModelProvider` into its
own crate with simpler dependencies.
Release Notes:
- N/A
---------
Co-authored-by: John Tur <john-tur@outlook.com>
Self-Review Checklist:
- [X] I've reviewed my own diff for quality, security, and reliability
- [ ] Unsafe blocks (if any) have justifying comments
- [ ] The content is consistent with the [UI/UX
checklist](https://github.com/zed-industries/zed/blob/main/CONTRIBUTING.md#uiux-checklist)
- [ ] Tests cover the new/changed behavior
- [ ] Performance impact has been considered and is acceptable
Release Notes:
- Added the option for highlights from languages to specify fallbacks.
That means that if you have a pattern with the captures `@second.capture
@first.capture`, Zed will first try resolving a highlight from your
theme for the code fragment using the first capture, then look for the
second capture if no match for the first capture could be found.
---------
Co-authored-by: Kirill Bulatov <kirill@zed.dev>
This PR removes the imports query and all surrounding support from the
language_core crate.
The imports query was experimented with during the addition of Zeta 2.
However, it has been unused for a few months now and the direction does
not seemt to be pursued anymore.
Thus, removing the support here.
Release Notes:
- N/A
This extracts a `language_core` crate from the existing `language`
crate, and creates a `grammars` data crate. The goal is to separate
tree-sitter grammar infrastructure, language configuration, and LSP
adapter types from the heavier buffer/editor integration layer in
`language`.
## Motivation
The `language` crate pulls in `text`, `theme`, `settings`, `rpc`,
`task`, `fs`, `clock`, `sum_tree`, and `fuzzy` — all of which are needed
for buffer integration (`Buffer`, `SyntaxMap`, `Outline`,
`DiagnosticSet`) but not for grammar parsing or language configuration.
Extracting the core types lets downstream consumers depend on
`language_core` without pulling in the full integration stack.
## Dependency graph after extraction
```
language_core ← gpui, lsp, tree-sitter, util, collections
grammars ← language_core, rust_embed, tree-sitter-{rust,python,...}
language ← language_core, text, theme, settings, rpc, task, fs, ...
languages ← language, grammars
```
## What moved to `language_core`
- `Grammar`, `GrammarId`, and all query config/builder types
- `LanguageConfig`, `LanguageMatcher`, bracket/comment/indent config
types
- `HighlightMap`, `HighlightId` (theme-dependent free functions
`highlight_style` and `highlight_name` stay in `language`)
- `LanguageName`, `LanguageId`
- `LanguageQueries`, `QUERY_FILENAME_PREFIXES`
- `CodeLabel`, `CodeLabelBuilder`, `Symbol`
- `Diagnostic`, `DiagnosticSourceKind`
- `Toolchain`, `ToolchainScope`, `ToolchainList`, `ToolchainMetadata`
- `ManifestName`
- `SoftWrap`
- LSP data types: `BinaryStatus`, `ServerHealth`,
`LanguageServerStatusUpdate`, `PromptResponseContext`, `ToLspPosition`
## What stays in `language`
- `Buffer`, `BufferSnapshot`, `SyntaxMap`, `Outline`, `DiagnosticSet`,
`LanguageScope`
- `LspAdapter`, `CachedLspAdapter`, `LspAdapterDelegate` (reference
`Arc<Language>` and `WorktreeId`)
- `ToolchainLister`, `LanguageToolchainStore` (reference `task` and
`settings` types)
- `ManifestQuery`, `ManifestProvider`, `ManifestDelegate` (reference
`WorktreeId`)
- Parser/query cursor pools, `PLAIN_TEXT`, point conversion functions
## What the `grammars` crate provides
- Embedded `.scm` query files and `config.toml` files for all built-in
languages (via `rust_embed`)
- `load_queries(name)`, `load_config(name)`,
`load_config_for_feature(name, grammars_loaded)`, and `get_file(path)`
functions
- `native_grammars()` for tree-sitter grammar registration (behind
`load-grammars` feature)
## Pre-cleanup (also in this PR)
- Removed unused `Option<&Buffer>` from
`LspAdapter::process_diagnostics`
- Removed unused `&App` from `LspAdapter::retain_old_diagnostic`
- Removed `fs: &dyn Fs` from `ToolchainLister` trait methods
(`PythonToolchainProvider` captures `fs` at construction time instead)
- Moved `Diagnostic`/`DiagnosticSourceKind` out of `buffer.rs` into
their own module
## Backward compatibility
The `language` crate re-exports everything from `language_core`, so
existing `use language::Grammar` (etc.) continues to work unchanged. The
only downstream change required is importing `CodeLabelExt` where
`.fallback_for_completion()` is called on the now-foreign `CodeLabel`
type.
Release Notes:
- N/A
---------
Co-authored-by: Agus Zubiaga <agus@zed.dev>
Co-authored-by: Tom Houlé <tom@tomhoule.com>
Finn Evers