This extracts a `language_core` crate from the existing `language`
crate, and creates a `grammars` data crate. The goal is to separate
tree-sitter grammar infrastructure, language configuration, and LSP
adapter types from the heavier buffer/editor integration layer in
`language`.
## Motivation
The `language` crate pulls in `text`, `theme`, `settings`, `rpc`,
`task`, `fs`, `clock`, `sum_tree`, and `fuzzy` — all of which are needed
for buffer integration (`Buffer`, `SyntaxMap`, `Outline`,
`DiagnosticSet`) but not for grammar parsing or language configuration.
Extracting the core types lets downstream consumers depend on
`language_core` without pulling in the full integration stack.
## Dependency graph after extraction
```
language_core ← gpui, lsp, tree-sitter, util, collections
grammars ← language_core, rust_embed, tree-sitter-{rust,python,...}
language ← language_core, text, theme, settings, rpc, task, fs, ...
languages ← language, grammars
```
## What moved to `language_core`
- `Grammar`, `GrammarId`, and all query config/builder types
- `LanguageConfig`, `LanguageMatcher`, bracket/comment/indent config
types
- `HighlightMap`, `HighlightId` (theme-dependent free functions
`highlight_style` and `highlight_name` stay in `language`)
- `LanguageName`, `LanguageId`
- `LanguageQueries`, `QUERY_FILENAME_PREFIXES`
- `CodeLabel`, `CodeLabelBuilder`, `Symbol`
- `Diagnostic`, `DiagnosticSourceKind`
- `Toolchain`, `ToolchainScope`, `ToolchainList`, `ToolchainMetadata`
- `ManifestName`
- `SoftWrap`
- LSP data types: `BinaryStatus`, `ServerHealth`,
`LanguageServerStatusUpdate`, `PromptResponseContext`, `ToLspPosition`
## What stays in `language`
- `Buffer`, `BufferSnapshot`, `SyntaxMap`, `Outline`, `DiagnosticSet`,
`LanguageScope`
- `LspAdapter`, `CachedLspAdapter`, `LspAdapterDelegate` (reference
`Arc<Language>` and `WorktreeId`)
- `ToolchainLister`, `LanguageToolchainStore` (reference `task` and
`settings` types)
- `ManifestQuery`, `ManifestProvider`, `ManifestDelegate` (reference
`WorktreeId`)
- Parser/query cursor pools, `PLAIN_TEXT`, point conversion functions
## What the `grammars` crate provides
- Embedded `.scm` query files and `config.toml` files for all built-in
languages (via `rust_embed`)
- `load_queries(name)`, `load_config(name)`,
`load_config_for_feature(name, grammars_loaded)`, and `get_file(path)`
functions
- `native_grammars()` for tree-sitter grammar registration (behind
`load-grammars` feature)
## Pre-cleanup (also in this PR)
- Removed unused `Option<&Buffer>` from
`LspAdapter::process_diagnostics`
- Removed unused `&App` from `LspAdapter::retain_old_diagnostic`
- Removed `fs: &dyn Fs` from `ToolchainLister` trait methods
(`PythonToolchainProvider` captures `fs` at construction time instead)
- Moved `Diagnostic`/`DiagnosticSourceKind` out of `buffer.rs` into
their own module
## Backward compatibility
The `language` crate re-exports everything from `language_core`, so
existing `use language::Grammar` (etc.) continues to work unchanged. The
only downstream change required is importing `CodeLabelExt` where
`.fallback_for_completion()` is called on the now-foreign `CodeLabel`
type.
Release Notes:
- N/A
---------
Co-authored-by: Agus Zubiaga <agus@zed.dev>
Co-authored-by: Tom Houlé <tom@tomhoule.com>
Dropping deep tree-sitter Trees can be quite slow due to deallocating
lots of memory (in the 10s of milliseconds for big diffs). To avoid
blocking the main thread, we offload the drop operation to a background
thread.
Instead of a static thread we could also integrate this with the gpui
app or use the executors, but both of that would require threading that
through as a field which I don't think is too great either
Release Notes:
- N/A *or* Added/Fixed/Improved ...
Helps #48601
<img width="1649" height="1071" alt="image"
src="https://github.com/user-attachments/assets/ff3dfee0-cc65-430f-a5fa-b4b4c36e8183"
/>
`syntax_layers` does some offset conversion that might require getting
some chunks from the rope, which is quite expensive. For detecting
autoclose, we only use the language from those syntax layers, so having
a short path that skips all the conversion should skip some sum_tree
traversals.
I'm pretty sure other places would benefit from this as well, but I
haven't searched them yet.
Release Notes:
- N/A
---------
Co-authored-by: Smit Barmase <heysmitbarmase@gmail.com>
Closes#46104
Release Notes:
- Fixed language injections sticking after language comment removed
---
This is working well for the Python SQL comments
```python
from pyspark.sql import SparkSession
spark = SparkSession.builder.getOrCreate()
# sql
cmd = "SELECT col1, col2 FROM tbl"
df = spark.sql( # sql
"""
WITH cte AS (
SELECT col1, col2, COUNT(*) AS n
FROM tbl
GROUP BY ALL
)
SELECT * FROM cte
"""
)
```
And go comments
```go
package main
func test() {
var _ = /* sql */ `SELECT id, name FROM products`
var _ = /* sql */ "SELECT id, name FROM products"
var _ = /* sql */ `SELECT id, name FROM products`
var _ = /* sql*/ "SELECT id, name FROM products"
}
```
Note: `f04b252dd9` was a simpler implementation that worked for
non-inline comments in Python, but produced the following Go behavior
---------
Co-authored-by: Joseph T. Lyons <JosephTLyons@gmail.com>
While at it, annotate more functions that are potentially related to
language parsing in buffers.
Also, on macOS, in order to actually have callstack frames properly
recorded by Tracy, you need to manually run `dsymutil` on the binary.
Release Notes:
- N/A
We use `block_with_timeout` to give the reparse task a millisecond to
complete, and if it takes longer we put the work off to the background.
The reason for this is that we want tree-sitter based features to feel
snappy.
The reparse task is non-cooperative though, it has no yield points,
giving us no place to actually check for our timeout, meaning we will
always drive it to completion and block for the entire duration.
This kicks out the `block_with_timeout` in favor of using the treesitter
progress callback to handle timeouts instead.
Release Notes:
- Improved responsiveness with very language language files on edits
To do
* [x] Default to no context retrieval. Allow opting in to LSP-based
retrieval via a setting (for users in `zeta2` feature flag)
* [x] Feed this context to models when enabled
* [x] Make the zeta2 context view work well with LSP retrieval
* [x] Add a UI for the setting (for feature-flagged users)
* [x] Ensure Zeta CLI `context` command is usable
---
* [ ] Filter out LSP definitions that are too large / entire files (e.g.
modules)
* [ ] Introduce timeouts
* [ ] Test with other LSPs
* [ ] Figure out hangs
Release Notes:
- N/A
---------
Co-authored-by: Ben Kunkle <ben@zed.dev>
Co-authored-by: Agus Zubiaga <agus@zed.dev>
It is easy for us to get the two fields out of sync causing weird
problems, there is no reason to have both here so.
Release Notes:
- N/A *or* Added/Fixed/Improved ...
Co-authored by: Antonio Scandurra <antonio@zed.dev>
We've realized that a lot of the logic within an
`EditPredictionProvider` is not specific to a particular edit prediction
model / service. Rather, it is just the generic state management
required to perform edit predictions at all in Zed. We want to move to a
setup where there's one "built-in" edit prediction provider in Zed,
which can be pointed at different edit prediction models. The only logic
that is different for different models is how we construct the prompt,
send the request, and parse the output.
This PR also changes the behavior of the staff-only `zeta2` feature flag
so that in only gates your *ability* to use Zeta2, but you can still use
your local settings file to choose between different edit prediction
models/services: zeta1, zeta2, and sweep.
This PR also makes zeta1's outcome reporting and prediction-rating
features work with all prediction models, not just zeta1.
To do:
* [x] remove duplicated logic around sending cloud requests between
zeta1 and zeta2
* [x] port the outcome reporting logic from zeta to zeta2.
* [x] get the "rate completions" modal working with all EP models
* [x] display edit prediction diff
* [x] show edit history events
* [x] remove the original `zeta` crate.
Release Notes:
- N/A
---------
Co-authored-by: Agus Zubiaga <agus@zed.dev>
Co-authored-by: Ben Kunkle <ben@zed.dev>
Closes#40632
**TL;DR:** The `wrap selections in tag` action was unavailable in ERB
files, even when the cursor was positioned in HTML content (outside of
Ruby code blocks). This happened because `syntax_layer_at()` incorrectly
returned the Ruby language for positions that were actually in HTML.
**NOTE:** I am not familiar with that part of Zed so it could be that
the fix here is completely incorrect.
Previously, `syntax_layer_at` incorrectly reported injected languages
(e.g., Ruby in ERB files) even when the cursor was in the base language
content (HTML). This broke actions like `wrap selections in tag` that
depend on language-specific configuration.
The issue had two parts:
1. Missing start boundary check: The filter only checked if a layer's
end was after the cursor (`end_byte() > offset`), not if it started
before, causing layers outside the cursor position to be included. See
the `BEFORE` video: when I click on the HTML part it reports `Ruby`
language instead of `HTML`.
2. Wrong boundary reference for injections: For injected layers with
`included_sub_ranges` (like Ruby code blocks in ERB), checking the root
node boundaries returned the entire file range instead of the actual
injection ranges.
This fix:
- Adds the containment check using half-open range semantics [start,
end) for root node boundaries. That ensures proper reporting of the
detected language when a cursor (`|`) is located right after the
injection:
```
<body>
<%= yield %>|
</body>
```
- Checks `included_sub_ranges` for injected layers to determine if the
cursor is actually within an injection
- Falls back to root node boundaries for base layers without sub-ranges.
This is the original behavior.
Fixes ERB language support where actions should be available based on
the cursor's actual language context. I think that also applies to some
other template languages like HEEX (Phoenix) and `*.pug`. On short
videos below you can see how I navigate through the ERB template and the
terminal on the right outputs the detected language if you apply the
following patch:
```diff
diff --git i/crates/editor/src/editor.rs w/crates/editor/src/editor.rs
index 15af61f5d2..54a8e0ae37 100644
--- i/crates/editor/src/editor.rs
+++ w/crates/editor/src/editor.rs
@@ -10671,6 +10671,7 @@ impl Editor {
for selection in self.selections.disjoint_anchors_arc().iter() {
if snapshot
.language_at(selection.start)
+ .inspect(|language| println!("Detected language: {:?}", language))
.and_then(|lang| lang.config().wrap_characters.as_ref())
.is_some()
{
```
**Before:**
https://github.com/user-attachments/assets/3f8358f4-d343-462e-b6b1-3f1f2e8c533d
**After:**
https://github.com/user-attachments/assets/c1b9f065-1b44-45a2-8a24-76b7d812130d
Here is the ERB template:
```
<!DOCTYPE html>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<style>
/* Email styles need to be inline */
</style>
</head>
<body>
<%= yield %>
</body>
</html>
```
Release Notes:
- N/A
This removes around 900 unnecessary clones, ranging from cloning a few
ints all the way to large data structures and images.
A lot of these were fixed using `cargo clippy --fix --workspace
--all-targets`, however it often breaks other lints and needs to be run
again. This was then followed up with some manual fixing.
I understand this is a large diff, but all the changes are pretty
trivial. Rust is doing some heavy lifting here for us. Once I get it up
to speed with main, I'd appreciate this getting merged rather sooner
than later.
Release Notes:
- N/A
This is a bit of a readability improvement IMHO; I often find myself
confused when dealing when dimension pairs, as there's no easy way to
jump to the implementation of a dimension for tuples to remind myself
for the n-th time how exactly that impl works. Now it should be possible
to jump directly to that impl.
Another bonus is that Dimension supports 3-ary tuples as well - by using
a () as a default value of a 3rd dimension.
Release Notes:
- N/A
This gets rid of the need to pass context to all cursor functions. In
practice context is always immutable when interacting with cursors.
A nicety of this is in the follow-up PR we will be able to implement
Iterator for all Cursors/filter cursors (hell, we may be able to get rid
of filter cursor altogether, as it is just a custom `filter` impl on
iterator trait).
Release Notes:
- N/A
Todo:
* [x] Open diffed files as regular buffers
* [x] Update diff when buffers change
* [x] Show diffed filenames in the tab title
* [x] Investigate why syntax highlighting isn't reliably handled for old
text
* [x] remove unstage/restore buttons
Release Notes:
- Adds `zed --diff A B` to show the diff between the two files
---------
Co-authored-by: Max Brunsfeld <maxbrunsfeld@gmail.com>
Co-authored-by: Ben Brandt <benjamin.j.brandt@gmail.com>
Co-authored-by: Agus Zubiaga <agus@zed.dev>
Closes#27596
The problem in this case was incorrect identification of which language
(layer) contains the selection.
Language layer selection incorrectly assumed that the deepest
`SyntaxLayer` containing a range was the most specific. This worked for
Markdown (base document + injected subtrees) but failed for PHP, where
`injection.combined` injections are used to make HTML logically function
as the base layer, despite being at a greater depth in the layer stack.
This caused HTML to be incorrectly identified as the most specific
language for PHP ranges.
The solution is to track included sub-ranges for syntax layers and
filter out layers that don't contain a sub-range covering the desired
range. The top-level layer is never filtered to ensure gaps between
sibling nodes always have a fallback language, as the top-level layer is
likely more correct than the default language settings.
Release Notes:
- Fixed an issue in PHP where PHP language settings would be
occasionally overridden by HTML language settings
https://github.com/zed-industries/zed/issues/30972 brought up another
case where our context is not enough to track the actual source of the
issue: we get a general top-level error without inner error.
The reason for this was `.ok_or_else(|| anyhow!("failed to read HEAD
SHA"))?; ` on the top level.
The PR finally reworks the way we use anyhow to reduce such issues (or
at least make it simpler to bubble them up later in a fix).
On top of that, uses a few more anyhow methods for better readability.
* `.ok_or_else(|| anyhow!("..."))`, `map_err` and other similar error
conversion/option reporting cases are replaced with `context` and
`with_context` calls
* in addition to that, various `anyhow!("failed to do ...")` are
stripped with `.context("Doing ...")` messages instead to remove the
parasitic `failed to` text
* `anyhow::ensure!` is used instead of `if ... { return Err(...); }`
calls
* `anyhow::bail!` is used instead of `return Err(anyhow!(...));`
Release Notes:
- N/A
Closes#4271
Implemented by kicking of a task on the main thread at the end of
`Editor::handle_input` which waits for the buffer to be re-parsed before
checking if JSX tag completion possible based on the recent edits, and
if it is then it spawns a task on the background thread to generate the
edits to be auto-applied to the buffer
Release Notes:
- Added support for auto-closing of JSX tags
---------
Co-authored-by: Cole Miller <cole@zed.dev>
Co-authored-by: Max Brunsfeld <max@zed.dev>
Co-authored-by: Marshall Bowers <git@maxdeviant.com>
Co-authored-by: Mikayla <mikayla@zed.dev>
Co-authored-by: Peter Tripp <peter@zed.dev>
This reverts commit 9ef0501853 due to a
panic.
```
{
"thread": "main",
"payload": "9 is not a valid char boundary in path \"crates/…/LiveKitBridge/\"",
"location_data": {
"file": "crates/file_finder/src/file_finder.rs",
"line": 646
}
}
```
Release Notes:
- N/A
This PR makes progress on #7711 by identifying any common prefix of the
paths in the file finder's search results, and replacing the "interior"
of that prefix---every path segment but the first and last---with `...`,
when a heuristic indicates that the longest path would otherwise
overflow the modal.
The elision is not applied to any segment that contains a match for the
search query.
There may be more work to do on #7711 in the case of long result paths
that do not share a significant common prefix.
Release Notes:
- Improved display of long paths in the file finder modal
Co-authored-by: Max <max@zed.dev>
I didn't update it to 0.25 because its Wasm support seems to be
partially broken due to
https://github.com/tree-sitter/tree-sitter/pull/3938: it didn't
introduce a check that the Wasm module's ABI is new enough to include
supertype info while parsing it, and so in the case where it isn't it
ends up interpreting random bytes as the number of supertypes, causing
out-of-bounds memory accesses.
Closes#24489
Release Notes:
- Fixed a rare crash during syntax highlighting
This is quite experimental and untested in languages other than Rust.
It's written to attempt to do something sensible in many languages. Due
to its experimental nature, just releasing to staff, and so not
including it in release notes. Future release note might be "Improved
diagnostic excerpts by using syntactic info to determine the context
lines to show."
Release Notes:
- N/A
Co-Authored-By: Max <max@zed.dev>
Release Notes:
- vim: Added motions `[[`, `[]`, `]]`, `][` for navigating by section,
`[m`, `]m`, `[M`, `]M` for navigating by method, and `[*`, `]*`, `[/`,
`]/` for comments. These currently only work for languages built in to
Zed, as they are powered by new tree-sitter queries.
- vim: Added new text objects: `ic`, `ac` for inside/around classes,
`if`,`af` for functions/methods, and `g c` for comments. These currently
only work for languages built in to Zed, as they are powered by new
tree-sitter queries.
---------
Co-authored-by: Max <max@zed.dev>
We are going to use this in the multi-buffer to produce a summary for an
`Excerpt` that contains a `Range<Anchor>`.
Release Notes:
- N/A
Co-authored-by: Nathan <nathan@zed.dev>
This lets us provide a context when constructing the zero value. We need
it so we can require anchors to be associated with a buffer id, which
we're doing as part of simplifying the multibuffer API.
Release Notes:
- N/A
Co-authored-by: Nathan <nathan@zed.dev>
/cc @mrnugget
Release Notes:
- Fixed an issue where toggling inline completions in a markdown file
did not work correctly
---------
Co-authored-by: Marshall <marshall@zed.dev>
This allows us to detect the language from the extension if we use paths
in fenced code blocks.
Release Notes:
- You can now use file paths ending in a language-specific file
extension at the start of markdown code blocks.
The biggest hurdle turned out to be use of `Arc<Language>` in maps, as
`clippy::mutable_key_type` started triggering on it (due to - I suppose
- internal mutability on `HighlightMap`?). I switched over to using
`LanguageId` as the key type in some of the callsites, as that's what
`Language` uses anyways for it's hash/eq, though I've still had to
suppress the lint outside of language crate.
/cc @maxdeviant , le clippy guru.
Release Notes:
- N/A
