## Summary
Addresses #16965
This PR adds support for **opening and saving** files with legacy
encodings (non-UTF-8).
Previously, Zed failed to open files encoded in Shift-JIS, EUC-JP, Big5,
etc., displaying a "Could not open file" error screen. This PR
implements automatic encoding detection upon opening and ensures the
original encoding is preserved when saving.
## Implementation Details
1. **Worktree (Loading)**:
* Updated `load_file` to use `chardetng` for automatic encoding
detection.
* Files are decoded to UTF-8 internal strings for editing, while
preserving the detected `Encoding` metadata.
2. **Language / Buffer**:
* Added an `encoding` field to the `Buffer` struct to store the detected
encoding.
3. **Worktree (Saving)**:
* Updated `write_file` to accept the stored encoding.
* **Performance Optimization**:
* **UTF-8 Path**: Uses the existing optimized `fs.save` (streaming
chunks directly from Rope), ensuring no performance regression for the
vast majority of files.
* **Legacy Encoding Path**: Implemented a fallback that converts the
Rope to a contiguous `String/Bytes` in memory, re-encodes it to the
target format (e.g., Shift-JIS), and writes it to disk.
* *Note*: This fallback involves memory allocation, but it is necessary
to support legacy encodings without refactoring the `fs` crate's
streaming interfaces.
## Changes
- `crates/worktree`:
- Add dependencies: `encoding_rs`, `chardetng`.
- Update `load_file` to detect encoding and decode content.
- Update `write_file` to handle re-encoding on save.
- `crates/language`: Add `encoding` field and accessors to `Buffer`.
- `crates/project`: Pass encoding information between Worktree and
Buffer.
- `crates/vim`: Update `:w` command to use the new `write_file`
signature.
## Verification
I validated this manually using a Rust script to generate test files
with various encodings.
**Results:**
* ✅ **Success (Opened & Saved correctly):**
* **Japanese:** `Shift-JIS` (CP932), `EUC-JP`, `ISO-2022-JP`
* **Chinese:** `Big5` (Traditional), `GBK/GB2312` (Simplified)
* **Western/Unicode:** `Windows-1252` (CP1252), `UTF-16LE`, `UTF-16BE`
* ⚠️ **limitations (Detection accuracy):**
* Some specific encodings like `KOI8-R` or generic `Latin1` (ISO-8859-1)
may partially display replacement characters (`?`) depending on the file
content length. This is a known limitation of the heuristic detection
library (`chardetng`) rather than the saving logic.
Release Notes:
- Added support for opening and saving files with legacy encodings
(Shift-JIS, Big5, etc.)
---------
Co-authored-by: CrazyboyQCD <53971641+CrazyboyQCD@users.noreply.github.com>
Co-authored-by: Conrad Irwin <conrad.irwin@gmail.com>
Closes#43722
Release Notes:
- Fixed an issue where auto-indentation didn’t work correctly for Python
code blocks in Markdown.
---------
Co-authored-by: Smit Barmase <heysmitbarmase@gmail.com>
Currently we have a single cache for this data shared between all
snapshots which is incorrect, as we might update the cache to a new
version while having old snapshots around which then may try to access
new data with old offsets/rows.
Release Notes:
- N/A *or* Added/Fixed/Improved ...
Fixes a bug that led to us unnecessarily restarting a language server
when we were looking at a single file of a given language.
Release Notes:
- Fixed a bug that led to Zed sometimes starting an excessive amount of
language servers
To do
* [x] Default to no context retrieval. Allow opting in to LSP-based
retrieval via a setting (for users in `zeta2` feature flag)
* [x] Feed this context to models when enabled
* [x] Make the zeta2 context view work well with LSP retrieval
* [x] Add a UI for the setting (for feature-flagged users)
* [x] Ensure Zeta CLI `context` command is usable
---
* [ ] Filter out LSP definitions that are too large / entire files (e.g.
modules)
* [ ] Introduce timeouts
* [ ] Test with other LSPs
* [ ] Figure out hangs
Release Notes:
- N/A
---------
Co-authored-by: Ben Kunkle <ben@zed.dev>
Co-authored-by: Agus Zubiaga <agus@zed.dev>
Closes https://github.com/zed-industries/zed/issues/41935
The registration ID responsible for generating each diagnostic is now
tracked. This allows us to replace only the diagnostics from the same
registration ID when a pull diagnostics report is applied.
Additionally, various deficiencies in our support for pull diagnostics
have been fixed:
- Document pulls are issued for all open buffers, not just the edited
one. A shorter debounce is used for the edited buffer. Workspace
diagnostics are also now ignored for open buffers.
- Tracking of `lastResultId` is improved.
- Stored pull diagnostics are discarded when the corresponding buffer is
closed.
Release Notes:
- Improved compatibility with language servers that use the "pull
diagnostics" feature of Language Server Protocol.
---------
Co-authored-by: Kirill Bulatov <mail4score@gmail.com>
Co-authored-by: Kirill Bulatov <kirill@zed.dev>
Gist is we only need to block the foreground thread for reparsing if
immediate language changes are useful to the user. That is usually only
the case when they edit the buffer
Release Notes:
- Improved performance of large project searches and project diffs
Co-authored by: David Kleingeld <david@zed.dev>
Zeta evals now include a character n-gram metric adapted for multi-edit diffs (“delta chrF”). It works as follows:
1. Reconstruct the original, golden (expected), and actual texts from unified diffs.
- "original": the text before any edits
- "golden": the text after applying the expected edits
- "actual": the text after applying the actual edits
2. Compute n-gram count deltas between original→golden and original→actual.
- n-grams are computed as in chrF (max n=6, whitespace ignored).
3. Compare these deltas to assess how well the actual edits match the expected edits.
- As in standard chrF, classify n-grams as true positives, false positives, and false negatives, and report the F-beta score with beta=2.
Release Notes:
- N/A
We've realized that a lot of the logic within an
`EditPredictionProvider` is not specific to a particular edit prediction
model / service. Rather, it is just the generic state management
required to perform edit predictions at all in Zed. We want to move to a
setup where there's one "built-in" edit prediction provider in Zed,
which can be pointed at different edit prediction models. The only logic
that is different for different models is how we construct the prompt,
send the request, and parse the output.
This PR also changes the behavior of the staff-only `zeta2` feature flag
so that in only gates your *ability* to use Zeta2, but you can still use
your local settings file to choose between different edit prediction
models/services: zeta1, zeta2, and sweep.
This PR also makes zeta1's outcome reporting and prediction-rating
features work with all prediction models, not just zeta1.
To do:
* [x] remove duplicated logic around sending cloud requests between
zeta1 and zeta2
* [x] port the outcome reporting logic from zeta to zeta2.
* [x] get the "rate completions" modal working with all EP models
* [x] display edit prediction diff
* [x] show edit history events
* [x] remove the original `zeta` crate.
Release Notes:
- N/A
---------
Co-authored-by: Agus Zubiaga <agus@zed.dev>
Co-authored-by: Ben Kunkle <ben@zed.dev>
Deals with https://github.com/zed-industries/zed/issues/5259
Highlights brackets with different colors based on their depth.
Uses existing tree-sitter queries from brackets.scm to find brackets,
uses theme's accents to color them.
https://github.com/user-attachments/assets/cc5f3aba-22fa-446d-9af7-ba6e772029da
1. Adds `colorize_brackets` language setting that allows, per language
or globally for all languages, to configure whether Zed should color the
brackets for a particular language.
Disabled for all languages by default.
2. Any given language can opt-out a certain bracket pair by amending the
brackets.scm like `("\"" @open "\"" @close) ` -> `(("\"" @open "\""
@close) (#set! rainbow.exclude))`
3. Brackets are using colors from theme accents, which can be overridden
as
```jsonc
"theme_overrides": {
"One Dark": {
"accents": ["#ff69b4", "#7fff00", "#ff1493", "#00ffff", "#ff8c00", "#9400d3"]
}
},
```
Release Notes:
- Added bracket colorization (rainbow brackets) support. Use
`colorize_brackets` language setting to enable.
---------
Co-authored-by: MrSubidubi <dev@bahn.sh>
Co-authored-by: Lukas Wirth <lukas@zed.dev>
Co-authored-by: MrSubidubi <finn@zed.dev>
Co-authored-by: Lukas Wirth <me@lukaswirth.dev>
Co-authored-by: Smit Barmase <heysmitbarmase@gmail.com>
This PR introduces a new `MultiBufferOffset` new type wrapping size. The
goal of this is to make it clear at the type level when we are
interacting with offsets of a multi buffer versus offsets of a language
/ text buffer. This improves readability of things quite a bit by making
it clear what kind of offsets one is working with while also reducing
accidental bugs by using the wrong kin of offset for the wrong API.
This PR also uncovered two minor bugs due to that.
Does not yet introduce the MultiBufferPoint equivalent, that is for a
follow up PR.
Release Notes:
- N/A *or* Added/Fixed/Improved ...
We now correctly use bracket ranges from the deepest syntax layer when
finding enclosing brackets.
Release Notes:
- Fixed an issue where `MoveToEnclosingBracket` didn’t work correctly
inside Markdown code blocks.
- Fixed an issue where unmatched forward/backward Vim motions didn’t
work correctly inside Markdown code blocks.
---------
Co-authored-by: MuskanPaliwal <muskan10112002@gmail.com>
Release Notes:
- Adds support for `:[range]w {file}`
- This writes the lines in the range to the specified
- Adds support for `:[range]w`
- This replaces the current file with the selected lines
Since we removed the filtering step during context gathering, we want
the model to perform more targeted searches. This PR tweaks search tool
schema allowing the model to search within syntax nodes such as `impl`
blocks or methods.
This is what the query schema looks like now:
```rust
/// Search for relevant code by path, syntax hierarchy, and content.
#[derive(Debug, Clone, Serialize, Deserialize, JsonSchema)]
pub struct SearchToolQuery {
/// 1. A glob pattern to match file paths in the codebase to search in.
pub glob: String,
/// 2. Regular expressions to match syntax nodes **by their first line** and hierarchy.
///
/// Subsequent regexes match nodes within the full content of the nodes matched by the previous regexes.
///
/// Example: Searching for a `User` class
/// ["class\s+User"]
///
/// Example: Searching for a `get_full_name` method under a `User` class
/// ["class\s+User", "def\sget_full_name"]
///
/// Skip this field to match on content alone.
#[schemars(length(max = 3))]
#[serde(default)]
pub syntax_node: Vec<String>,
/// 3. An optional regular expression to match the final content that should appear in the results.
///
/// - Content will be matched within all lines of the matched syntax nodes.
/// - If syntax node regexes are provided, this field can be skipped to include as much of the node itself as possible.
/// - If no syntax node regexes are provided, the content will be matched within the entire file.
pub content: Option<String>,
}
```
We'll need to keep refining this, but the core implementation is ready.
Release Notes:
- N/A
---------
Co-authored-by: Ben <ben@zed.dev>
Co-authored-by: Max <max@zed.dev>
Co-authored-by: Max Brunsfeld <maxbrunsfeld@gmail.com>
Closes#40632
**TL;DR:** The `wrap selections in tag` action was unavailable in ERB
files, even when the cursor was positioned in HTML content (outside of
Ruby code blocks). This happened because `syntax_layer_at()` incorrectly
returned the Ruby language for positions that were actually in HTML.
**NOTE:** I am not familiar with that part of Zed so it could be that
the fix here is completely incorrect.
Previously, `syntax_layer_at` incorrectly reported injected languages
(e.g., Ruby in ERB files) even when the cursor was in the base language
content (HTML). This broke actions like `wrap selections in tag` that
depend on language-specific configuration.
The issue had two parts:
1. Missing start boundary check: The filter only checked if a layer's
end was after the cursor (`end_byte() > offset`), not if it started
before, causing layers outside the cursor position to be included. See
the `BEFORE` video: when I click on the HTML part it reports `Ruby`
language instead of `HTML`.
2. Wrong boundary reference for injections: For injected layers with
`included_sub_ranges` (like Ruby code blocks in ERB), checking the root
node boundaries returned the entire file range instead of the actual
injection ranges.
This fix:
- Adds the containment check using half-open range semantics [start,
end) for root node boundaries. That ensures proper reporting of the
detected language when a cursor (`|`) is located right after the
injection:
```
<body>
<%= yield %>|
</body>
```
- Checks `included_sub_ranges` for injected layers to determine if the
cursor is actually within an injection
- Falls back to root node boundaries for base layers without sub-ranges.
This is the original behavior.
Fixes ERB language support where actions should be available based on
the cursor's actual language context. I think that also applies to some
other template languages like HEEX (Phoenix) and `*.pug`. On short
videos below you can see how I navigate through the ERB template and the
terminal on the right outputs the detected language if you apply the
following patch:
```diff
diff --git i/crates/editor/src/editor.rs w/crates/editor/src/editor.rs
index 15af61f5d2..54a8e0ae37 100644
--- i/crates/editor/src/editor.rs
+++ w/crates/editor/src/editor.rs
@@ -10671,6 +10671,7 @@ impl Editor {
for selection in self.selections.disjoint_anchors_arc().iter() {
if snapshot
.language_at(selection.start)
+ .inspect(|language| println!("Detected language: {:?}", language))
.and_then(|lang| lang.config().wrap_characters.as_ref())
.is_some()
{
```
**Before:**
https://github.com/user-attachments/assets/3f8358f4-d343-462e-b6b1-3f1f2e8c533d
**After:**
https://github.com/user-attachments/assets/c1b9f065-1b44-45a2-8a24-76b7d812130d
Here is the ERB template:
```
<!DOCTYPE html>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<style>
/* Email styles need to be inline */
</style>
</head>
<body>
<%= yield %>
</body>
</html>
```
Release Notes:
- N/A
When doing a project wide search in zed on windows for `hang`, zed
starts to freeze for a couple seconds ultimately starting to error with
`Not enough quota is available to process this command.` when
dispatching windows messages. The cause for this is that we simply
overload the windows message pump due to the sheer amount of foreground
tasks we spawn when we populate the project search.
This PR is an attempt at reducing this.
Release Notes:
- Reduced hangs and stutters in large project file searches
Closes#40581
Release Notes:
- git: No longer save clean files when staging (to avoid triggering
unnecessary rebuilds in external file watchers like vite)
This simplifies some code and is also more correct in some others (I
believe some of these might've overflowed causing panics in sentry)
Release Notes:
- N/A *or* Added/Fixed/Improved ...
- Notable change is the use of a newtype for `ReplicaId`
- Fixes `WorktreeStore::create_remote_worktree` creating a remote
worktree with the local replica id, though this is not currently used
- Fixes observing the `Agent` (that is following the agent) causing
global clocks to allocate 65535 elements
- Shrinks the size of `Global` a bit. In a local or non-collab remote
session it won't ever allocate still.
Release Notes:
- N/A *or* Added/Fixed/Improved ...
This removes a long-standing thing we've done, which is send a `DidSave`
notification to the language server for the clean parts of a
multi-buffer. However, it seems like the intent of that notification is
to tell the language server to reload the file from disk.
As we didn't actually write those files to disk, it seems clearer to not
send this notification; and just remove this whole code-path.
Release Notes:
- Fixed a race where autosave in a multibuffer could cause unsaved
buffers to appear saved
Make `resolve_full_path` use the appropriate separators, and return a
`String`.
As part of fixing the fallout from that type change, this also fixes a
bunch of places in the agent code that were using `std::path::Path`
operations on paths that could be non-local, by changing them to operate
instead on strings and use the project's `PathStyle`.
This clears the way a bit for making `full_path` also return a string
instead of a `PathBuf`, but I've left that for a follow-up.
Release Notes:
- N/A
Closes https://github.com/zed-industries/zed/issues/38690Closes#37353
### Background
On Windows, paths are normally separated by `\`, unlike mac and linux
where they are separated by `/`. When editing code in a project that
uses a different path style than your local system (e.g. remoting from
Windows to Linux, using WSL, and collaboration between windows and unix
users), the correct separator for a path may differ from the "native"
separator.
Previously, to work around this, Zed converted paths' separators in
numerous places. This was applied to both absolute and relative paths,
leading to incorrect conversions in some cases.
### Solution
Many code paths in Zed use paths that are *relative* to either a
worktree root or a git repository. This PR introduces a dedicated type
for these paths called `RelPath`, which stores the path in the same way
regardless of host platform, and offers `Path`-like manipulation APIs.
RelPath supports *displaying* the path using either separator, so that
we can display paths in a style that is determined at runtime based on
the current project.
The representation of absolute paths is left untouched, for now.
Absolute paths are different from relative paths because (except in
contexts where we know that the path refers to the local filesystem)
they should generally be treated as opaque strings. Currently we use a
mix of types for these paths (std::path::Path, String, SanitizedPath).
Release Notes:
- N/A
---------
Co-authored-by: Cole Miller <cole@zed.dev>
Co-authored-by: Piotr Osiewicz <24362066+osiewicz@users.noreply.github.com>
Co-authored-by: Peter Tripp <petertripp@gmail.com>
Co-authored-by: Smit Barmase <heysmitbarmase@gmail.com>
Co-authored-by: Lukas Wirth <me@lukaswirth.dev>
Co-Authored-By: Ben K <ben@zed.dev>
Co-Authored-By: Anthony <anthony@zed.dev>
Co-Authored-By: Mikayla <mikayla@zed.dev>
Release Notes:
- settings: Major internal changes to settings. The primary user-facing
effect is that some settings which did not make sense in project
settings files are no-longer read from there. (For example the inline
blame settings)
---------
Co-authored-by: Ben Kunkle <ben@zed.dev>
Co-authored-by: Mikayla Maki <mikayla.c.maki@gmail.com>
Co-authored-by: Anthony <anthony@zed.dev>
#35053 split out these utility functions. I found the names / doc
comments a bit confusing so this improves that. Before that PR there was
also a mild inefficiency - it would walk the cursor all the way down to
a leaf and then back up to an ancestor.
Release Notes:
- N/A
## Context
While looking into: #32051 and #16120 with instruments, I noticed that
`TabSnapshot::to_tab_point` and `TabSnapshot::to_fold_point` are a
common bottleneck between the two issues. This PR takes the first steps
into closing the stated issues by improving the performance of both
those functions.
### Method
`to_tab_point` and `to_fold_point` iterate through each character in
their rows to find tab characters and translate those characters into
their respective transformations. This PR changes this iteration to take
advantage of the tab character bitmap in the `Rope` data structure and
goes directly to each tab character when iterating.
The tab bitmap is now passed from each layer in-between the `Rope` to
the `TabMap`.
### Testing
I added several randomized tests to ensure that the new `to_tab_point`
and `to_fold_point` functions have the same behavior as the old methods
they're replacing. I also added `test_random_chunk_bitmap` on each layer
the tab bitmap is passed up to the `TabMap` to make sure that the bitmap
being passed is transformed correctly between the layers of
`DisplayMap`.
`test_random_chunk_bitmap` was added to these layers:
- buffer
- multi buffer
- custom_highlights
- inlay_map
- fold_map
## Benchmarking
I setup benchmarks with criterion that is runnable via `cargo bench -p
editor --profile=release-fast`. When benchmarking I had my laptop
plugged in and did so from the terminal with a minimal amount of
processes running. I'm also on a m4 max
### Results
#### To Tab Point
Went from completing 6.8M iterations in 5s with an average time of
`736.13 ns` to `683.38 ns` which is a `-7.1875%` improvement
#### To Fold Point
Went from completing 6.8M iterations in 5s with an average time of
`736.55 ns` to `682.40 ns` which is a `-7.1659%` improvement
#### Editor render
Went from having an average render time of `62.561 µs` to `57.216 µs`
which is a `-8.8248%` improvement
#### Build Buffer with one long line
Went from having an average buffer build time of `3.2549 ms` to `3.2635
ms` which is a `+0.2151%` regression within the margin of error
#### Editor with 1000 multi cursor input
Went from having an average edit time of `133.05 ms` to `122.96 ms`
which is a `-7.5776%` improvement
Release Notes:
- N/A
---------
Co-authored-by: Remco Smits <djsmits12@gmail.com>
Co-authored-by: Cole Miller <cole@zed.dev>
Co-authored-by: Piotr Osiewicz <24362066+osiewicz@users.noreply.github.com>
