This is necessary to remove some `util` dependencies from crates, as
well as better sharing for our projects. This also includes the WIP
AbsPath abstraction as well as some bug fixes from internal tooling.
Release Notes:
- N/A or Added/Fixed/Improved ...
Closes EP-193
Edit prediction context collection previously reused the editor
goto-definition / goto-type-definition path, which resolves every LSP
result into a `LocationLink` — opening a buffer per target (worktree
creation, buffer registration, anchor conversion) before edit prediction
gets a chance to filter. On remote projects the host additionally
created a peer-visible buffer per link and the client waited on each
one.
This adds a bespoke request path that returns raw results first and only
does the expensive work for results that survive filtering:
- New `EditPredictionDefinition { path: ProjectPath, range:
Range<Unclipped<PointUtf16>> }` boundary type. LSP wire types never
leave `lsp_command.rs`; the workspace-only filter and URI→`ProjectPath`
resolution happen while normalizing the LSP response, before any buffer
exists, so the type encodes the workspace-only invariant.
- New `GetEditPredictionDefinitions` /
`GetEditPredictionTypeDefinitions` LSP commands and proto messages.
Responses carry only path + UTF-16 range — the host never calls
`create_buffer_for_peer` and the client never waits on remote buffers.
- One public API: `Project::edit_prediction_definitions(buffer,
position, include_type_definitions, cx)` fires both LSP requests
concurrently and returns a merged, deduped list ("not capable" = empty).
The definition/type-definition split was never used downstream, so
`CacheEntry` now holds a single list and the fetch pipeline runs one
task per identifier.
- `edit_prediction_context` dedupes raw results before opening buffers;
survivors open via `project.open_buffer(ProjectPath)` (skipping the
invisible-worktree/yarn machinery of `open_local_buffer_via_lsp`, and
working identically on remote projects), then clip → anchor →
`MAX_TARGET_LEN`.
- Removes the `workspace_only` flag from `GetDefinitions` /
`GetTypeDefinitions`: edit prediction was its only user. The editor path
always sent `false`, which proto3 doesn't encode, so normal
goto-definition requests are wire-identical; old peers still sending
`true` get unfiltered results (graceful degradation). The proto field
numbers are `reserved`.
Tested with a local test proving filtering precedes buffer opening (an
out-of-workspace target never spawns a worktree, an oversized target is
excluded from related files) and a collab test proving the remote
round-trip returns correct paths/ranges without opening buffers on the
client.
Release Notes:
- N/A
# Objective
Closes EP-9
- Describe the objective or issue this PR addresses.
- If you're fixing a specific issue, use "Fixes #X" for each issue as
[described in the GitHub
docs](https://docs.github.com/en/issues/tracking-your-work-with-issues/using-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword).
## Solution
- Describe the solution used to achieve the objective above.
## Testing
- Did you test these changes? If so, how?
- Are there any parts that need more testing?
- How can other people (reviewers) test your changes? Is there anything
specific they need to know?
- If relevant, what platforms did you test these changes on, and are
there any important ones you can't test?
## Self-Review Checklist:
- [x] I've reviewed my own diff for quality, security, and reliability
- [x] Unsafe blocks (if any) have justifying comments
- [x] The content adheres to Zed's UI standards
([UX/UI](https://github.com/zed-industries/zed/blob/main/CONTRIBUTING.md#uiux-checklist)
and
[icon](https://github.com/zed-industries/zed/blob/main/crates/icons/README.md)
guidelines)
- [x] Tests cover the new/changed behavior
- [x] Performance impact has been considered and is acceptable
## Showcase
> This section is optional. If this PR does not include a visual change
or does not add a new user-facing feature, you can delete this section.
- Help others understand the result of this PR by showcasing your
awesome work!
- If this PR includes a visual change, consider adding a screenshot,
GIF, or video
- A before/after comparison is very useful for changes to existing
features!
While a showcase should aim to be brief and digestible, you can use a
toggleable section to save space on longer showcases:
<details>
<summary>Click to view showcase</summary>
My super cool demos here
</details>
---
Release Notes:
- N/A or Added/Fixed/Improved ...
---------
Co-authored-by: Oleksiy Syvokon <oleksiy.syvokon@gmail.com>
Co-authored-by: Oleksiy Syvokon <oleksiy@zed.dev>
Self-Review Checklist:
- [x] I've reviewed my own diff for quality, security, and reliability
- [x] Unsafe blocks (if any) have justifying comments
- [x] The content adheres to Zed's UI standards
([UX/UI](https://github.com/zed-industries/zed/blob/main/CONTRIBUTING.md#uiux-checklist)
and
[icon](https://github.com/zed-industries/zed/blob/main/crates/icons/README.md)
guidelines)
- [x] Tests cover the new/changed behavior
- [x] Performance impact has been considered and is acceptable
Closes #ISSUE
Release Notes:
- N/A or Added/Fixed/Improved ...
- Replace full sort and truncate with a budgeted top-N approach for
context identifiers using truncate_to_bottom_n_sorted_by.
- Introduce MAX_DIAGNOSTICS = 10 and prune diagnostics to this limit
with select_nth_unstable_by_key, then sort the remainder.
Self-Review Checklist:
- [x] I've reviewed my own diff for quality, security, and reliability
- [x] Unsafe blocks (if any) have justifying comments
- [x] The content adheres to Zed's UI standards
([UX/UI](https://github.com/zed-industries/zed/blob/main/CONTRIBUTING.md#uiux-checklist)
and
[icon](https://github.com/zed-industries/zed/blob/main/crates/icons/README.md)
guidelines)
- [x] Tests cover the new/changed behavior
- [x] Performance impact has been considered and is acceptable
Release Notes:
- Removed small allocations on the edit_predictions critical path by
using select_nth and sort_unstable
Co-authored-by: Nathan Barry <nathanbarry474@gmail.com>
Co-authored-by: Anthony Eid <hello@anthonyeid.me>
This change adds metrics for evaluating editable context collectors and
the collectors themselves:
- `cursor excerpt`: small area around the cursor.
- `current-file`: the full file that contains the cursor.
- `edit-history`: excerpts around edit history edits.
- `edit-history-file`: full files referenced by the edit history.
- `git-log`: files related to the cursor file according to recent git
co-change history.
- `oracle-file`: full files referenced by the expected patches. This is
an oracle/evaluation-only source because it uses the target patch.
Individual collectors can be enabled by passing a comma-separated list
like this: `ep context --type=edit-history,current-file`. There's also a
preset, `--type=editable`, which will enable all collectors except
oracle.
This change also adds the
`.prompt_input.related_files[0].excerpts[0].context_source` field.
Editable context is only accessible via ep-cli for now.
Release Notes:
- N/A
---------
Co-authored-by: Eric Holk <eric@zed.dev>
Self-Review Checklist:
- [x] I've reviewed my own diff for quality, security, and reliability
- [x] Unsafe blocks (if any) have justifying comments
- [x] The content is consistent with the [UI/UX
checklist](https://github.com/zed-industries/zed/blob/main/CONTRIBUTING.md#uiux-checklist)
- [ ] Tests cover the new/changed behavior
- [x] Performance impact has been considered and is acceptable
Closes #ISSUE
Release Notes:
- N/A or Added/Fixed/Improved ...
No, sadly, the title is not a typo. See
https://www.githubstatus.com/incidents/zsg1lk7w13cf for the context.
I'll read with joy and popcorn through that root cause analysis.
It makes literally zero sense what happened here, but for some completly
bonkers reason GitHub completely messed up the merge queue with
https://github.com/zed-industries/zed/pull/54632.
I have no idea how it happened. It makes literally zero sense. A PR
going into the merge queue should have the same LoC when getting out of
it. GitHub obviously does not check this. GitHub causes extra work with
a feature that is supposed to save time.
Thanks, I guess.
Release Notes:
- N/A
---------
Co-authored-by: Danilo Leal <daniloleal09@gmail.com>
This PR brings back the button to filter remote branches when accessing
the title bar's branch picker with the mouse. It was unintentionally
removed when we introduced the new worktree picker.
Release Notes:
- N/A
Self-Review Checklist:
- [x] I've reviewed my own diff for quality, security, and reliability
- [x] Unsafe blocks (if any) have justifying comments
- [x] The content is consistent with the [UI/UX
checklist](https://github.com/zed-industries/zed/blob/main/CONTRIBUTING.md#uiux-checklist)
- [ ] Tests cover the new/changed behavior
- [x] Performance impact has been considered and is acceptable
Closes #ISSUE
Release Notes:
- N/A or Added/Fixed/Improved ...
Skip intrinsic TSX/JSX tag names when gathering identifiers for edit
prediction context.
I bisected a TypeScript regression to
https://github.com/zed-industries/zed/pull/49748. That change caused
edit prediction context to query `definition` / `typeDefinition` for
lowercase JSX tags like `p`, which could trigger incorrect unversioned
TypeScript diagnostics, such as `Cannot find name 'p'`.
This change filters those intrinsic tag names out while still keeping
user-defined component tags like `Component`.
Self-Review Checklist:
- [x] I've reviewed my own diff for quality, security, and reliability
- [x] Unsafe blocks (if any) have justifying comments
- [x] The content is consistent with the [UI/UX
checklist](https://github.com/zed-industries/zed/blob/main/CONTRIBUTING.md#uiux-checklist)
- [ ] Tests cover the new/changed behavior
- [x] Performance impact has been considered and is acceptable
Closes#51316
Release Notes:
- Fixed a TypeScript issue that could show incorrect diagnostics caused
by edit prediction context lookups.
Co-authored-by: Ben Kunkle <ben@zed.dev>
This extracts a `language_core` crate from the existing `language`
crate, and creates a `grammars` data crate. The goal is to separate
tree-sitter grammar infrastructure, language configuration, and LSP
adapter types from the heavier buffer/editor integration layer in
`language`.
## Motivation
The `language` crate pulls in `text`, `theme`, `settings`, `rpc`,
`task`, `fs`, `clock`, `sum_tree`, and `fuzzy` — all of which are needed
for buffer integration (`Buffer`, `SyntaxMap`, `Outline`,
`DiagnosticSet`) but not for grammar parsing or language configuration.
Extracting the core types lets downstream consumers depend on
`language_core` without pulling in the full integration stack.
## Dependency graph after extraction
```
language_core ← gpui, lsp, tree-sitter, util, collections
grammars ← language_core, rust_embed, tree-sitter-{rust,python,...}
language ← language_core, text, theme, settings, rpc, task, fs, ...
languages ← language, grammars
```
## What moved to `language_core`
- `Grammar`, `GrammarId`, and all query config/builder types
- `LanguageConfig`, `LanguageMatcher`, bracket/comment/indent config
types
- `HighlightMap`, `HighlightId` (theme-dependent free functions
`highlight_style` and `highlight_name` stay in `language`)
- `LanguageName`, `LanguageId`
- `LanguageQueries`, `QUERY_FILENAME_PREFIXES`
- `CodeLabel`, `CodeLabelBuilder`, `Symbol`
- `Diagnostic`, `DiagnosticSourceKind`
- `Toolchain`, `ToolchainScope`, `ToolchainList`, `ToolchainMetadata`
- `ManifestName`
- `SoftWrap`
- LSP data types: `BinaryStatus`, `ServerHealth`,
`LanguageServerStatusUpdate`, `PromptResponseContext`, `ToLspPosition`
## What stays in `language`
- `Buffer`, `BufferSnapshot`, `SyntaxMap`, `Outline`, `DiagnosticSet`,
`LanguageScope`
- `LspAdapter`, `CachedLspAdapter`, `LspAdapterDelegate` (reference
`Arc<Language>` and `WorktreeId`)
- `ToolchainLister`, `LanguageToolchainStore` (reference `task` and
`settings` types)
- `ManifestQuery`, `ManifestProvider`, `ManifestDelegate` (reference
`WorktreeId`)
- Parser/query cursor pools, `PLAIN_TEXT`, point conversion functions
## What the `grammars` crate provides
- Embedded `.scm` query files and `config.toml` files for all built-in
languages (via `rust_embed`)
- `load_queries(name)`, `load_config(name)`,
`load_config_for_feature(name, grammars_loaded)`, and `get_file(path)`
functions
- `native_grammars()` for tree-sitter grammar registration (behind
`load-grammars` feature)
## Pre-cleanup (also in this PR)
- Removed unused `Option<&Buffer>` from
`LspAdapter::process_diagnostics`
- Removed unused `&App` from `LspAdapter::retain_old_diagnostic`
- Removed `fs: &dyn Fs` from `ToolchainLister` trait methods
(`PythonToolchainProvider` captures `fs` at construction time instead)
- Moved `Diagnostic`/`DiagnosticSourceKind` out of `buffer.rs` into
their own module
## Backward compatibility
The `language` crate re-exports everything from `language_core`, so
existing `use language::Grammar` (etc.) continues to work unchanged. The
only downstream change required is importing `CodeLabelExt` where
`.fallback_for_completion()` is called on the now-foreign `CodeLabel`
type.
Release Notes:
- N/A
---------
Co-authored-by: Agus Zubiaga <agus@zed.dev>
Co-authored-by: Tom Houlé <tom@tomhoule.com>
### Context
This fixes a bug where the sidebar would show a newly created git
worktree thread as its own project while the recently created workspace
is loading its git state. The fix is adding project APIs to await the
initial worktree store scan and then each git repo initial snapshot;
then awaiting on them before adding the new workspace to the
multi-workspace.
### Architecture:
I added the `Worktree::Remote::wait_for_snapshot` API to
`Worktree::Local` to enable `WorktreeStore` to await for both remote and
local projects until there's an initial scan. The `WorktreeStore` uses
the watcher pattern so it can update the initial scan state whenever
visible worktrees are added or removed from the store.
Before you mark this PR as ready for review, make sure that you have:
- [x] Added solid test coverage and/or screenshots from doing manual
testing
- [x] Done a self-review taking into account security and performance
aspects
- [x] Aligned any UI changes with the [UI
checklist](https://github.com/zed-industries/zed/blob/main/CONTRIBUTING.md#uiux-checklist)
Release Notes:
- N/A
---------
Co-authored-by: Ben Kunkle <ben@zed.dev>
Co-authored-by: cameron <cameron.studdstreet@gmail.com>
Co-authored-by: Zed Zippy <234243425+zed-zippy[bot]@users.noreply.github.com>
This will help with test times (in some cases), as nextest cannot figure
out whether a given rdep is actually an alive edge of the build graph
Closes #ISSUE
Before you mark this PR as ready for review, make sure that you have:
- [ ] Added a solid test coverage and/or screenshots from doing manual
testing
- [ ] Done a self-review taking into account security and performance
aspects
- [ ] Aligned any UI changes with the [UI
checklist](https://github.com/zed-industries/zed/blob/main/CONTRIBUTING.md#uiux-checklist)
Release Notes:
- N/A
We store the byte distance between the cursor and references to each
definition. When including excerpts in the prompt, we prioritize them in
the order of proximity. I've updated the Edit Prediction Context view to
display the excerpt's `order`, and sorting the files in order of their
excerpt with the lowest order.
Release Notes:
- N/A
This fixes an issue that we noticed in particular with Mercury edit
predictions.
* [x] fix storage to not go stale
* [x] exclude excerpts that intersect the cursor excerpt
* [x] see if string representation of excerpts can be cached, to avoid
rebuilding it on every prediction
Release Notes:
- N/A
---------
Co-authored-by: Ben Kunkle <ben@zed.dev>
This PR restructures the commands of the Edit Prediction CLI (now called
`ep`), to support some flows that are important for the training
process:
* generating zeta2 prompt and expected output, without running
predictions
* scoring outputs that are generated by a system other than the
production code (to evaluate the model during training)
To achieve this, we've restructured the CLI commands so that they all
take as input, and produce as output, a consistent, uniform data format:
a set of one or more `Example` structs, expressible either as the
original markdown format, or as a JSON lines. The `Example` struct
starts with the basic fields that are in human-readable eval format, but
contain a number of optional fields that are filled in by different
steps in the processing pipeline (`context`, `predict`, `format-prompt`,
and `score`).
### To do
* [x] Adjust the teacher model output parsing to use the full buffer
contents
* [x] Move udiff to cli
* [x] Align `format-prompt` with Zeta2's production code
* [x] Change score output to assume same provider
* [x] Move pretty reporting to `eval` command
* [x] Store cursor point in addition to cursor offset
* [x] Rename `edit_prediction_cli2` -> `edit_prediction_cli` (nuke the
old one)
Release Notes:
- N/A
---------
Co-authored-by: Oleksiy Syvokon <oleksiy@zed.dev>
Co-authored-by: Agus Zubiaga <agus@zed.dev>
Co-authored-by: Ben Kunkle <ben@zed.dev>
For qualified identifiers we end up requesting both the definition of
the module and the item within it, but we only want the latter. At the
moment, we can't skip the request altogether, because we can't tell them
apart from the highlights query. However, we can tell from the target
range length, because it should be small for individual definitions as
it only covers their name, not the whole body.
Release Notes:
- N/A
Gist is we only need to block the foreground thread for reparsing if
immediate language changes are useful to the user. That is usually only
the case when they edit the buffer
Release Notes:
- Improved performance of large project searches and project diffs
Co-authored by: David Kleingeld <david@zed.dev>
I noticed we had some typos that were getting through CI, but it looks
like the new version of `typos` catches them. So I updated it and fixed
them.
Release Notes:
- N/A
We've been considering removing workspace-hack for a couple reasons:
- Lukas ran into a situation where its build script seemed to be causing
spurious rebuilds. This seems more likely to be a cargo bug than an
issue with workspace-hack itself (given that it has an empty build
script), but we don't necessarily want to take the time to hunt that
down right now.
- Marshall mentioned hakari interacts poorly with automated crate
updates (in our case provided by rennovate) because you'd need to have
`cargo hakari generate && cargo hakari manage-deps` after their changes
and we prefer to not have actions that make commits.
Currently removing workspace-hack causes our workspace to grow from
~1700 to ~2000 crates being built (depending on platform), which is
mainly a problem when you're building the whole workspace or running
tests across the the normal and remote binaries (which is where
feature-unification nets us the most sharing). It doesn't impact
incremental times noticeably when you're just iterating on `-p zed`, and
we'll hopefully get these savings back in the future when
rust-lang/cargo#14774 (which re-implements the functionality of hakari)
is finished.
Release Notes:
- N/A
Adds a new `NumberedLines` format which is similar to `MarkedExcerpt`
but each line is prefixed with its line number.
Also fixes a bug where contagious snippets wouldn't get merged.
Release Notes:
- N/A
---------
Co-authored-by: Michael Sloan <mgsloan@gmail.com>
Co-authored-by: Michael <michael@zed.dev>
Before this change, it would save every buffer and wait for diagnostics.
For rust analyzer this would cause a lot of rechecking and greatly slow
down the analysis
Release Notes:
- N/A
Co-authored-by: Agus <agus@zed.dev>
Also skips indexing files that don't have a suffix that indicates a
known language, and skips when the language doesn't have an outline
grammar.
Release Notes:
- N/A
---------
Co-authored-by: Agus <agus@zed.dev>
Closes https://github.com/zed-industries/zed/issues/38690Closes#37353
### Background
On Windows, paths are normally separated by `\`, unlike mac and linux
where they are separated by `/`. When editing code in a project that
uses a different path style than your local system (e.g. remoting from
Windows to Linux, using WSL, and collaboration between windows and unix
users), the correct separator for a path may differ from the "native"
separator.
Previously, to work around this, Zed converted paths' separators in
numerous places. This was applied to both absolute and relative paths,
leading to incorrect conversions in some cases.
### Solution
Many code paths in Zed use paths that are *relative* to either a
worktree root or a git repository. This PR introduces a dedicated type
for these paths called `RelPath`, which stores the path in the same way
regardless of host platform, and offers `Path`-like manipulation APIs.
RelPath supports *displaying* the path using either separator, so that
we can display paths in a style that is determined at runtime based on
the current project.
The representation of absolute paths is left untouched, for now.
Absolute paths are different from relative paths because (except in
contexts where we know that the path refers to the local filesystem)
they should generally be treated as opaque strings. Currently we use a
mix of types for these paths (std::path::Path, String, SanitizedPath).
Release Notes:
- N/A
---------
Co-authored-by: Cole Miller <cole@zed.dev>
Co-authored-by: Piotr Osiewicz <24362066+osiewicz@users.noreply.github.com>
Co-authored-by: Peter Tripp <petertripp@gmail.com>
Co-authored-by: Smit Barmase <heysmitbarmase@gmail.com>
Co-authored-by: Lukas Wirth <me@lukaswirth.dev>
The edit prediction debug tools has been renamed to zeta2 inspector
because it's now zeta specific. It will now always display the last
prediction request context, prompt, and model response.
Release Notes:
- N/A
---------
Co-authored-by: Bennet <bennet@zed.dev>
Co-authored-by: Bennet Bo Fenner <bennetbo@gmx.de>
Creates a new `EditPredictionProvider` for zeta2, that requests
completions from a new cloud endpoint including context from the new
`edit_prediction_context` crate. This is not ready for use, but it
allows us to iterate.
Release Notes:
- N/A
---------
Co-authored-by: Michael Sloan <michael@zed.dev>
Co-authored-by: Bennet <bennet@zed.dev>
Co-authored-by: Bennet Bo Fenner <bennetbo@gmx.de>
Lukas Wirth