Closes#60424
## Problem
With the repro from #60424, staging the first deletion hunk marked the
second deletion as staged in the UI even though git still had it
unstaged, and clicking the (incorrectly shown) Unstage button inserted a
duplicate copy of the hunk's contents into the git index on every click.
The root cause is ambiguous hunk placement. The committed text contains
repeated `end\n\n` line runs, so the deletion hunks can "slide": more
than one placement produces a minimal diff. The uncommitted diff (HEAD
vs worktree) anchored the remaining deletion at one row while the
unstaged diff (index vs worktree), recomputed after the partial stage,
anchored the same logical deletion at a different row. Everything that
correlates hunks across those two diffs assumes they agree on positions:
- the secondary-status matching in `hunks_intersecting_range_impl` found
no unstaged hunk at the uncommitted hunk's rows and reported it as
staged (`NoSecondaryHunk`);
- the worktree→index projection in `compute_uncommitted_index_edits`
treated the hunk's position as unchanged text and, on unstage, inserted
the hunk's HEAD content at an index position that already contained it,
duplicating it on every request.
## Fix
Upgrade `imara-diff` from 0.1.8 to 0.2.0 and run
`Diff::postprocess_lines` (imara-diff's port of git's xdiff
slider/indent heuristic) after computing hunks in `buffer_diff`. This
canonicalizes the placement of ambiguous hunks based only on their local
content, so diffs of the same buffer against different base texts anchor
the same logical change at the same rows. A bonus is that Zed's hunk
placement now matches `git diff`'s output for such cases (git has used
the indent heuristic by default since 2.11).
As defense in depth, `compute_uncommitted_index_edits` now drops a
pure-insertion index edit whose content is already present at the target
position, so a stale secondary status can no longer duplicate index
content.
The imara-diff 0.2 API removed the `Sink` trait and the top-level
`diff()` function, so the other call sites (`language/text_diff.rs`,
`zeta_prompt/udiff.rs`, `edit_prediction_metrics/reversal.rs`) are
migrated mechanically to `Diff::compute` + `hunks()` with unchanged
behavior (0.2 also renamed `lines_with_terminator` to `lines` and
changed the default `&str` tokenization to include terminators; the
unified-diff builders keep terminator-less tokens via `str::lines()`).
## Testing
- New regression test `test_staging_hunks_with_ambiguous_placement`
replays the exact repro from #60424 (same file contents): stages the
first deletion, asserts the remaining hunks keep their unstaged status
and the index matches exactly, then issues repeated unstage requests for
the unstaged hunk and asserts the index is unchanged. Before the fix
this test showed the second hunk flipping to staged and the index
growing by one copy of the deleted block per unstage request.
- `buffer_diff`, `language`, `zeta_prompt`, `edit_prediction_metrics`,
`multi_buffer`, `git_ui`, `editor`, and the `project` integration suite
pass.
- One test expectation updated: `editor::test_fold_function_bodies`
asserted the old placement of an ambiguous deletion (blank line before
comment); the canonicalized placement (comment before blank line)
matches what `git diff` produces for the same texts.
Release Notes:
- Fixed staging a hunk sometimes marking a different hunk as staged (and
subsequent unstaging corrupting the git index) when the diff contained
repeated lines
([#60424](https://github.com/zed-industries/zed/issues/60424)).
---------
Co-authored-by: Cole Miller <cole@zed.dev>
- [x] I've reviewed my own diff for quality, security, and reliability
- [x] Unsafe blocks (if any) have justifying comments
- [x] The content adheres to Zed's UI standards
([UX/UI](https://github.com/zed-industries/zed/blob/main/CONTRIBUTING.md#uiux-checklist)
and
[icon](https://github.com/zed-industries/zed/blob/main/crates/icons/README.md)
guidelines)
- [x] Tests cover the new/changed behavior
- [x] Performance impact has been considered and is acceptable
The fight against entropy continues...
Release Notes:
- N/A or Added/Fixed/Improved ...
# Objective
Part of EP-10
Closes EP-118
- Describe the objective or issue this PR addresses.
- If you're fixing a specific issue, use "Fixes #X" for each issue as
[described in the GitHub
docs](https://docs.github.com/en/issues/tracking-your-work-with-issues/using-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword).
## Solution
- Describe the solution used to achieve the objective above.
## Testing
- Did you test these changes? If so, how?
- Are there any parts that need more testing?
- How can other people (reviewers) test your changes? Is there anything
specific they need to know?
- If relevant, what platforms did you test these changes on, and are
there any important ones you can't test?
## Self-Review Checklist:
- [x] I've reviewed my own diff for quality, security, and reliability
- [x] Unsafe blocks (if any) have justifying comments
- [x] The content adheres to Zed's UI standards
([UX/UI](https://github.com/zed-industries/zed/blob/main/CONTRIBUTING.md#uiux-checklist)
and
[icon](https://github.com/zed-industries/zed/blob/main/crates/icons/README.md)
guidelines)
- [x] Tests cover the new/changed behavior
- [x] Performance impact has been considered and is acceptable
## Showcase
> This section is optional. If this PR does not include a visual change
or does not add a new user-facing feature, you can delete this section.
- Help others understand the result of this PR by showcasing your
awesome work!
- If this PR includes a visual change, consider adding a screenshot,
GIF, or video
- A before/after comparison is very useful for changes to existing
features!
While a showcase should aim to be brief and digestible, you can use a
toggleable section to save space on longer showcases:
<details>
<summary>Click to view showcase</summary>
My super cool demos here
</details>
---
Release Notes:
- N/A or Added/Fixed/Improved ...
Co-authored-by: Nbarry <nbarry@zed.dev>
# Objective
Closes EP-9
- Describe the objective or issue this PR addresses.
- If you're fixing a specific issue, use "Fixes #X" for each issue as
[described in the GitHub
docs](https://docs.github.com/en/issues/tracking-your-work-with-issues/using-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword).
## Solution
- Describe the solution used to achieve the objective above.
## Testing
- Did you test these changes? If so, how?
- Are there any parts that need more testing?
- How can other people (reviewers) test your changes? Is there anything
specific they need to know?
- If relevant, what platforms did you test these changes on, and are
there any important ones you can't test?
## Self-Review Checklist:
- [x] I've reviewed my own diff for quality, security, and reliability
- [x] Unsafe blocks (if any) have justifying comments
- [x] The content adheres to Zed's UI standards
([UX/UI](https://github.com/zed-industries/zed/blob/main/CONTRIBUTING.md#uiux-checklist)
and
[icon](https://github.com/zed-industries/zed/blob/main/crates/icons/README.md)
guidelines)
- [x] Tests cover the new/changed behavior
- [x] Performance impact has been considered and is acceptable
## Showcase
> This section is optional. If this PR does not include a visual change
or does not add a new user-facing feature, you can delete this section.
- Help others understand the result of this PR by showcasing your
awesome work!
- If this PR includes a visual change, consider adding a screenshot,
GIF, or video
- A before/after comparison is very useful for changes to existing
features!
While a showcase should aim to be brief and digestible, you can use a
toggleable section to save space on longer showcases:
<details>
<summary>Click to view showcase</summary>
My super cool demos here
</details>
---
Release Notes:
- N/A or Added/Fixed/Improved ...
---------
Co-authored-by: Oleksiy Syvokon <oleksiy.syvokon@gmail.com>
Co-authored-by: Oleksiy Syvokon <oleksiy@zed.dev>
Self-Review Checklist:
- [x] I've reviewed my own diff for quality, security, and reliability
- [x] Unsafe blocks (if any) have justifying comments
- [x] The content is consistent with the [UI/UX
checklist](https://github.com/zed-industries/zed/blob/main/CONTRIBUTING.md#uiux-checklist)
- [x] Tests cover the new/changed behavior
- [x] Performance impact has been considered and is acceptable
Closes #ISSUE
Release Notes:
- N/A or Added/Fixed/Improved ...
Self-Review Checklist:
- [x] I've reviewed my own diff for quality, security, and reliability
- [x] Unsafe blocks (if any) have justifying comments
- [x] The content adheres to Zed's UI standards
([UX/UI](https://github.com/zed-industries/zed/blob/main/CONTRIBUTING.md#uiux-checklist)
and
[icon](https://github.com/zed-industries/zed/blob/main/crates/icons/README.md)
guidelines)
- [x] Tests cover the new/changed behavior
- [x] Performance impact has been considered and is acceptable
Closes #ISSUE
Release Notes:
- N/A or Added/Fixed/Improved ...
---------
Co-authored-by: zed-zippy[bot] <234243425+zed-zippy[bot]@users.noreply.github.com>
- Replace full sort and truncate with a budgeted top-N approach for
context identifiers using truncate_to_bottom_n_sorted_by.
- Introduce MAX_DIAGNOSTICS = 10 and prune diagnostics to this limit
with select_nth_unstable_by_key, then sort the remainder.
Self-Review Checklist:
- [x] I've reviewed my own diff for quality, security, and reliability
- [x] Unsafe blocks (if any) have justifying comments
- [x] The content adheres to Zed's UI standards
([UX/UI](https://github.com/zed-industries/zed/blob/main/CONTRIBUTING.md#uiux-checklist)
and
[icon](https://github.com/zed-industries/zed/blob/main/crates/icons/README.md)
guidelines)
- [x] Tests cover the new/changed behavior
- [x] Performance impact has been considered and is acceptable
Release Notes:
- Removed small allocations on the edit_predictions critical path by
using select_nth and sort_unstable
Co-authored-by: Nathan Barry <nathanbarry474@gmail.com>
Co-authored-by: Anthony Eid <hello@anthonyeid.me>
Self-Review Checklist:
- [x] I've reviewed my own diff for quality, security, and reliability
- [x] Unsafe blocks (if any) have justifying comments
- [x] The content is consistent with the [UI/UX
checklist](https://github.com/zed-industries/zed/blob/main/CONTRIBUTING.md#uiux-checklist)
- [x] Tests cover the new/changed behavior
- [x] Performance impact has been considered and is acceptable
Closes #ISSUE
Release Notes:
- N/A or Added/Fixed/Improved ...
Co-authored-by: Nbarry <nbarry@zed.dev>
This change adds metrics for evaluating editable context collectors and
the collectors themselves:
- `cursor excerpt`: small area around the cursor.
- `current-file`: the full file that contains the cursor.
- `edit-history`: excerpts around edit history edits.
- `edit-history-file`: full files referenced by the edit history.
- `git-log`: files related to the cursor file according to recent git
co-change history.
- `oracle-file`: full files referenced by the expected patches. This is
an oracle/evaluation-only source because it uses the target patch.
Individual collectors can be enabled by passing a comma-separated list
like this: `ep context --type=edit-history,current-file`. There's also a
preset, `--type=editable`, which will enable all collectors except
oracle.
This change also adds the
`.prompt_input.related_files[0].excerpts[0].context_source` field.
Editable context is only accessible via ep-cli for now.
Release Notes:
- N/A
---------
Co-authored-by: Eric Holk <eric@zed.dev>
- limit diagnostics sent to teacher
- improve parallelism in `ep` commands when using `--max-parallelism` by
only grouping by repo when instructed too (repo grouping is only useful
for context collection)
- update rejected and rated queries to also fetch settled editable
region
Self-Review Checklist:
- [x] I've reviewed my own diff for quality, security, and reliability
- [x] Unsafe blocks (if any) have justifying comments
- [x] The content is consistent with the [UI/UX
checklist](https://github.com/zed-industries/zed/blob/main/CONTRIBUTING.md#uiux-checklist)
- [x] Tests cover the new/changed behavior
- [x] Performance impact has been considered and is acceptable
Closes #ISSUE
Release Notes:
- N/A or Added/Fixed/Improved ...
Self-Review Checklist:
- [x] I've reviewed my own diff for quality, security, and reliability
- [x] Unsafe blocks (if any) have justifying comments
- [x] The content is consistent with the [UI/UX
checklist](https://github.com/zed-industries/zed/blob/main/CONTRIBUTING.md#uiux-checklist)
- [x] Tests cover the new/changed behavior
- [x] Performance impact has been considered and is acceptable
Closes #ISSUE
Release Notes:
- N/A or Added/Fixed/Improved ...
Self-Review Checklist:
- [x] I've reviewed my own diff for quality, security, and reliability
- [x] Unsafe blocks (if any) have justifying comments
- [x] The content is consistent with the [UI/UX
checklist](https://github.com/zed-industries/zed/blob/main/CONTRIBUTING.md#uiux-checklist)
- [ ] Tests cover the new/changed behavior
- [x] Performance impact has been considered and is acceptable
Closes https://github.com/zed-industries/zed/issues/52585
Release Notes:
- Fixed local zeta2 edit predictions using the wrong prompt format.
- Truncated diagnostics to fit ~2000 tokens
- Moved the diagnostics section before the cursor snippet
- Increased total tokens limit for this prompt to 8192 tokens
Release Notes:
- N/A
Co-authored-by: Ben Kunkle <ben@zed.dev>
Self-Review Checklist:
- [x] I've reviewed my own diff for quality, security, and reliability
- [x] Unsafe blocks (if any) have justifying comments
- [x] The content is consistent with the [UI/UX
checklist](https://github.com/zed-industries/zed/blob/main/CONTRIBUTING.md#uiux-checklist)
- [x] Tests cover the new/changed behavior
- [x] Performance impact has been considered and is acceptable
Closes #ISSUE
Release Notes:
- N/A
Self-Review Checklist:
- [x] I've reviewed my own diff for quality, security, and reliability
- [x] Unsafe blocks (if any) have justifying comments
- [x] The content is consistent with the [UI/UX
checklist](https://github.com/zed-industries/zed/blob/main/CONTRIBUTING.md#uiux-checklist)
- [x] Tests cover the new/changed behavior
- [x] Performance impact has been considered and is acceptable
Closes #ISSUE
Release Notes:
- N/A or Added/Fixed/Improved ...
Self-Review Checklist:
- [x] I've reviewed my own diff for quality, security, and reliability
- [x] Unsafe blocks (if any) have justifying comments
- [x] The content is consistent with the [UI/UX
checklist](https://github.com/zed-industries/zed/blob/main/CONTRIBUTING.md#uiux-checklist)
- [x] Tests cover the new/changed behavior
- [x] Performance impact has been considered and is acceptable
Closes #ISSUE
Release Notes:
- N/A or Added/Fixed/Improved ...
Self-Review Checklist:
- [x] I've reviewed my own diff for quality, security, and reliability
- [x] Unsafe blocks (if any) have justifying comments
- [x] The content is consistent with the [UI/UX
checklist](https://github.com/zed-industries/zed/blob/main/CONTRIBUTING.md#uiux-checklist)
- [x] Tests cover the new/changed behavior
- [x] Performance impact has been considered and is acceptable
Closes #ISSUE
Release Notes:
- N/A or Added/Fixed/Improved ...
Self-Review Checklist:
- [x] I've reviewed my own diff for quality, security, and reliability
- [x] Unsafe blocks (if any) have justifying comments
- [x] The content is consistent with the [UI/UX
checklist](https://github.com/zed-industries/zed/blob/main/CONTRIBUTING.md#uiux-checklist)
- [x] Tests cover the new/changed behavior
- [x] Performance impact has been considered and is acceptable
Closes #ISSUE
Release Notes:
- N/A or Added/Fixed/Improved ...
Self-Review Checklist:
- [x] I've reviewed my own diff for quality, security, and reliability
- [ ] Unsafe blocks (if any) have justifying comments
- [ ] The content is consistent with the [UI/UX
checklist](https://github.com/zed-industries/zed/blob/main/CONTRIBUTING.md#uiux-checklist)
- [x] Tests cover the new/changed behavior
- [x] Performance impact has been considered and is acceptable
Closes #ISSUE
Release Notes:
- N/A or Added/Fixed/Improved ...
This PR mostly moves some code around. It also adds a high-level
`format_expected_output` function that routes patch formatting to
specific prompt formats. This was `zeta_prompt` can format
`expected_output` for training.
Keeping everything prompt-related in a "pure" module (with no heavy
dependencies) makes it easier to write bindings.
Release Notes:
- N/A
Closes#52489
The special token check in `prompt_input_contains_special_tokens` used
`String::contains()` to look for `=======\n` in the buffer. This meant
any line containing `=======` (like `// =======` section separators)
would cause edit predictions to bail out entirely.
Fixed by only matching when the token appears at the start of a line,
since the git merge markers are always placed at line boundaries in the
prompt.
Added tests for both the helper function and a regression test for the
reported issue.
---------
Co-authored-by: Oleksiy Syvokon <oleksiy.syvokon@gmail.com>
This PR adds `v0318`, which is just like `v0316` but with bigger blocks.
It seems to perform best so far.
It also adds heuristics to avoid placing the marker token on closing
braces, which results in much nicer blocks.
Finally, it fixes a bug with inserting `<|marker_N|>` mid-line in
`v0316` and `v0317`.
Release Notes:
- N/A
V0316 differences from V0306:
- Use `<|marker_N|><|marker_N|>` instead of `NO_EDITS`
- All intermediate markers are now emitted (should be more friendly to
speculative decoding)
- No newline padding around marker tags
- No git conflict markers; we now use SeedCoder's EOS tag instead
V0317 is like V0316, but marker numbers are relative to the cursor block
(`<|marker-2|> ... -1 ... -0 ... +1 ... <|marker+2|>`)
Release Notes:
- N/A
Closes #ISSUE
Before you mark this PR as ready for review, make sure that you have:
- [ ] Added a solid test coverage and/or screenshots from doing manual
testing
- [ ] Done a self-review taking into account security and performance
aspects
- [ ] Aligned any UI changes with the [UI
checklist](https://github.com/zed-industries/zed/blob/main/CONTRIBUTING.md#uiux-checklist)
Release Notes:
- N/A *or* Added/Fixed/Improved ...
This format generates fewer token while maintaining the quality:
```
Model Generated tokens ↓ DeltaChrF ↑
0306-seed-multi-regions 46,239 80.62
0304-seed-no-edits 110,871 80.61
0303-seed 271,457 79.62
```
In addition to the student format, this change adds a new teacher
prompt. It seems to be worse than the original, but I haven't optimized
it at all. Keeping it for now as a base for potential improvements.
Release Notes:
- N/A
Closes #ISSUE
Before you mark this PR as ready for review, make sure that you have:
- [ ] Added a solid test coverage and/or screenshots from doing manual
testing
- [ ] Done a self-review taking into account security and performance
aspects
- [ ] Aligned any UI changes with the [UI
checklist](https://github.com/zed-industries/zed/blob/main/CONTRIBUTING.md#uiux-checklist)
Release Notes:
- N/A *or* Added/Fixed/Improved ...
Previously we didn't distinguish between an empty `.related_files[]` and
a case where context collection hadn't run yet. As a result, context
retrieval was always attempted for examples with empty `related_files`.
Release Notes:
- N/A
Closes #ISSUE
Before you mark this PR as ready for review, make sure that you have:
- [ ] Added a solid test coverage and/or screenshots from doing manual
testing
- [ ] Done a self-review taking into account security and performance
aspects
- [ ] Aligned any UI changes with the [UI
checklist](https://github.com/zed-industries/zed/blob/main/CONTRIBUTING.md#uiux-checklist)
Release Notes:
- N/A *or* Added/Fixed/Improved ...
Closes #ISSUE
Before you mark this PR as ready for review, make sure that you have:
- [ ] Added a solid test coverage and/or screenshots from doing manual
testing
- [ ] Done a self-review taking into account security and performance
aspects
- [ ] Aligned any UI changes with the [UI
checklist](https://github.com/zed-industries/zed/blob/main/CONTRIBUTING.md#uiux-checklist)
Release Notes:
- N/A *or* Added/Fixed/Improved ...
Closes #ISSUE
Before you mark this PR as ready for review, make sure that you have:
- [ ] Added a solid test coverage and/or screenshots from doing manual
testing
- [ ] Done a self-review taking into account security and performance
aspects
- [ ] Aligned any UI changes with the [UI
checklist](https://github.com/zed-industries/zed/blob/main/CONTRIBUTING.md#uiux-checklist)
Release Notes:
- N/A *or* Added/Fixed/Improved ...
Closes #ISSUE
Before you mark this PR as ready for review, make sure that you have:
- [ ] Added a solid test coverage and/or screenshots from doing manual
testing
- [ ] Done a self-review taking into account security and performance
aspects
- [ ] Aligned any UI changes with the [UI
checklist](https://github.com/zed-industries/zed/blob/main/CONTRIBUTING.md#uiux-checklist)
Release Notes:
- N/A *or* Added/Fixed/Improved ...
---------
Co-authored-by: Max <max@zed.dev>
In some evals, the teacher produced hallucinations, seemingly due to
context rot. This makes the zeta prompt crate's budgeted rendering
usable by the teacher, so that it can truncate the list of excerpts.
I've also cleaned up the implementation of zeta_prompt's
`format_related_files_within_budget`, and changed the behavior so that
it filters the the excerpts by priority but renders the files in their
original order.
Release Notes:
- N/A
We store the byte distance between the cursor and references to each
definition. When including excerpts in the prompt, we prioritize them in
the order of proximity. I've updated the Edit Prediction Context view to
display the excerpt's `order`, and sorting the files in order of their
excerpt with the lowest order.
Release Notes:
- N/A
Previously, we were not computing excerpt regions correctly for EP
examples captured from prod. This PR fixes that, and also simplifies the
data flow in the EP CLI. Examples either come from a concise spec (like
the markdown evals), or are collected from prod. Either way, we compute
from them a `ZetaPromptInput`, and the downstream steps like
prompt-formatting and scoring are derived from that.
Release Notes:
- N/A
---------
Co-authored-by: Ben Kunkle <ben@zed.dev>
