Commit Graph
22 Commits
Author SHA1 Message Date
Ben KunkleandNbarry 6b5193dd54 zeta_prompt: Add V4 edit prediction API types (#59869)
# Objective

Part of EP-10
Closes EP-118

- Describe the objective or issue this PR addresses.
- If you're fixing a specific issue, use "Fixes #X" for each issue as
[described in the GitHub
docs](https://docs.github.com/en/issues/tracking-your-work-with-issues/using-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword).

## Solution

- Describe the solution used to achieve the objective above.

## Testing

- Did you test these changes? If so, how?
- Are there any parts that need more testing?
- How can other people (reviewers) test your changes? Is there anything
specific they need to know?
- If relevant, what platforms did you test these changes on, and are
there any important ones you can't test?

## Self-Review Checklist:

- [x] I've reviewed my own diff for quality, security, and reliability
- [x] Unsafe blocks (if any) have justifying comments
- [x] The content adheres to Zed's UI standards
([UX/UI](https://github.com/zed-industries/zed/blob/main/CONTRIBUTING.md#uiux-checklist)
and
[icon](https://github.com/zed-industries/zed/blob/main/crates/icons/README.md)
guidelines)
- [x] Tests cover the new/changed behavior
- [x] Performance impact has been considered and is acceptable

## Showcase

> This section is optional. If this PR does not include a visual change
or does not add a new user-facing feature, you can delete this section.

- Help others understand the result of this PR by showcasing your
awesome work!
- If this PR includes a visual change, consider adding a screenshot,
GIF, or video
- A before/after comparison is very useful for changes to existing
features!

While a showcase should aim to be brief and digestible, you can use a
toggleable section to save space on longer showcases:

<details>
  <summary>Click to view showcase</summary>

My super cool demos here

</details>

---

Release Notes:

- N/A or Added/Fixed/Improved ...

Co-authored-by: Nbarry <nbarry@zed.dev>
2026-06-25 14:07:41 +00:00
Ben Kunkle e36459aead ep: Jump example capture (#58236)
Self-Review Checklist:

- [x] I've reviewed my own diff for quality, security, and reliability
- [x] Unsafe blocks (if any) have justifying comments
- [x] The content is consistent with the [UI/UX
checklist](https://github.com/zed-industries/zed/blob/main/CONTRIBUTING.md#uiux-checklist)
- [x] Tests cover the new/changed behavior
- [x] Performance impact has been considered and is acceptable

Closes #ISSUE

Release Notes:

- N/A or Added/Fixed/Improved ...
2026-06-01 20:18:48 +00:00
Ben Kunkle 9f56ed26a5 ep: Increase example capture rate (#57208)
Self-Review Checklist:

- [x] I've reviewed my own diff for quality, security, and reliability
- [x] Unsafe blocks (if any) have justifying comments
- [x] The content is consistent with the [UI/UX
checklist](https://github.com/zed-industries/zed/blob/main/CONTRIBUTING.md#uiux-checklist)
- [x] Tests cover the new/changed behavior
- [x] Performance impact has been considered and is acceptable

Closes #ISSUE

Release Notes:

- N/A or Added/Fixed/Improved ...
2026-05-21 15:23:20 +00:00
Ben Kunkle f89fb5b252 Consolidate prompt formatting logic into zeta_prompt (#53079)
Self-Review Checklist:

- [x] I've reviewed my own diff for quality, security, and reliability
- [ ] Unsafe blocks (if any) have justifying comments
- [ ] The content is consistent with the [UI/UX
checklist](https://github.com/zed-industries/zed/blob/main/CONTRIBUTING.md#uiux-checklist)
- [x] Tests cover the new/changed behavior
- [x] Performance impact has been considered and is acceptable

Closes #ISSUE

Release Notes:

- N/A or Added/Fixed/Improved ...
2026-04-06 16:29:37 -04:00
Oleksiy Syvokon 67b2177aaf ep: Add two new prompt formats (#51836)
V0316 differences from V0306:
- Use `<|marker_N|><|marker_N|>` instead of `NO_EDITS`
- All intermediate markers are now emitted (should be more friendly to
speculative decoding)
- No newline padding around marker tags
- No git conflict markers; we now use SeedCoder's EOS tag instead

V0317 is like V0316, but marker numbers are relative to the cursor block
(`<|marker-2|> ... -1 ... -0 ... +1 ... <|marker+2|>`)

Release Notes:

- N/A
2026-03-18 17:02:06 +02:00
Max BrunsfeldandBen Kunkle a98e4651a2 Fix handling of excerpt regions in EP CLI (#49936)
Previously, we were not computing excerpt regions correctly for EP
examples captured from prod. This PR fixes that, and also simplifies the
data flow in the EP CLI. Examples either come from a concise spec (like
the markdown evals), or are collected from prod. Either way, we compute
from them a `ZetaPromptInput`, and the downstream steps like
prompt-formatting and scoring are derived from that.

Release Notes:

- N/A

---------

Co-authored-by: Ben Kunkle <ben@zed.dev>
2026-02-23 17:08:09 -08:00
Oleksiy Syvokon 8ef2b9b06e ep: Parse "user accepted prediction" markers in evals (#49598)
Also:
- Add two evals
- Remove duplicated Example 6 from the teacher prompt 

Release Notes:

- N/A
2026-02-19 13:01:07 +00:00
Oleksiy Syvokon a3d5f56be8 ep: Encode cursor position in the predicted patch (#49450)
Release Notes:

- N/A
2026-02-18 09:55:26 +00:00
Max Brunsfeld 4315ed2ca1 Filter collection of snowflake requests to those with latest schemas (#49424)
This allows us to just pull requests that have the latest EP request
schema with the `predicted` boolean on events in the edit history.

Release Notes:

- N/A
2026-02-17 16:37:28 -08:00
Max Brunsfeld 274a5f7ee4 Fix EP CLI issues found when generating new teacher predictions (#49327)
Release Notes:

- N/A
2026-02-17 09:33:34 +02:00
f3b95842c5 Unify zeta endpoints (#48900)
- [ ] Tests or screenshots needed?
- [ ] Code Reviewed
- [ ] Manual QA

Release Notes:

- N/A

---------

Co-authored-by: Zed Zippy <234243425+zed-zippy[bot]@users.noreply.github.com>
Co-authored-by: Max <max@zed.dev>
Co-authored-by: Max Brunsfeld <maxbrunsfeld@gmail.com>
2026-02-12 15:24:24 -05:00
Oleksiy Syvokon d09e6cd7b9 ep: Store cursor position in global coordinates (#48469)
This also fixes the isolated whitespace change metric, which is not
takes cursor position into account (it's okay to insert a newline if it
has a cursor on it)

Release Notes:

- N/A
2026-02-05 12:31:12 +00:00
Max Brunsfeld 84b8478185 Allow zeta2 to predict next cursor position along with edits (#47916)
* [x] capture and store teacher model's predicted cursor position
* [x] provide cursor position to student during distillation
* [x] eval cursor positions
* [x] parse and apply cursor position predictions at runtime

Release Notes:

- N/A
2026-01-31 16:16:32 +00:00
Ben Kunkle 18516486b2 ep_cli: Only check cursor excerpt for reversals (#48044)
Closes #ISSUE

Release Notes:

- N/A *or* Added/Fixed/Improved ...
2026-01-30 16:01:16 -05:00
Ben Kunkle a5d2dc5122 ep_cli: rated-after: query (#47906)
Closes #ISSUE

Release Notes:

- N/A *or* Added/Fixed/Improved ...
2026-01-28 16:36:44 -08:00
Max Brunsfeld 44df5c6a85 Allow EP cli to fetch rejections from snowflake (#47628)
Release Notes:

- N/A
2026-01-26 10:49:44 +00:00
Ben KunkleandZed Zippy ff556c5c22 zeta2: Include context in captured examples (#47516)
Closes #ISSUE

Release Notes:

- N/A *or* Added/Fixed/Improved ...

---------

Co-authored-by: Zed Zippy <234243425+zed-zippy[bot]@users.noreply.github.com>
2026-01-23 17:18:50 -05:00
Max Brunsfeld c0a0e8b236 Add a 'rejected patch' field to example specs, for DPO examples (#47043)
The `capture example` action now populates the markdown file with a noop
"Rejected Patch", so that you can easily specify the good and bad
output.

Release Notes:

- N/A
2026-01-18 20:25:23 -08:00
Agus Zubiaga 8d5c13c071 ep cli: Resume from output file (#46293) 2026-01-07 21:50:46 -03:00
Max Brunsfeld 5ba7ece91e Improve support for collecting edit prediction training and eval examples (#45914)
* Fix some bugs in capture of EP examples from running app
* Tweak markdown format for EP examples
    * Store repo and revision in TOML front matter
    * Represent cursor position using a comment line
* Allow multiple expected patches in evals
* Remove line-based scoring criteria for evals
* Add a `synthesize` subcommand to the EP cli that generates examples
from git commits

Release Notes:

- N/A
2026-01-03 16:08:35 -08:00
Max Brunsfeld 329970bff1 Improve edit prediction example capture (#45536)
This PR improves the `edit prediction: Capture Example` in several ways:
* fixed bugs in how the uncommitted diff was calculated
* added a `edit_predictions.examples_dir` setting that can be set in
order to have the action automatically save examples into the given
folder
* moved the action into the `edit_predictions` crate, in preparation for
collecting this data passively from end users, when they have opted in
to data sharing, similar to what we did for Zeta 1

Release Notes:

- N/A
2025-12-22 20:40:02 +00:00
Max Brunsfeld 5b5c1e371f Add an action for capturing your last edit as an edit prediction example (#44841)
This PR adds a staff-only button to the edit prediction menu for
capturing your current editing session as edit prediction example file.

When you click that button, it opens a markdown tab with the example. By
default, the most recent change that you've made is used as the expected
patch, and all of the previous events are used as the editing history.

<img width="303" height="123" alt="Screenshot 2025-12-14 at 6 58 33 PM"
src="https://github.com/user-attachments/assets/600c7bf2-7cf4-4d27-8cd4-8bb70d0b20b0"
/>

Release Notes:

- N/A
2025-12-14 20:50:48 -08:00