Closes #ISSUE
We're hitting request size limits on the new larger dataset, so we need
to do pre-batching. A 16k chunk size was chosen arbitrarily but seems to
be a good tradeoff between number of chunks / within request size limit
Release Notes:
- N/A *or* Added/Fixed/Improved ...
This command takes raw LLM outputs (`predictions.actual_output`) that
could be generated elsewhere and parses them into a canonical unified
diff (`predictions.actual_patch`).
This is useful for simplifying the evaluation pipeline and for rerunning
the parser without having to generate LLM outputs.
Release Notes:
- N/A
The `capture example` action now populates the markdown file with a noop
"Rejected Patch", so that you can easily specify the good and bad
output.
Release Notes:
- N/A
We were including changes due to Buffer.reload in the edit history.
Release Notes:
- N/A
---------
Co-authored-by: Oleksiy Syvokon <oleksiy.syvokon@gmail.com>
Co-authored-by: Agus Zubiaga <agus@zed.dev>
Co-authored-by: Ben Kunkle <ben@zed.dev>
Co-authored-by: Zed Zippy <234243425+zed-zippy[bot]@users.noreply.github.com>
We can specify prompt version in the provider name itself, like this
`--provider zeta2:0113`.
This kind of tag will also be stored in the `provider` field of
jsonlines files.
This drops the `--version` parameter.
Release Notes:
- N/A
This one does `fim_prefix`, `fim_middle`, and `fim_suffix` in that
order, in the prompt, instead of putting the current middle last.
Release Notes:
- N/A
---------
Co-authored-by: Agus Zubiaga <agus@zed.dev>
Co-authored-by: Ben Kunkle <ben@zed.dev>
This fixes an issue that we noticed in particular with Mercury edit
predictions.
* [x] fix storage to not go stale
* [x] exclude excerpts that intersect the cursor excerpt
* [x] see if string representation of excerpts can be cached, to avoid
rebuilding it on every prediction
Release Notes:
- N/A
---------
Co-authored-by: Ben Kunkle <ben@zed.dev>
Editable region was different for Zeta2 and Teacher, leading to "Edits
outside of editable region" errors.
Release Notes:
- N/A
Co-authored-by: Agus Zubiaga <agus@zed.dev>
This makes running `predict` with the teacher model much faster, when
there are many examples.
Release Notes:
- N/A
---------
Co-authored-by: Ben Kunkle <ben@zed.dev>
Some code got added to `workspace` that prevents us from running tests
for the `edit_prediction(cli)` crates specifically without the
`test-support` feature flag.
Release Notes:
- N/A
Co-authored-by: Ben Kunkle <ben@zed.dev>
Adds a new `ep split` command that splits JSONL datasets into multiple
output files with stratification by `repository_url` when present.
Example usage:
ep split input.jsonl train.jsonl=80% valid.jsonl=rest
Release Notes:
- N/A
- Replace `is_new_file: bool` with `FileStatus` enum
(Created/Modified/Deleted) in udiff.rs to properly track file status
through diff operations
- Handle deleted files in `apply_diff` by calling
`project.delete_file()`
- Fix diff serialization in reorder_patch.rs to output `+++ /dev/null`
for file deletions and parse both `--- /dev/null` and `+++ /dev/null`
correctly
- Add bounds check for edit ranges exceeding buffer length
Also includes edit_prediction_cli improvements:
- Track `context_range` and `editable_range` in ExampleBuffer for more
precise prompt formatting
- Export MAX_CONTEXT_TOKENS and MAX_REWRITE_TOKENS from zeta2
- Wait for buffer parsing before computing ranges
- Respect NO_COLOR env var and enable info-level logging
Release Notes:
- N/A
Co-authored-by: Agus Zubiaga <agus@zed.dev>
`extract_last_codeblock` was using `find()` to locate closing fences,
which would match backticks anywhere in the text. This caused incorrect
parsing when the content contained inline backticks or nested code
blocks, resulting in wrong diffs.
Fix by requiring the closing fence to be preceded by a newline.
Release Notes:
- N/A
When a revision like `abcd~1` wasn't found locally, the function would
fetch and then return `FETCH_HEAD`, which points to the tip of the
fetched branch rather than the requested revision expression.
Now it re-resolves the original revision after fetching, correctly
handling tilde expressions and other git revision syntax.
Release Notes:
- N/A
The 'X of Y examples failed' message was counting completed steps/tasks
instead of actual examples. For example, with 2 examples each going
through Load and Context steps, it would report '1 of 3' instead of '1
of 2'.
Release Notes:
- N/A
Sometimes git locks are left over from crashed runs. We now assume
there's only one process of the CLI running and clean them up. If we
want to run multiple processes at the same time, we should consider our
own file-based lock, but it seems fine for an internal tool.
Release Notes:
- N/A
* Added `--repo` and `--name` flags for running only examples with a
specific name, or repo (substring matching)
* Fixed a race condition that caused hangs when running multiple
examples at the same repo and sha
* Fixed a bug where scoring was completely wrong because I had passed
the arguments to `apply_diff_to_string` in the wrong order
* The current evals now run quickly and without errors.
Release Notes:
- N/A
Oleksiy Syvokon