Closes #ISSUE
Measure the model reversing user edits programatically, for a more
robust and consistent measurement than LLM-as-judge
Release Notes:
- N/A *or* Added/Fixed/Improved ...
Many editors such as vim and emacs support "modelines", a comment at the
beginning of the file that allows the file type to be explicitly
specified along with per-file specific settings
- The amount of configurations, style and settings mapping cannot be
handled in one go, so this opens up a lot of potential improvements.
- I left out the possiblity to have "zed" specific modelines for now,
but this could be potentially interesting.
- Mapping the mode or filetype to zed language names isn't obvious
either. We may want to make it configurable.
This is my first contribution to zed, be kind. I struggled a bit to find
the right place to add those settings. I use a similar approach as done
with editorconfig (merge_with_editorconfig). There might be better ways.
Closes#4762
Release Notes:
- Add basic emacs/vim modeline support.
Supersedes #41899, changes:
- limit reading to the first and last 1kb
- add documentation
- more variables handled
- add Arc around ModelineSettings to avoid extra cloning
- changed the way mode -> language mapping is done, thanks to
`modeline_aliases` language config
- drop vim ex: support
- made "Local Variables:" handling a separate commit, so we can drop it
easily
- various code style improvements
---------
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Conrad Irwin <conrad.irwin@gmail.com>
Closes #ISSUE
We're hitting request size limits on the new larger dataset, so we need
to do pre-batching. A 16k chunk size was chosen arbitrarily but seems to
be a good tradeoff between number of chunks / within request size limit
Release Notes:
- N/A *or* Added/Fixed/Improved ...
This command takes raw LLM outputs (`predictions.actual_output`) that
could be generated elsewhere and parses them into a canonical unified
diff (`predictions.actual_patch`).
This is useful for simplifying the evaluation pipeline and for rerunning
the parser without having to generate LLM outputs.
Release Notes:
- N/A
The `capture example` action now populates the markdown file with a noop
"Rejected Patch", so that you can easily specify the good and bad
output.
Release Notes:
- N/A
We were including changes due to Buffer.reload in the edit history.
Release Notes:
- N/A
---------
Co-authored-by: Oleksiy Syvokon <oleksiy.syvokon@gmail.com>
Co-authored-by: Agus Zubiaga <agus@zed.dev>
Co-authored-by: Ben Kunkle <ben@zed.dev>
Co-authored-by: Zed Zippy <234243425+zed-zippy[bot]@users.noreply.github.com>
We can specify prompt version in the provider name itself, like this
`--provider zeta2:0113`.
This kind of tag will also be stored in the `provider` field of
jsonlines files.
This drops the `--version` parameter.
Release Notes:
- N/A
This one does `fim_prefix`, `fim_middle`, and `fim_suffix` in that
order, in the prompt, instead of putting the current middle last.
Release Notes:
- N/A
---------
Co-authored-by: Agus Zubiaga <agus@zed.dev>
Co-authored-by: Ben Kunkle <ben@zed.dev>
This fixes an issue that we noticed in particular with Mercury edit
predictions.
* [x] fix storage to not go stale
* [x] exclude excerpts that intersect the cursor excerpt
* [x] see if string representation of excerpts can be cached, to avoid
rebuilding it on every prediction
Release Notes:
- N/A
---------
Co-authored-by: Ben Kunkle <ben@zed.dev>
Editable region was different for Zeta2 and Teacher, leading to "Edits
outside of editable region" errors.
Release Notes:
- N/A
Co-authored-by: Agus Zubiaga <agus@zed.dev>
This makes running `predict` with the teacher model much faster, when
there are many examples.
Release Notes:
- N/A
---------
Co-authored-by: Ben Kunkle <ben@zed.dev>
Some code got added to `workspace` that prevents us from running tests
for the `edit_prediction(cli)` crates specifically without the
`test-support` feature flag.
Release Notes:
- N/A
Co-authored-by: Ben Kunkle <ben@zed.dev>
Adds a new `ep split` command that splits JSONL datasets into multiple
output files with stratification by `repository_url` when present.
Example usage:
ep split input.jsonl train.jsonl=80% valid.jsonl=rest
Release Notes:
- N/A
Ben Kunkle