The `ep split-commit --split-point` argument now accepts split point
kinds:
* fim -- next edit is right under the cursor.
* same-file-near -- next edit is close to the cursor.
* same-file-far -- next edit is more than 30-lines away from cursor.
* cross-file -- next edit is in another file.
This lets us create datasets partitioned by kind more efficiently
compared to filtering them after the fact.
Release Notes:
- N/A
When generating a training or evaluation example with `ep split-commit`,
the cursor sampling logic becomes:
1. 80% chance of cursor being at the end of the source patch
2. 20% chance of cursor being at the beginning of the target patch
3. 20% chance of adding a jitter offset (same line, ±5 columns for now)
Release Notes:
- N/A
Self-Review Checklist:
- [x] I've reviewed my own diff for quality, security, and reliability
- [x] Unsafe blocks (if any) have justifying comments
- [x] The content is consistent with the [UI/UX
checklist](https://github.com/zed-industries/zed/blob/main/CONTRIBUTING.md#uiux-checklist)
- [x] Tests cover the new/changed behavior
- [x] Performance impact has been considered and is acceptable
Closes #ISSUE
Release Notes:
- N/A or Added/Fixed/Improved ...
Previously, we were not computing excerpt regions correctly for EP
examples captured from prod. This PR fixes that, and also simplifies the
data flow in the EP CLI. Examples either come from a concise spec (like
the markdown evals), or are collected from prod. Either way, we compute
from them a `ZetaPromptInput`, and the downstream steps like
prompt-formatting and scoring are derived from that.
Release Notes:
- N/A
---------
Co-authored-by: Ben Kunkle <ben@zed.dev>
Other changes:
- Changed tokenization to more code-aware tokenization from split-commit
- Fixed word-diff implementation which was inefficient and sometimes
incorrect
Release Notes:
- N/A
The `capture example` action now populates the markdown file with a noop
"Rejected Patch", so that you can easily specify the good and bad
output.
Release Notes:
- N/A
1. Handle diffs with no trailing new lines
2. ep: Don't assume workdir name in edit history paths
3. Fix `imitate_human_edits()` for pure insertions
Release Notes:
- N/A
---------
Co-authored-by: Agus Zubiaga <agus@zed.dev>
1. `apply_diff` will create a file if the diff says so (by starting with
`--- /dev/null`)
2. Update examples format to match recent changes
3. `ep split-commits` can work with a stream of inputs and generate `n`
samples per input
4. Unicode handling fixes
Release Notes:
- N/A
Generates a training or evaluation example from a
chronologically-ordered commit. This is a port from the Python codebase
(except for the reorder_patch.rs, which was originally written in Rust
in).
Release Notes:
- N/A
Oleksiy Syvokon