Commit Graph
6 Commits
Author SHA1 Message Date
Max Brunsfeld d4e89f9587 Add edit prediction evals that test related excerpt usage (#50256)
I've also fixed a race condition with the programmatic context retrieval
in the CLI, which was causing no excerpts to be fetched for the Rust
examples.

Release Notes:

- N/A
2026-02-26 15:36:29 -08:00
Max Brunsfeld 3da9adc4e7 Edit prediction: add some typescript evals, fix a teacher model failure mode (#50141)
In some evals, the teacher produced hallucinations, seemingly due to
context rot. This makes the zeta prompt crate's budgeted rendering
usable by the teacher, so that it can truncate the list of excerpts.

I've also cleaned up the implementation of zeta_prompt's
`format_related_files_within_budget`, and changed the behavior so that
it filters the the excerpts by priority but renders the files in their
original order.

Release Notes:

- N/A
2026-02-25 15:47:08 -08:00
Oleksiy Syvokon 67d9e7d011 ep: Fix teacher parser and other fixes (#49863)
Release Notes:

- N/A
2026-02-23 07:18:04 +00:00
Oleksiy Syvokon 3129d7e6b5 ep: Parse "user accepted prediction" markers in evals (#49598)
Also:
- Add two evals
- Remove duplicated Example 6 from the teacher prompt 

Release Notes:

- N/A
2026-02-19 13:01:07 +00:00
Oleksiy Syvokon d4af86001c ep: Update teacher prompt (#49489)
* More conservative predictions for prose
* Explain "user accepted prediction" in the teacher prompt
* Sonnet 4.6 support
* Don't strip comments in teacher prompt's edit history

Release Notes:

- N/A
2026-02-18 15:26:47 +00:00
Max Brunsfeld 6cbeb84880 Tune edit prediction teacher to leave fewer blank spots in predictions (#49315)
Release Notes:

- N/A
2026-02-16 16:38:43 -08:00