Finn Eitreim
fdf144fb72
## Context This PR fixes one of the issues in #51905, where model outputs are full of errant `\` characters. heres the problem: As the response is streamed back to zed, we accumulate the message chunks and and need to convert those chunks to valid json, to do that we use `partial_json_fixer::fix_json`, when the last character of a chunk is `\`, the `fix_json` has to escape that backslash, because its inside of a string (if it isn't, its invalid json and the tool call will crash) and other wise you would end up escaping the end `"` and everything would be messed up. why is this a problem for zed: T_0 is the output at some step. T_1 is the output at the next step. the `fix_json` system is meant to be used by replacing T_0 with T_1, however in the editor, replacing the entirety of T_0 with T_1 would be slow/cause flickering/etc.. so we calculate the difference between T_0 and T_1 and just add it to the current buffer state. So when a chunk ends on `\`, we end up with something like `... end of line\\"}` at the end of T_0, in T_1, this becomes `... end of line\n ...`. then when we add the new chunk from T_1, it just picks up after the \n because its tracking the length to manage the deltas. ## How to Review utils.rs: fix_streamed_json => remove trailing backslashes from incoming json streams so that `partial_json_fixer::fix_json` doesn't try to escape them. other files: call fix_streamed_json before passing to `serde_json` I had claude write a bunch of tests while I was working on the fix, which I have kept in for now, but the end functionality of fix_streamed_json is really simple now, so maybe these arent really needed. ## Videos Behavior Before: https://github.com/user-attachments/assets/f23f5579-b2e1-4d71-9e24-f15ea831de52 Behavior After: https://github.com/user-attachments/assets/40acdc23-4522-4621-be28-895965f4f262 ## Self-Review Checklist <!-- Check before requesting review: --> - [x] I've reviewed my own diff for quality, security, and reliability - [x] Unsafe blocks (if any) have justifying comments - [x] The content is consistent with the [UI/UX checklist](https://github.com/zed-industries/zed/blob/main/CONTRIBUTING.md#uiux-checklist) - [x] Tests cover the new/changed behavior - [x] Performance impact has been considered and is acceptable Release Notes: - language_models: fixed partial json streaming
Zed
Welcome to Zed, a high-performance, multiplayer code editor from the creators of Atom and Tree-sitter.
Installation
On macOS, Linux, and Windows you can download Zed directly or install Zed via your local package manager (macOS/Linux/Windows).
Other platforms are not yet available:
- Web (tracking issue)
Developing Zed
Contributing
See CONTRIBUTING.md for ways you can contribute to Zed.
Also... we're hiring! Check out our jobs page for open roles.
Licensing
License information for third party dependencies must be correctly provided for CI to pass.
We use cargo-about to automatically comply with open source licenses. If CI is failing, check the following:
- Is it showing a
no license specifiederror for a crate you've created? If so, addpublish = falseunder[package]in your crate's Cargo.toml. - Is the error
failed to satisfy license requirementsfor a dependency? If so, first determine what license the project has and whether this system is sufficient to comply with this license's requirements. If you're unsure, ask a lawyer. Once you've verified that this system is acceptable add the license's SPDX identifier to theacceptedarray inscript/licenses/zed-licenses.toml. - Is
cargo-aboutunable to find the license for a dependency? If so, add a clarification field at the end ofscript/licenses/zed-licenses.toml, as specified in the cargo-about book.
Sponsorship
Zed is developed by Zed Industries, Inc., a for-profit company.
If you’d like to financially support the project, you can do so via GitHub Sponsors. Sponsorships go directly to Zed Industries and are used as general company revenue. There are no perks or entitlements associated with sponsorship.