marius851000andBennet Bo Fenner
78de9335df
# Objective - Tool call can output image. Ollama did not support that, and dropped them when generating the Ollama request. The images are now properly added. - Fixes #64054 ## Solution - Convert the data from Zed’s internal structure to Ollama’s structure - Should avoid allocation if there is no images (the most common path) (untested). ## Testing - Tested with qwen3.6:35b via Ollama. It can read and describe image, with query such as "Describe zébrette_le_matin_22_novembre_1913.jpg" (the file not being directly embedded). The agent use the read_file tool. ## Self-Review Checklist: - [x] I've reviewed my own diff for quality, security, and reliability - [x] Unsafe blocks (if any) have justifying comments (there are none) - [x] The content adheres to Zed's UI standards ([UX/UI](https://github.com/zed-industries/zed/blob/main/CONTRIBUTING.md#uiux-checklist) and [icon](https://github.com/zed-industries/zed/blob/main/crates/icons/README.md) guidelines) (no UI changes) - [ ] Tests cover the new/changed behavior - [x] Performance impact has been considered and is acceptable ## Showcase <details> <summary>Transcription of a test query</summary> ## User Describe zébrette_le_matin_22_novembre_1913.jpg ## Assistant <thinking> The user is asking me to describe an image file: "zébrètte_le_matin_22_novembre_1913.jpg". This appears to be a newspaper/magazine cover or page from "Le Matin" dated November 22, 1913. I need to read this image file to see what's in it. Let me find it first. </thinking> Let me locate and examine that image file first. **Tool Call: No matches** Status: Completed ```json { "offset": 0, "current_matches_page": [], "all_matches_len": 0 } ``` ## Assistant <thinking> Let me try different variations of the filename. </thinking> **Tool Call: 1 match** Status: Completed file:///home/marius/sec/a/z%C3%A9brette_le_matin_22_novembre_1913.jpg ## Assistant <thinking> Found it. Let me read the image file. </thinking> **Tool Call: Read file `zébrette_le_matin_22_novembre_1913.jpg`** Status: Completed `Image` ## Assistant <thinking> Now I can describe the image. It's a black-and-white illustration from what appears to be a French newspaper ("Le Matin" dated November 22, 1913). The image depicts an elegantly dressed woman descending a staircase. She is wearing: - An elaborate headdress with what looks like ostrich feathers or plumes - A decorative crown or tiara - Long earrings - Multiple strands of pearls around her neck - A flowing, ornate dress with fur trim (a stole/cape) - A long necklace The style is Art Nouveau/Art Déco. She appears to be a society figure attending some gala or fancy dress event. The illustration has a halftone dot pattern typical of newspaper printing from that era. I'm wondering what "zébrètte" refers to—it might be the name of the person depicted, the caption associated with this illustration, or perhaps a specific fashion item. </thinking> ## Description: *Zébrètte* (Le Matin, 22 novembre 1913) This is a black-and-white newspaper halftone illustration from the French daily **Le Matin**, dated November 22, 1913. It depicts an elegantly dressed woman descending a staircase — likely covering a high-society or gala event. ### Key details: - **Headdress**: The most striking feature is an elaborate plumed hat with large ostrich-style feathers rising above her head, paired with a decorative crown/tiara. - **Attire**: She wears a flowing gown adorned with ornate patterns, draped under a luxurious fur-trimmed stole (or capelet). Her dress has intricate embellishments on the bodice and legs of the skirt. - **Accessories**: Multiple strands of pearls around her neck, long dangling earrings, and an additional long necklace chain add to the opulent look. - **Pose & setting**: She steps down a staircase with a swirling Art Nouveau–style railing behind her, suggesting she has just arrived at or is leaving a grand event (a ball, premiere, or society gathering). - **Artistic style**: Rendered in a sketchy, expressive line-drawing style typical of early 20th-century French illustration. The halftone dot pattern reveals it was printed in a newspaper. The caption title *"Zébrètte"* likely refers to either the nickname of the woman depicted or a costume/carnival reference (suggesting "striped" or zebra-print), though without the surrounding text from the original page, that detail remains ambiguous. --- The image in question, I had used as an illustration for one my blog post, as said by the AI from the Le Matin of the 22 november 1913, via Gallica (https://mariusdavid.fr/blog/z%C3%A9bressez%C3%A9brellez%C3%A9brette/) <img width="944" height="2488" alt="image" src="https://github.com/user-attachments/assets/3075a56e-b1ec-4906-ab8e-97d32400dc75" /> </details> --- Release Notes: - Fixed Ollama not being able to access images from tool call --------- Co-authored-by: Bennet Bo Fenner <bennetbo@gmx.de>
Zed
Welcome to Zed, a high-performance, multiplayer code editor from the creators of Atom and Tree-sitter.
Installation
On macOS, Linux, and Windows you can download Zed directly or install Zed via your local package manager (macOS/Linux/Windows).
Other platforms are not yet available:
- Web (tracking discussion)
Developing Zed
Contributing
See CONTRIBUTING.md for ways you can contribute to Zed.
Also... we're hiring! Check out our jobs page for open roles.
Licensing
Zed source code is licensed primarily under GPL-3.0-or-later, with Apache-2.0 components where marked.
License information for third party dependencies must be correctly provided for CI to pass.
We use cargo-about to automatically comply with open source licenses. If CI is failing, check the following:
- Is it showing a
no license specifiederror for a crate you've created? If so, addpublish = falseunder[package]in your crate's Cargo.toml. - Is the error
failed to satisfy license requirementsfor a dependency? If so, first determine what license the project has and whether this system is sufficient to comply with this license's requirements. If you're unsure, ask a lawyer. Once you've verified that this system is acceptable add the license's SPDX identifier to theacceptedarray inscript/licenses/zed-licenses.toml. - Is
cargo-aboutunable to find the license for a dependency? If so, add a clarification field at the end ofscript/licenses/zed-licenses.toml, as specified in the cargo-about book.
Sponsorship
Zed is developed by Zed Industries, Inc., a for-profit company.
If you’d like to financially support the project, you can do so via GitHub Sponsors. Sponsorships go directly to Zed Industries and are used as general company revenue. There are no perks or entitlements associated with sponsorship.