Fix zeta2 prompt format selection (#55338)

Self-Review Checklist:

- [x] I've reviewed my own diff for quality, security, and reliability
- [x] Unsafe blocks (if any) have justifying comments
- [x] The content is consistent with the [UI/UX
checklist](https://github.com/zed-industries/zed/blob/main/CONTRIBUTING.md#uiux-checklist)
- [ ] Tests cover the new/changed behavior
- [x] Performance impact has been considered and is acceptable

Closes https://github.com/zed-industries/zed/issues/52585

Release Notes:

- Fixed local zeta2 edit predictions using the wrong prompt format.
This commit is contained in:
Ben Kunkle authored and GitHub committed 2026-05-08 13:19:33 +00:00
1 parent f87f0c0e1d
commit 6bc4b4b7e4
10 files changed
+163 -38

No files matched your search

+63 -5
View File
@@ -286,11 +286,29 @@ After adding your API key, Codestral will appear in the provider dropdown in the
}
```
### Self-Hosted OpenAI-compatible servers
### Local and self-hosted models
You can use any self-hosted server that implements the OpenAI completion API format. This works with vLLM, llama.cpp server, LocalAI, and other compatible servers.
You can use local or self-hosted edit prediction models through Ollama or any server that implements the OpenAI completion API format. This works with Ollama, vLLM, llama.cpp server, LocalAI, and other compatible servers.
#### Configuration
#### Ollama
Set `ollama` as your provider and configure the local model:
```json [settings]
{
"edit_predictions": {
"provider": "ollama",
"ollama": {
"api_url": "http://localhost:11434",
"model": "qwen2.5-coder:7b-base",
"prompt_format": "infer",
"max_output_tokens": 512
}
}
}
```
#### OpenAI-compatible servers
Set `open_ai_compatible_api` as your provider and configure the API endpoint:
@@ -302,7 +320,7 @@ Set `open_ai_compatible_api` as your provider and configure the API endpoint:
"api_url": "http://localhost:8080/v1/completions",
"model": "deepseek-coder-6.7b-base",
"prompt_format": "deepseek_coder",
"max_output_tokens": 64
"max_output_tokens": 512
}
}
}
@@ -310,15 +328,55 @@ Set `open_ai_compatible_api` as your provider and configure the API endpoint:
The `prompt_format` setting controls how code context is formatted for the model. Use `"infer"` to detect the format from the model name, or specify one explicitly:
- `zeta` - Zeta 1 format
- `zeta2` - Zeta 2 format
- `zeta2_1` - Zeta 2.1 format
- `code_llama` - CodeLlama format: `<PRE> prefix <SUF> suffix <MID>`
- `star_coder` - StarCoder format: `<fim_prefix>prefix<fim_suffix>suffix<fim_middle>`
- `deepseek_coder` - DeepSeek format with special unicode markers
- `qwen` - Qwen/CodeGemma format: `<|fim_prefix|>prefix<|fim_suffix|>suffix<|fim_middle|>`
- `code_gemma` - CodeGemma format: `<|fim_prefix|>prefix<|fim_suffix|>suffix<|fim_middle|>`
- `codestral` - Codestral format: `[SUFFIX]suffix[PREFIX]prefix`
- `glm` - GLM-4 format with code markers
- `infer` - Auto-detect from model name (default)
Your server must implement the OpenAI `/v1/completions` endpoint. Edit predictions will send POST requests with this format:
With `"prompt_format": "infer"`, Zed automatically uses Zeta 2 format for models named `zeta2` and Zeta 2.1 format for models named `zeta2.1`.
For example, to use Zeta 2 with Ollama:
```json [settings]
{
"edit_predictions": {
"provider": "ollama",
"ollama": {
"api_url": "http://localhost:11434",
"model": "zeta2",
"prompt_format": "infer",
"max_output_tokens": 512
}
}
}
```
To use Zeta 2.1 with an OpenAI-compatible server:
```json [settings]
{
"edit_predictions": {
"provider": "open_ai_compatible_api",
"open_ai_compatible_api": {
"api_url": "http://localhost:8080/v1/completions",
"model": "zeta2.1",
"prompt_format": "infer",
"max_output_tokens": 512
}
}
}
```
You can also set `"prompt_format": "zeta2"` or `"prompt_format": "zeta2_1"` explicitly when the model name does not match.
Your OpenAI-compatible server must implement the OpenAI `/v1/completions` endpoint. Edit predictions will send POST requests with this format:
```json
{