yenru0/zed
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

open_router: Support effort-based reasoning (#61308)

Browse Source 
# Objective

- This PR adds support for effort-based reasoning for OpenRouter models
- This PR correctly configures whether OpenRouter models support
reasoning or not
- Fixes #58843 (already closed, was converted to #58848 )

## Solution

- The OpenRouter `/v1/models` endpoint includes data about supported
effort levels (see
[docs](https://openrouter.ai/docs/guides/best-practices/reasoning-tokens#discovering-per-model-reasoning-options))
and model capabilities
- This PR uses this data to populate the "effort" dropdown in the agent
panel (compare #56411 which added a similar feature for OpenAI) and to
provide better model metadata (`supports_thinking`,
`supports_disabling_thinking`).
- If a model supports `max_tokens`-based thinking, effort levels will
still be shown. OpenRouter automatically converts effort levels into an
appropriate `max_tokens` budget
([docs](https://openrouter.ai/docs/guides/best-practices/reasoning-tokens#reasoning-effort-level)).
This way users are not limited to a fixed `max_tokens` budget.
- Some models return `"supported_parameters": ["reasoning"]` with
`"reasoning": {"mandatory": false}`. From the docs I am not quite sure
what that's supposed to mean. The PR interprets these models with a
binary thinking on/off switch.

The PR also fixes the inverted logic where
`supports_parallel_tool_calls` would always return `false`, which would
then result in `parallel_tool_calls: None` for the actual API call
(which enables parallel tool calls by default). So the result of
`supports_parallel_tool_calls` was interpreted opposite to what I would
expect. The PR now returns `true` from `supports_parallel_tool_calls`
and adapts the corresponding call. There is no change in behavior
(`parallel_tool_calls` are always enabled in practice), it just fixes
the code logic.

## Testing

I tested the changes manually in an instance started via `cargo run`:
- I tested using GLM 5.2 and GPT 5.5. The reasoning effort is being
shown in the agent panel. Changing the effort level changes the amount
of thinking that the model does (e.g. on "Extra High" I consistently see
thinking blocks, on lower levels I sometimes do. I don't see thinking
blocks when thinking is disabled).
- Some models with `"reasoning": {"mandatory": false}` can be called
with effort levels. I am not sure if it actually makes a difference, but
the API doesn't reject it. Maybe there are configuration options that
are not currently exposed by this PR. I think the approach in this PR is
a safe default though.
- To test this PR:
  1. Build it via `cargo run`
  2. Set an OpenRouter API Key
3. Open the Agent panel and select different models. Notice that the
effort level selector and the thinking toggle is dynamically
shown/hidden/populated based on the model.

## Self-Review Checklist:

- [x] I've reviewed my own diff for quality, security, and reliability
- [x] Unsafe blocks (if any) have justifying comments
- [x] The content adheres to Zed's UI standards
([UX/UI](https://github.com/zed-industries/zed/blob/main/CONTRIBUTING.md#uiux-checklist)
and
[icon](https://github.com/zed-industries/zed/blob/main/crates/icons/README.md)
guidelines)
- [x] Tests cover the new/changed behavior
- [x] Performance impact has been considered and is acceptable

—

Release Notes:

- Added support for effort-based reasoning when using OpenRouter models

---------

Co-authored-by: Bennet Bo Fenner <bennet@zed.dev>
Co-authored-by: Bennet Bo Fenner <bennetbo@gmx.de>
This commit is contained in:
Kim WittenburgBennet Bo FennerBennet Bo Fenner
 authored and GitHub committed 2026-08-26 17:39:50 +00:00
1 parent 732d67cac8
commit 2f3ceb37ce
3 files changed
+234 -29
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

crates/language_models/src/provider/open_router.rs
+178 -24
View File
@@ -6,15 +6,17 @@ use gpui::{App, AppContext, AsyncApp, Context, Entity, SharedString, Task};
use http_client::{CustomHeaders, HttpClient};
use language_model::{
ApiKeyConfiguration, ApiKeyState, AuthenticateError, EnvVar, IconOrSvg, LanguageModel,
LanguageModelCompletionError, LanguageModelCompletionEvent, LanguageModelId, LanguageModelName,
LanguageModelProvider, LanguageModelProviderId, LanguageModelProviderName,
LanguageModelProviderState, LanguageModelRequest, LanguageModelToolChoice,
LanguageModelToolResultContent, LanguageModelToolSchemaFormat, LanguageModelToolUse,
MessageContent, ProviderSettingsView, RateLimiter, Role, StopReason, TokenUsage, env_var,
LanguageModelCompletionError, LanguageModelCompletionEvent, LanguageModelEffortLevel,
LanguageModelId, LanguageModelName, LanguageModelProvider, LanguageModelProviderId,
LanguageModelProviderName, LanguageModelProviderState, LanguageModelRequest,
LanguageModelToolChoice, LanguageModelToolResultContent, LanguageModelToolSchemaFormat,
LanguageModelToolUse, MessageContent, ProviderSettingsView, RateLimiter, Role, StopReason,
TokenUsage, env_var,
};
use open_ai::completion::ReasoningDetailsAccumulator;
use open_router::{
Model, ModelMode as OpenRouterModelMode, OPEN_ROUTER_API_URL, ResponseStreamEvent, list_models,
Model, ModelMode as OpenRouterModelMode, OPEN_ROUTER_API_URL, ReasoningEffort,
ResponseStreamEvent, list_models,
};
use settings::{OpenRouterAvailableModel as AvailableModel, Settings, SettingsStore};
use sha2::{Digest as _, Sha256};
@@ -222,13 +224,27 @@ impl LanguageModelProvider for OpenRouterLanguageModelProvider {
let mut settings_models = Vec::new();
for model in &Self::settings(cx).available_models {
let mode = model.mode.unwrap_or_default();
let (supported_efforts, default_effort) =
if matches!(mode, OpenRouterModelMode::Adaptive) {
(
model.reasoning_effort.into_iter().collect(),
model.reasoning_effort,
)
} else {
(Vec::new(), None)
};
settings_models.push(open_router::Model {
name: model.name.clone(),
display_name: model.display_name.clone(),
max_tokens: model.max_tokens,
supports_tools: model.supports_tools,
supports_images: model.supports_images,
mode: model.mode.unwrap_or_default(),
mode,
supported_efforts,
default_effort,
supports_max_tokens: false,
mandatory_reasoning: false,
provider: model.provider.clone(),
});
}
@@ -351,7 +367,37 @@ impl LanguageModel for OpenRouterLanguageModel {
}
fn supports_thinking(&self) -> bool {
matches!(self.model.mode, OpenRouterModelMode::Thinking { .. })
matches!(
self.model.mode,
OpenRouterModelMode::Thinking { .. } | OpenRouterModelMode::Adaptive
)
}
fn supports_disabling_thinking(&self) -> bool {
!self.model.mandatory_reasoning
}
fn supported_effort_levels(&self) -> Vec<LanguageModelEffortLevel> {
let efforts: &[ReasoningEffort] = if !self.model.supported_efforts.is_empty() {
&self.model.supported_efforts
} else if self.model.supports_max_tokens {
&ReasoningEffort::OPENAI_COMPATIBLE_SELECTABLE
} else {
return Vec::new();
};
let default_effort = self.model.default_effort.or_else(|| {
self.model
.supports_max_tokens
.then_some(ReasoningEffort::Medium)
});
efforts
.iter()
.map(|&effort| LanguageModelEffortLevel {
name: effort.label().into(),
value: effort.value().into(),
is_default: Some(effort) == default_effort,
})
.collect()
}
fn tool_input_format(&self) -> LanguageModelToolSchemaFormat {
@@ -575,23 +621,40 @@ pub fn into_open_router(
stop: request.stop,
temperature: request.temperature.unwrap_or(0.4),
max_tokens: max_output_tokens,
parallel_tool_calls: if model.supports_parallel_tool_calls() && !request.tools.is_empty() {
Some(false)
} else {
None
},
parallel_tool_calls: (!request.tools.is_empty() && !model.supports_parallel_tool_calls())
.then_some(false),
usage: open_router::RequestUsage { include: true },
reasoning: if request.thinking_allowed
&& let OpenRouterModelMode::Thinking { budget_tokens } = model.mode
{
Some(open_router::Reasoning {
effort: None,
max_tokens: budget_tokens,
exclude: Some(false),
enabled: Some(true),
})
} else {
None
reasoning: match model.mode {
OpenRouterModelMode::Adaptive if request.thinking_allowed => {
Some(open_router::Reasoning {
enabled: Some(true),
effort: request
.thinking_effort
.as_deref()
.and_then(|e| e.parse::<ReasoningEffort>().ok()),
max_tokens: None,
exclude: None,
})
}
OpenRouterModelMode::Thinking { budget_tokens } if request.thinking_allowed => {
Some(open_router::Reasoning {
enabled: Some(true),
effort: None,
max_tokens: budget_tokens,
exclude: None,
})
}
OpenRouterModelMode::Adaptive | OpenRouterModelMode::Thinking { .. }
if !model.mandatory_reasoning =>
{
Some(open_router::Reasoning {
enabled: Some(false),
effort: None,
max_tokens: None,
exclude: None,
})
}
_ => None,
},
tools: request
.tools
@@ -1098,6 +1161,9 @@ mod tests {
Some(false),
None,
None,
None,
false,
None,
);
let thread_id = "internal-thread-id";
let request = LanguageModelRequest {
@@ -1134,6 +1200,9 @@ mod tests {
Some(false),
None,
None,
None,
false,
None,
);
let request = LanguageModelRequest {
@@ -1285,6 +1354,9 @@ mod tests {
Some(false),
None,
None,
None,
false,
None,
);
let request = LanguageModelRequest {
@@ -1348,6 +1420,9 @@ mod tests {
Some(false),
None,
None,
None,
false,
None,
);
let request = LanguageModelRequest {
@@ -1400,4 +1475,83 @@ mod tests {
}
}
}
#[gpui::test]
async fn test_into_open_router_sends_requested_effort() {
let model = open_router::Model::new(
"z-ai/glm-5.2",
Some("GLM 5.2"),
Some(1_048_576),
Some(true),
Some(false),
Some(OpenRouterModelMode::Adaptive),
Some(vec![
open_router::ReasoningEffort::XHigh,
open_router::ReasoningEffort::High,
]),
Some(open_router::ReasoningEffort::High),
false,
None,
);
let request = LanguageModelRequest {
messages: vec![language_model::LanguageModelRequestMessage {
role: Role::User,
content: vec![MessageContent::Text("Hello".to_string())],
cache: false,
reasoning_details: None,
}],
thinking_allowed: true,
thinking_effort: Some("xhigh".to_string()),
..Default::default()
};
let result = into_open_router(request, &model, None).unwrap();
let reasoning = result.reasoning.expect("reasoning should be set");
assert_eq!(reasoning.effort, Some(open_router::ReasoningEffort::XHigh));
assert_eq!(
reasoning.max_tokens, None,
"max_tokens should not be sent when effort is used"
);
assert_eq!(reasoning.exclude, None);
assert_eq!(reasoning.enabled, Some(true));
}
#[gpui::test]
async fn test_into_open_router_disables_reasoning_when_thinking_not_allowed() {
let model = open_router::Model::new(
"z-ai/glm-5.2",
Some("GLM 5.2"),
Some(1_048_576),
Some(true),
Some(false),
Some(OpenRouterModelMode::Adaptive),
Some(vec![
open_router::ReasoningEffort::XHigh,
open_router::ReasoningEffort::High,
]),
Some(open_router::ReasoningEffort::High),
false,
None,
);
let request = LanguageModelRequest {
messages: vec![language_model::LanguageModelRequestMessage {
role: Role::User,
content: vec![MessageContent::Text("Hello".to_string())],
cache: false,
reasoning_details: None,
}],
thinking_allowed: false,
thinking_effort: Some("high".to_string()),
..Default::default()
};
let result = into_open_router(request, &model, None).unwrap();
let reasoning = result.reasoning.expect("reasoning should be set");
assert_eq!(reasoning.enabled, Some(false));
assert_eq!(
reasoning.effort, None,
"effort should not be sent when reasoning is disabled"
);
assert_eq!(reasoning.max_tokens, None);
}
}
crates/open_router/src/open_router.rs
+55 -5
View File
@@ -3,6 +3,7 @@ use futures::{AsyncReadExt, StreamExt, stream::BoxStream};
use http_client::{
AsyncBody, CustomHeaders, HttpClient, Method, Request as HttpRequest, RequestBuilderExt, http,
};
pub use language_model_core::ReasoningEffort;
use open_ai::ChatCompletionStreamEvent;
use serde::{Deserialize, Serialize};
use serde_json::Value;
@@ -81,6 +82,12 @@ pub struct Model {
pub supports_images: Option<bool>,
#[serde(default)]
pub mode: ModelMode,
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub supported_efforts: Vec<ReasoningEffort>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub default_effort: Option<ReasoningEffort>,
pub supports_max_tokens: bool,
pub mandatory_reasoning: bool,
pub provider: Option<Provider>,
}
@@ -94,6 +101,9 @@ impl Model {
Some(false),
Some(ModelMode::Default),
None,
None,
false,
None,
)
}
@@ -104,6 +114,9 @@ impl Model {
supports_tools: Option<bool>,
supports_images: Option<bool>,
mode: Option<ModelMode>,
supported_efforts: Option<Vec<ReasoningEffort>>,
default_effort: Option<ReasoningEffort>,
supports_max_tokens: bool,
provider: Option<Provider>,
) -> Self {
Self {
@@ -113,6 +126,10 @@ impl Model {
supports_tools,
supports_images,
mode: mode.unwrap_or(ModelMode::Default),
supported_efforts: supported_efforts.unwrap_or_default(),
default_effort,
supports_max_tokens,
mandatory_reasoning: false,
provider,
}
}
@@ -138,7 +155,7 @@ impl Model {
}
pub fn supports_parallel_tool_calls(&self) -> bool {
false
true
}
}
@@ -200,7 +217,7 @@ pub struct FunctionDefinition {
#[derive(Debug, Serialize, Deserialize)]
pub struct Reasoning {
#[serde(skip_serializing_if = "Option::is_none")]
pub effort: Option<String>,
pub effort: Option<ReasoningEffort>,
#[serde(skip_serializing_if = "Option::is_none")]
pub max_tokens: Option<u32>,
#[serde(skip_serializing_if = "Option::is_none")]
@@ -481,6 +498,8 @@ pub struct ModelEntry {
pub supported_parameters: Vec<String>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub architecture: Option<ModelArchitecture>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub reasoning: Option<ModelReasoning>,
}
#[derive(Default, Debug, Clone, PartialEq, Deserialize)]
@@ -489,6 +508,20 @@ pub struct ModelArchitecture {
pub input_modalities: Vec<String>,
}
#[derive(Default, Debug, Clone, PartialEq, Deserialize)]
pub struct ModelReasoning {
#[serde(default, skip_serializing_if = "Option::is_none")]
pub default_effort: Option<ReasoningEffort>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub default_enabled: Option<bool>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub mandatory: Option<bool>,
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub supported_efforts: Vec<ReasoningEffort>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub supports_max_tokens: Option<bool>,
}
pub async fn stream_completion(
client: &dyn HttpClient,
api_url: &str,
@@ -610,12 +643,29 @@ pub async fn list_models(
.supported_parameters
.contains(&"reasoning".to_string())
{
ModelMode::Thinking {
budget_tokens: Some(4_096),
}
ModelMode::Adaptive
} else {
ModelMode::Default
},
supported_efforts: entry
.reasoning
.as_ref()
.map(|r| r.supported_efforts.clone())
.unwrap_or_default()
.into_iter()
.rev()
.collect(),
default_effort: entry.reasoning.as_ref().and_then(|r| r.default_effort),
supports_max_tokens: entry
.reasoning
.as_ref()
.and_then(|r| r.supports_max_tokens)
.unwrap_or(false),
mandatory_reasoning: entry
.reasoning
.as_ref()
.and_then(|r| r.mandatory)
.unwrap_or(false),
provider: None,
})
.collect();
crates/settings_content/src/language_model.rs
+1
View File
@@ -590,6 +590,7 @@ pub struct OpenRouterAvailableModel {
pub supports_tools: Option<bool>,
pub supports_images: Option<bool>,
pub mode: Option<ModelMode>,
pub reasoning_effort: Option<ReasoningEffort>,
pub provider: Option<OpenRouterProvider>,
}