From 951357a9161761e6c07da2d134aec2d0cc874f2d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E1=B4=80=E1=B4=8D=E1=B4=9B=E1=B4=8F=E1=B4=80=E1=B4=87?= =?UTF-8?q?=CA=80?= Date: Fri, 28 Aug 2026 13:19:29 +0000 Subject: [PATCH] editor: Add language detection for untitled buffers (#61412) # Objective Closes #4868. Reintroduce automatic language detection for user-created untitled buffers without allowing it to override language choices made by users or owning surfaces. More details: + https://github.com/zed-industries/zed/pull/61201 + https://github.com/zed-industries/zed/pull/61351 ## Solution VS Code distinguishes between inferred and explicitly selected languages, and only continues automatic detection in the former state. This change introduces a buffer-local opt-in flag with the same purpose. It defaults to disabled and is enabled only for user-created untitled buffers and restored untitled buffers that did not persist a language. Choosing a language through the language picker disables detection, while automatically detected languages remain eligible for later redetection. Buffers whose language is assigned by their owning surface, such as Channel Notes, never opt in. Rather than adding persisted state or changing the database schema, restoration reuses the existing optional serialized language. An opted-in buffer that remains Plain Text is serialized without a language and resumes detection after restoration. Once automatic detection has selected a language, that language is serialized normally and restoration leaves detection disabled, effectively treating it as an explicit selection. This also matches VS Code's effective default after restoration. Candidate selection adopts VS Code's confidence-group filtering. Results below the minimum confidence are discarded, and candidate groups are accepted only after a clear confidence gap. The detector then selects the first available Zed language without eagerly loading every candidate. I then added a separate switching rule: Zed keeps the current language unless another candidate has a clear confidence advantage, reducing language changes as model scores fluctuate during editing. Model inference runs off-thread on a bounded sample, and stale results are discarded. I also evaluated penalizing YAML and using stricter global switching thresholds, but neither reliably resolves the ambiguity between Markdown and YAML. In the reproduction from #61351, Betlang assigns YAML more than 80% confidence during editing and peaks around 90%. This is reasonable from the content alone: `#` denotes a YAML comment, while `-` denotes a sequence item. Adding ordinary prose or a fenced code block shifts the model strongly toward Markdown. A fixed penalty would overfit this example and risk suppressing legitimate YAML detection. Since known-language buffers do not opt in and manual language selection takes precedence, I consider this level of ambiguity acceptable and prefer not to introduce language-specific score tuning. Model inference itself is fast. As noted in #61201: > In release builds on Linux with an Intel Core i5-13600KF, the `betlang::detect` call alone typically completes within 3 ms when processing the maximum sampled input. The main concern is the work surrounding inference. Without debouncing, rapid typing creates a snapshot and launches a background detection task on every keystroke, while the detected language may change before the user has finished typing. This change therefore adds a 200 ms debounce delay, allowing detection to run and update the language only after the user pauses. ## Testing Automated coverage verifies that short inputs remain Plain Text, sufficient Rust content is detected as Rust, replacing it with Go content triggers redetection, and manually selecting Plain Text prevents subsequent automatic changes. Restoration coverage verifies that a stored language keeps detection disabled, while an empty language-less untitled buffer resumes detection. Beyond the automated tests, I manually tested automatic language detection while editing untitled buffers. ## Self-Review Checklist: - [x] I've reviewed my own diff for quality, security, and reliability - [x] Unsafe blocks (if any) have justifying comments - [x] The content adheres to Zed's UI standards ([UX/UI](https://github.com/zed-industries/zed/blob/main/CONTRIBUTING.md#uiux-checklist) and [icon](https://github.com/zed-industries/zed/blob/main/crates/icons/README.md) guidelines) - [x] Tests cover the new/changed behavior - [x] Performance impact has been considered and is acceptable ## Showcase https://github.com/user-attachments/assets/103fc731-e12e-48d6-ae89-03978f1c9d74 --- Release Notes: - Added automatic language detection for untitled buffers. --------- Co-authored-by: Max Stevens Co-authored-by: Christopher Biscardi --- Cargo.lock | 25 +++ Cargo.toml | 3 + assets/settings/default.json | 3 + crates/editor/Cargo.toml | 1 + crates/editor/src/editor.rs | 65 +++++++- crates/editor/src/editor_settings.rs | 2 + crates/editor/src/editor_tests.rs | 144 +++++++++++++++++- crates/editor/src/items.rs | 22 ++- crates/language/src/buffer.rs | 11 ++ crates/language_detection/Cargo.toml | 17 +++ crates/language_detection/LICENSE-GPL | 1 + .../src/language_detection.rs | 137 +++++++++++++++++ crates/project/src/lsp_store.rs | 13 +- crates/project/src/project.rs | 3 + crates/settings/src/vscode_import.rs | 1 + crates/settings_content/src/editor.rs | 6 + crates/settings_ui/src/page_data.rs | 16 +- crates/zed/src/zed.rs | 3 + docs/src/migrate/vs-code.md | 1 + docs/src/reference/all-settings.md | 10 ++ 20 files changed, 475 insertions(+), 9 deletions(-) create mode 100644 crates/language_detection/Cargo.toml create mode 120000 crates/language_detection/LICENSE-GPL create mode 100644 crates/language_detection/src/language_detection.rs diff --git a/Cargo.lock b/Cargo.lock index bd25d2b3fc..cda9a18032 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -2154,6 +2154,15 @@ dependencies = [ "zed_actions", ] +[[package]] +name = "betlang" +version = "0.1.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5f89b0929539eaee70109704ae4e345df438be6ab02e4dc8ac060e05098ad1b7" +dependencies = [ + "fearless_simd", +] + [[package]] name = "bigdecimal" version = "0.4.8" @@ -5602,6 +5611,7 @@ dependencies = [ "indoc", "itertools 0.14.0", "language", + "language_detection", "languages", "linkify", "log", @@ -6352,6 +6362,12 @@ dependencies = [ "simd-adler32", ] +[[package]] +name = "fearless_simd" +version = "0.4.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b97b65636e5b9ef369943878ac74335ba1c55c1cb6adbf1e2c293c624248d693" + [[package]] name = "feature_flags" version = "0.1.0" @@ -9551,6 +9567,15 @@ dependencies = [ "tree-sitter", ] +[[package]] +name = "language_detection" +version = "0.1.0" +dependencies = [ + "betlang", + "gpui", + "language", +] + [[package]] name = "language_extension" version = "0.1.0" diff --git a/Cargo.toml b/Cargo.toml index a496df4a04..a70740ab0f 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -116,6 +116,7 @@ members = [ "crates/keymap_editor", "crates/language", "crates/language_core", + "crates/language_detection", "crates/language_extension", "crates/language_model", "crates/language_model_core", @@ -383,6 +384,7 @@ json_schema_store = { path = "crates/json_schema_store" } keymap_editor = { path = "crates/keymap_editor" } language = { path = "crates/language" } language_core = { path = "crates/language_core" } +language_detection = { path = "crates/language_detection" } language_extension = { path = "crates/language_extension" } language_model = { path = "crates/language_model" } language_model_core = { path = "crates/language_model_core" } @@ -560,6 +562,7 @@ aws-smithy-runtime-api = { version = "1.9.2", features = ["http-1x", "client"] } aws-smithy-types = { version = "1.3.4", features = ["http-body-1-x"] } backtrace = "0.3" base64 = "0.22" +betlang = "0.1.1" bitflags = "2.6.0" block = "0.1" block2 = "0.6" diff --git a/assets/settings/default.json b/assets/settings/default.json index 9ad2ee3271..2fc3b3d0a2 100644 --- a/assets/settings/default.json +++ b/assets/settings/default.json @@ -383,6 +383,9 @@ "minimum_split_diff_width": 100, // Show method signatures in the editor, when inside parentheses. "auto_signature_help": false, + // Whether to automatically detect the language of an untitled buffer from its contents. + // Languages explicitly selected from the language selector are not changed. + "language_detection": true, // Whether to show the signature help after completion or a bracket pair inserted. // If `auto_signature_help` is enabled, this setting will be treated as enabled also. "show_signature_help_after_edits": false, diff --git a/crates/editor/Cargo.toml b/crates/editor/Cargo.toml index 09467381bd..49624f8ad4 100644 --- a/crates/editor/Cargo.toml +++ b/crates/editor/Cargo.toml @@ -56,6 +56,7 @@ indoc.workspace = true edit_prediction_types.workspace = true itertools.workspace = true language.workspace = true +language_detection.workspace = true linkify.workspace = true log.workspace = true lsp.workspace = true diff --git a/crates/editor/src/editor.rs b/crates/editor/src/editor.rs index c1aea14d1d..2e5506e206 100644 --- a/crates/editor/src/editor.rs +++ b/crates/editor/src/editor.rs @@ -188,6 +188,7 @@ use language::{ }, point_from_lsp, point_to_lsp, text_diff_with_options, }; +use language_detection::detect_language; use linked_editing_ranges::refresh_linked_ranges; use lsp::{ CodeActionKind, CompletionItemKind, CompletionTriggerKind, InsertTextFormat, InsertTextMode, @@ -295,6 +296,8 @@ const CURSOR_BLINK_INTERVAL: Duration = Duration::from_millis(500); const MAX_LINE_LEN: usize = 1024; const MIN_NAVIGATION_HISTORY_ROW_DELTA: i64 = 10; const MAX_SELECTION_HISTORY_LEN: usize = 1024; +const MIN_LANGUAGE_DETECTION_LEN: usize = 20; +const LANGUAGE_DETECTION_DEBOUNCE_TIMEOUT: Duration = Duration::from_millis(200); pub(crate) const CURSORS_VISIBLE_FOR: Duration = Duration::from_millis(2000); #[doc(hidden)] pub const CODE_ACTIONS_DEBOUNCE_TIMEOUT: Duration = Duration::from_millis(250); @@ -1155,6 +1158,7 @@ pub struct Editor { next_scroll_position: NextScrollCursorCenterTopBottom, addons: TypeIdHashMap>, registered_buffers: HashMap, + language_detection_task: Task<()>, load_diff_task: Option>>, diff_hunk_delegate: Option>, selection_mark_mode: bool, @@ -2498,6 +2502,7 @@ impl Editor { next_scroll_position: NextScrollCursorCenterTopBottom::default(), addons: Default::default(), registered_buffers: HashMap::default(), + language_detection_task: Task::ready(()), _scroll_cursor_center_top_bottom_task: Task::ready(()), selection_mark_mode: false, toggle_fold_multiple_buffers: Task::ready(()), @@ -2931,6 +2936,9 @@ impl Editor { cx.spawn_in(window, async move |workspace, cx| { let buffer = create.await?; + buffer.update(cx, |buffer, _| { + buffer.set_content_language_detection_enabled(true); + }); workspace.update_in(cx, |workspace, window, cx| { let editor = cx.new(|cx| Editor::for_buffer(buffer, Some(project.clone()), window, cx)); @@ -2978,6 +2986,9 @@ impl Editor { cx.spawn_in(window, async move |workspace, cx| { let buffer = create.await?; + buffer.update(cx, |buffer, _| { + buffer.set_content_language_detection_enabled(true); + }); workspace.update_in(cx, move |workspace, window, cx| { workspace.split_item( direction, @@ -9860,8 +9871,9 @@ impl Editor { cx.emit(EditorEvent::TitleChanged); } + let buffer_id = buffer.read(cx).remote_id(); + if self.project.is_some() { - let buffer_id = buffer.read(cx).remote_id(); self.register_buffer(buffer_id, cx); self.update_lsp_data(Some(buffer_id), window, cx); self.refresh_inlay_hints( @@ -9869,6 +9881,8 @@ impl Editor { cx, ); } + + self.detect_buffer_language(buffer_id, cx); } cx.emit(EditorEvent::BufferEdited); @@ -11126,6 +11140,55 @@ impl Editor { self.refresh_document_symbols(for_buffer, cx); } + fn is_eligible_for_language_detection(buffer: &Buffer) -> bool { + buffer.file().is_none() + && buffer.content_language_detection_enabled() + && buffer.len() >= MIN_LANGUAGE_DETECTION_LEN + } + + fn detect_buffer_language(&mut self, buffer_id: BufferId, cx: &mut Context) { + self.language_detection_task = Task::ready(()); + if !EditorSettings::get_global(cx).language_detection { + return; + } + let Some(buffer_entity) = self.buffer().read(cx).buffer(buffer_id) else { + return; + }; + let buffer = buffer_entity.read(cx); + if !Self::is_eligible_for_language_detection(buffer) { + return; + } + self.language_detection_task = cx.spawn(async move |_, cx| { + cx.background_executor() + .timer(LANGUAGE_DETECTION_DEBOUNCE_TIMEOUT) + .await; + let Some((buffer_snapshot, language_registry)) = + buffer_entity.read_with(cx, |buffer, cx| { + if !EditorSettings::get_global(cx).language_detection + || !Self::is_eligible_for_language_detection(buffer) + { + return None; + } + Some((buffer.snapshot(), buffer.language_registry()?)) + }) + else { + return; + }; + let buffer_version = buffer_snapshot.version().clone(); + let detected_language = + cx.update(|cx| detect_language(buffer_snapshot, language_registry, cx)); + if let Some(detected_language) = detected_language.await { + buffer_entity.update(cx, |buffer, cx| { + if !buffer.version().changed_since(&buffer_version) + && Self::is_eligible_for_language_detection(buffer) + { + buffer.set_language(Some(detected_language), cx); + } + }); + } + }); + } + fn register_visible_buffers(&mut self, cx: &mut Context) { if !self.lsp_data_enabled() { return; diff --git a/crates/editor/src/editor_settings.rs b/crates/editor/src/editor_settings.rs index 06ca28e4f9..32b7a024d4 100644 --- a/crates/editor/src/editor_settings.rs +++ b/crates/editor/src/editor_settings.rs @@ -51,6 +51,7 @@ pub struct EditorSettings { pub search_wrap: bool, pub search: SearchSettings, pub auto_signature_help: bool, + pub language_detection: bool, pub show_signature_help_after_edits: bool, pub go_to_definition_fallback: GoToDefinitionFallback, pub go_to_definition_scroll_strategy: GoToDefinitionScrollStrategy, @@ -302,6 +303,7 @@ impl Settings for EditorSettings { search_on_type: search.search_on_type.unwrap(), }, auto_signature_help: editor.auto_signature_help.unwrap(), + language_detection: editor.language_detection.unwrap(), show_signature_help_after_edits: editor.show_signature_help_after_edits.unwrap(), go_to_definition_fallback: editor.go_to_definition_fallback.unwrap(), go_to_definition_scroll_strategy: editor.go_to_definition_scroll_strategy.unwrap(), diff --git a/crates/editor/src/editor_tests.rs b/crates/editor/src/editor_tests.rs index d489c5e3a9..5ffa82dc04 100644 --- a/crates/editor/src/editor_tests.rs +++ b/crates/editor/src/editor_tests.rs @@ -37,8 +37,7 @@ use language::{ tree_sitter_python, }; use language_settings::Formatter; -use languages::markdown_lang; -use languages::rust_lang; +use languages::{language, markdown_lang, rust_lang}; use lsp::{CompletionParams, DEFAULT_LSP_REQUEST_TIMEOUT}; use multi_buffer::{IndentGuide, MultiBuffer, MultiBufferOffset, MultiBufferOffsetUtf16, PathKey}; use parking_lot::Mutex; @@ -9999,6 +9998,147 @@ async fn test_kill_ring_yank_pastes_accumulated_kill_at_each_cursor(cx: &mut Tes cx.assert_editor_state("aone\nˇ bone\nˇ"); } +#[gpui::test] +async fn test_editing_untitled_buffer_redetects_language(cx: &mut TestAppContext) { + init_test(cx, |_| {}); + update_test_editor_settings(cx, &|settings| { + settings.language_detection = Some(false); + }); + + let fs = FakeFs::new(cx.executor()); + let project = Project::test(fs, [], cx).await; + let go_language = language("go", tree_sitter_go::LANGUAGE.into()); + project.read_with(cx, |project, _| { + project.languages().add(rust_lang()); + project.languages().add(go_language.clone()); + }); + let buffer = project + .update(cx, |project, cx| project.create_buffer(None, true, cx)) + .await + .unwrap(); + buffer.update(cx, |buffer, _| { + buffer.set_content_language_detection_enabled(true); + }); + let window = cx.add_window(|window, cx| { + let editor = Editor::for_buffer(buffer.clone(), None, window, cx); + window.focus(&editor.focus_handle(cx), cx); + editor + }); + let editor = window.root(cx).unwrap(); + let cx = &mut VisualTestContext::from_window(*window, cx); + + assert_eq!( + buffer.read_with(cx, |buffer, _| buffer.language().unwrap().name()), + PLAIN_TEXT.name() + ); + + editor.update_in(cx, |editor, window, cx| { + editor.insert("fn main() { println!(\"hello\"); }", window, cx); + }); + cx.run_until_parked(); + cx.executor() + .advance_clock(LANGUAGE_DETECTION_DEBOUNCE_TIMEOUT); + cx.run_until_parked(); + + assert_eq!( + buffer.read_with(cx, |buffer, _| buffer.language().unwrap().name()), + PLAIN_TEXT.name() + ); + + update_test_editor_settings(cx, &|settings| { + settings.language_detection = Some(true); + }); + + editor.update_in(cx, |editor, window, cx| { + editor.select_all(&SelectAll, window, cx); + editor.insert("fn main() {}", window, cx); + }); + cx.run_until_parked(); + + assert_eq!( + buffer.read_with(cx, |buffer, _| buffer.language().unwrap().name()), + PLAIN_TEXT.name() + ); + + editor.update_in(cx, |editor, window, cx| { + editor.select_all(&SelectAll, window, cx); + editor.insert("fn main() { println!(\"hello\"); }", window, cx); + }); + cx.run_until_parked(); + + cx.executor() + .advance_clock(LANGUAGE_DETECTION_DEBOUNCE_TIMEOUT / 2); + editor.update_in(cx, |editor, window, cx| { + editor.insert(" ", window, cx); + }); + cx.run_until_parked(); + + cx.executor() + .advance_clock(LANGUAGE_DETECTION_DEBOUNCE_TIMEOUT / 2); + cx.run_until_parked(); + + assert_eq!( + buffer.read_with(cx, |buffer, _| buffer.language().unwrap().name()), + PLAIN_TEXT.name() + ); + + cx.executor() + .advance_clock(LANGUAGE_DETECTION_DEBOUNCE_TIMEOUT / 2); + cx.run_until_parked(); + + assert_eq!( + buffer.read_with(cx, |buffer, _| buffer.language().unwrap().name()), + rust_lang().name() + ); + + editor.update_in(cx, |editor, window, cx| { + editor.select_all(&SelectAll, window, cx); + editor.backspace(&Backspace, window, cx); + }); + cx.run_until_parked(); + + editor.update_in(cx, |editor, window, cx| { + cx.write_to_clipboard(ClipboardItem::new_string( + "package main\n\nimport \"fmt\"\n\nfunc main() { fmt.Println(\"hello\") }".to_string(), + )); + editor.paste(&Paste, window, cx); + }); + cx.run_until_parked(); + + assert_eq!( + buffer.read_with(cx, |buffer, _| buffer.language().unwrap().name()), + rust_lang().name() + ); + + cx.executor() + .advance_clock(LANGUAGE_DETECTION_DEBOUNCE_TIMEOUT); + cx.run_until_parked(); + + assert_eq!( + buffer.read_with(cx, |buffer, _| buffer.language().unwrap().name()), + go_language.name() + ); + + project.update(cx, |project, cx| { + project.set_language_for_buffer(&buffer, PLAIN_TEXT.clone(), cx); + }); + + editor.update_in(cx, |editor, window, cx| { + editor.select_all(&SelectAll, window, cx); + editor.insert("fn main() { println!(\"hello\"); }", window, cx); + }); + + cx.run_until_parked(); + cx.executor() + .advance_clock(LANGUAGE_DETECTION_DEBOUNCE_TIMEOUT); + cx.run_until_parked(); + + assert_eq!( + buffer.read_with(cx, |buffer, _| buffer.language().unwrap().name()), + PLAIN_TEXT.name() + ); +} + #[gpui::test] async fn test_clipboard(cx: &mut TestAppContext) { init_test(cx, |_| {}); diff --git a/crates/editor/src/items.rs b/crates/editor/src/items.rs index e9fea6a4c8..163602bf2b 100644 --- a/crates/editor/src/items.rs +++ b/crates/editor/src/items.rs @@ -1319,6 +1319,7 @@ impl SerializableItem for Editor { } => window.spawn(cx, { let project = project.clone(); async move |cx| { + let content_language_detection_enabled = language.is_none(); let language_registry = project.read_with(cx, |project, _| project.languages().clone()); @@ -1341,6 +1342,9 @@ impl SerializableItem for Editor { // Then set the text so that the dirty bit is set correctly buffer.update(cx, |buffer, cx| { + if content_language_detection_enabled { + buffer.set_content_language_detection_enabled(true); + } buffer.set_language_registry(language_registry); buffer.set_text(contents, cx); if let Some(entry) = buffer.peek_undo_stack() { @@ -1430,12 +1434,18 @@ impl SerializableItem for Editor { SerializedEditor { abs_path: None, contents: None, + language, .. } => window.spawn(cx, async move |cx| { let buffer = project .update(cx, |project, cx| project.create_buffer(None, true, cx)) .await .context("Failed to create buffer")?; + if language.is_none() { + buffer.update(cx, |buffer, _| { + buffer.set_content_language_detection_enabled(true); + }); + } cx.update(|window, cx| { cx.new(|cx| { @@ -1489,6 +1499,8 @@ impl SerializableItem for Editor { let is_dirty = buffer.read(cx).is_dirty(); let mtime = buffer.read(cx).saved_mtime(); + let content_language_detection_enabled = + buffer.read(cx).content_language_detection_enabled(); let snapshot = buffer.read(cx).snapshot(); @@ -1496,7 +1508,13 @@ impl SerializableItem for Editor { Some(cx.background_spawn(async move { let (contents, language) = if serialize_dirty_buffers && is_dirty { let contents = snapshot.text(); - let language = snapshot.language().map(|lang| lang.name().to_string()); + let language = snapshot.language().and_then(|language| { + if content_language_detection_enabled && *language == *PLAIN_TEXT { + None + } else { + Some(language.name().to_string()) + } + }); (Some(contents), language) } else { (None, None) @@ -2864,6 +2882,7 @@ mod tests { buffer.language().map(|lang| lang.name()), Some("Rust".into()) ); // Language should be set to Rust + assert!(!buffer.content_language_detection_enabled()); assert!(buffer.file().is_none()); // The buffer should not have an associated file }); } @@ -2938,6 +2957,7 @@ mod tests { let buffer = editor.buffer().read(cx).as_singleton().unwrap().read(cx); assert!(buffer.file().is_none()); + assert!(buffer.content_language_detection_enabled()); }); } diff --git a/crates/language/src/buffer.rs b/crates/language/src/buffer.rs index 2ff465f6ed..9c6321addb 100644 --- a/crates/language/src/buffer.rs +++ b/crates/language/src/buffer.rs @@ -115,6 +115,7 @@ pub struct Buffer { was_dirty_before_starting_transaction: Option, reload_task: Option>>, language: Option>, + content_language_detection_enabled: bool, autoindent_requests: Vec>, wait_for_autoindent_txs: Vec>, pending_autoindent: Option>, @@ -1144,6 +1145,7 @@ impl Buffer { wait_for_autoindent_txs: Default::default(), pending_autoindent: Default::default(), language: None, + content_language_detection_enabled: false, remote_selections: Default::default(), diagnostics: Default::default(), diagnostics_timestamp: Lamport::MIN, @@ -1293,6 +1295,7 @@ impl Buffer { merged_operations: Default::default(), }), language: self.language.clone(), + content_language_detection_enabled: self.content_language_detection_enabled, has_conflict: self.has_conflict, has_unsaved_edits: Cell::new(self.has_unsaved_edits.get_mut().clone()), _subscriptions: vec![cx.subscribe(&this, Self::on_base_buffer_event)], @@ -1476,6 +1479,14 @@ impl Buffer { self.has_bom = has_bom; } + pub fn set_content_language_detection_enabled(&mut self, enabled: bool) { + self.content_language_detection_enabled = enabled; + } + + pub fn content_language_detection_enabled(&self) -> bool { + self.content_language_detection_enabled + } + /// Assign a language to the buffer. pub fn set_language_async(&mut self, language: Option>, cx: &mut Context) { self.set_language_(language, cfg!(any(test, feature = "test-support")), cx); diff --git a/crates/language_detection/Cargo.toml b/crates/language_detection/Cargo.toml new file mode 100644 index 0000000000..b2d80c6f16 --- /dev/null +++ b/crates/language_detection/Cargo.toml @@ -0,0 +1,17 @@ +[package] +name = "language_detection" +version = "0.1.0" +edition.workspace = true +publish.workspace = true +license = "GPL-3.0-or-later" + +[lints] +workspace = true + +[lib] +path = "src/language_detection.rs" + +[dependencies] +betlang.workspace = true +gpui.workspace = true +language.workspace = true diff --git a/crates/language_detection/LICENSE-GPL b/crates/language_detection/LICENSE-GPL new file mode 120000 index 0000000000..89e542f750 --- /dev/null +++ b/crates/language_detection/LICENSE-GPL @@ -0,0 +1 @@ +../../LICENSE-GPL \ No newline at end of file diff --git a/crates/language_detection/src/language_detection.rs b/crates/language_detection/src/language_detection.rs new file mode 100644 index 0000000000..9dec72cc54 --- /dev/null +++ b/crates/language_detection/src/language_detection.rs @@ -0,0 +1,137 @@ +use gpui::{App, AppContext, Task}; +use language::{BufferSnapshot, Language, LanguageRegistry}; +use std::sync::Arc; + +const SAMPLE_BLOCK_SIZE: usize = 4096; +const MIN_LANGUAGE_DETECTION_CONFIDENCE: f32 = 0.2; +const MIN_LANGUAGE_DETECTION_CONFIDENCE_GAP: f32 = 0.2; +const MIN_LANGUAGE_SWITCH_CONFIDENCE_GAP: f32 = 0.5; + +fn language_registry_key(language: betlang::Language) -> Option<&'static str> { + let language_name = match language { + betlang::Language::Asm => "Assembly", + betlang::Language::C => "C", + betlang::Language::Clojure => "Clojure", + betlang::Language::CMake => "CMake", + betlang::Language::Cobol => "COBOL", + betlang::Language::Cpp => "C++", + betlang::Language::Cs => "CSharp", + betlang::Language::Css => "CSS", + betlang::Language::Dart => "Dart", + betlang::Language::Dockerfile => "Dockerfile", + betlang::Language::Elixir => "Elixir", + betlang::Language::Erlang => "Erlang", + betlang::Language::Gemfile => "Ruby", + betlang::Language::Gemspec => "Ruby", + betlang::Language::Go => "Go", + betlang::Language::Gradle => "Groovy", + betlang::Language::Groovy => "Groovy", + betlang::Language::Haskell => "Haskell", + betlang::Language::Html => "HTML", + betlang::Language::Ini => "ini", + betlang::Language::Java => "Java", + betlang::Language::JavaScript => "JavaScript", + betlang::Language::Json => "JSON", + betlang::Language::Julia => "Julia", + betlang::Language::Kotlin => "Kotlin", + betlang::Language::Lua => "Lua", + betlang::Language::Markdown => "Markdown", + betlang::Language::ObjectiveC => "Objective-C", + betlang::Language::Ocaml => "OCaml", + betlang::Language::Perl => "Perl", + betlang::Language::Php => "PHP", + betlang::Language::Powershell => "PowerShell", + betlang::Language::Python => "Python", + betlang::Language::R => "R", + betlang::Language::Ruby => "Ruby", + betlang::Language::Rust => "Rust", + betlang::Language::Scala => "Scala", + betlang::Language::Shell => "Shell Script", + betlang::Language::Sql => "SQL", + betlang::Language::Swift => "Swift", + betlang::Language::Toml => "TOML", + betlang::Language::TypeScript => "TypeScript", + betlang::Language::Verilog => "SystemVerilog", + betlang::Language::Xml => "XML", + betlang::Language::Yaml => "YAML", + _ => return None, + }; + Some(language_name) +} + +pub fn detect_language( + buffer: BufferSnapshot, + language_registry: Arc, + cx: &mut App, +) -> Task>> { + let source = extract_sample(&buffer); + let current_language_name = buffer.language().map(|language| language.name()); + cx.background_spawn(async move { + let detection = betlang::detect(source); + let (mut pending_languages, mut confirmed_languages) = (Vec::new(), Vec::new()); + // As in VS Code, retain only high-confidence candidate groups followed by a clear confidence gap. + for (score, language) in detection.top_languages() { + if pending_languages.last().is_some_and(|(previous_score, _)| { + *previous_score - score >= MIN_LANGUAGE_DETECTION_CONFIDENCE_GAP + }) { + confirmed_languages.append(&mut pending_languages); + } + if score < MIN_LANGUAGE_DETECTION_CONFIDENCE { + break; + } + pending_languages.push((score, language)); + } + let detected_language = + confirmed_languages + .into_iter() + .find_map(|(score, model_language)| { + language_registry + .available_language_for_name(language_registry_key(model_language)?) + .map(|language| (score, language)) + }); + let Some((score, language)) = detected_language else { + return None; + }; + if current_language_name == Some(language.name()) { + return None; + } + let current_language_score = current_language_name.and_then(|current_name| { + detection + .top_languages() + .find_map(|(score, model_language)| { + language_registry_key(model_language) + .is_some_and(|name| name == current_name.as_ref()) + .then_some(score) + }) + }); + // Prefer the current language unless another candidate has a clear confidence advantage. + if current_language_score.is_some_and(|current_score| { + score <= current_score + MIN_LANGUAGE_SWITCH_CONFIDENCE_GAP + }) { + return None; + } + language_registry + .load_language(language.id()) + .await + .ok() + .and_then(|language| language.ok()) + }) +} + +fn extract_sample(buffer: &BufferSnapshot) -> Vec { + let source_length = buffer.len(); + let ranges = if source_length <= SAMPLE_BLOCK_SIZE * 2 { + vec![0..source_length] + } else { + vec![ + 0..SAMPLE_BLOCK_SIZE, + source_length - SAMPLE_BLOCK_SIZE..source_length, + ] + }; + + ranges + .into_iter() + .flat_map(|range| buffer.bytes_in_range(range)) + .flat_map(|chunk| chunk.iter().copied()) + .collect() +} diff --git a/crates/project/src/lsp_store.rs b/crates/project/src/lsp_store.rs index e444f18418..96b8474bc4 100644 --- a/crates/project/src/lsp_store.rs +++ b/crates/project/src/lsp_store.rs @@ -5194,15 +5194,17 @@ impl LspStore { } this.update(cx, |this, cx| { - let mut plain_text_buffers = Vec::new(); + let mut buffers_to_detect = Vec::new(); let mut buffers_with_language = Vec::new(); let mut buffers_with_unknown_injections = Vec::new(); for handle in this.buffer_store.read(cx).buffers() { let buffer = handle.read(cx); if buffer.language().is_none() || buffer.language() == Some(&*language::PLAIN_TEXT) + || (buffer.file().is_some() + && buffer.content_language_detection_enabled()) { - plain_text_buffers.push(handle); + buffers_to_detect.push(handle); } else { if buffer.contains_unknown_injections() { buffers_with_unknown_injections.push(handle.clone()); @@ -5213,14 +5215,14 @@ impl LspStore { // Deprioritize the invisible worktrees so main worktrees' language servers can be started first, // and reused later in the invisible worktrees. - plain_text_buffers.sort_by_key(|buffer| { + buffers_to_detect.sort_by_key(|buffer| { Reverse( File::from_dyn(buffer.read(cx).file()) .map(|file| file.worktree.read(cx).is_visible()), ) }); - for buffer in plain_text_buffers { + for buffer in buffers_to_detect { this.detect_language_for_buffer(&buffer, cx); if let Some(local) = this.as_local_mut() { local.initialize_buffer(&buffer, cx); @@ -5342,6 +5344,9 @@ impl LspStore { if let Some(Ok(Ok(new_language))) = self.languages.load_language(language_id).now_or_never() { + buffer_handle.update(cx, |buffer, _| { + buffer.set_content_language_detection_enabled(false); + }); self.set_language_for_buffer(buffer_handle, new_language, cx); } } else { diff --git a/crates/project/src/project.rs b/crates/project/src/project.rs index 89375b306d..ed9194548f 100644 --- a/crates/project/src/project.rs +++ b/crates/project/src/project.rs @@ -4122,6 +4122,9 @@ impl Project { new_language: Arc, cx: &mut Context, ) { + buffer.update(cx, |buffer, _| { + buffer.set_content_language_detection_enabled(false); + }); self.lsp_store.update(cx, |lsp_store, cx| { lsp_store.set_language_for_buffer(buffer, new_language, cx) }) diff --git a/crates/settings/src/vscode_import.rs b/crates/settings/src/vscode_import.rs index 4051de95e9..3e25d4ecc6 100644 --- a/crates/settings/src/vscode_import.rs +++ b/crates/settings/src/vscode_import.rs @@ -245,6 +245,7 @@ impl VsCodeSettings { fn editor_settings_content(&self) -> EditorSettingsContent { EditorSettingsContent { auto_signature_help: self.read_bool("editor.parameterHints.enabled"), + language_detection: self.read_bool("workbench.editor.languageDetection"), autoscroll_on_clicks: None, cursor_blink: self.read_enum("editor.cursorBlinking", |s| match s { "blink" | "phase" | "expand" | "smooth" => Some(true), diff --git a/crates/settings_content/src/editor.rs b/crates/settings_content/src/editor.rs index aa965976f9..3abd295278 100644 --- a/crates/settings_content/src/editor.rs +++ b/crates/settings_content/src/editor.rs @@ -169,6 +169,12 @@ pub struct EditorSettingsContent { /// Default: false pub auto_signature_help: Option, + /// Whether to automatically detect the language of an untitled buffer from its contents. + /// Languages explicitly selected from the language selector are not changed. + /// + /// Default: true + pub language_detection: Option, + /// Whether to show the signature help pop-up after completions or bracket pairs inserted. /// /// Default: false diff --git a/crates/settings_ui/src/page_data.rs b/crates/settings_ui/src/page_data.rs index 66add381b9..5f00e971df 100644 --- a/crates/settings_ui/src/page_data.rs +++ b/crates/settings_ui/src/page_data.rs @@ -10035,9 +10035,23 @@ fn language_settings_data() -> Box<[SettingsPageItem]> { ] } - fn miscellaneous_section() -> [SettingsPageItem; 7] { + fn miscellaneous_section() -> [SettingsPageItem; 8] { [ SettingsPageItem::SectionHeader("Miscellaneous"), + SettingsPageItem::SettingItem(SettingItem { + title: "Language Detection", + description: "Whether to enable automatic language detection in unsaved buffers.", + field: Box::new(SettingField { + organization_override: None, + json_path: Some("language_detection"), + pick: |settings_content| settings_content.editor.language_detection.as_ref(), + write: |settings_content, value, _| { + settings_content.editor.language_detection = value; + }, + }), + metadata: None, + files: USER, + }), SettingsPageItem::SettingItem(SettingItem { title: "Word Diff Enabled", description: "Whether to enable word diff highlighting in the editor. When enabled, changed words within modified lines are highlighted to show exactly what changed.", diff --git a/crates/zed/src/zed.rs b/crates/zed/src/zed.rs index d177a67800..0f7c563b39 100644 --- a/crates/zed/src/zed.rs +++ b/crates/zed/src/zed.rs @@ -1345,6 +1345,9 @@ fn register_actions( let buffer = project.update(cx, |project, cx| { project.create_local_buffer("", None, true, cx) }); + buffer.update(cx, |buffer, _| { + buffer.set_content_language_detection_enabled(true); + }); let editor = cx.new(|cx| { Editor::for_buffer(buffer, Some(project), window, cx) }); diff --git a/docs/src/migrate/vs-code.md b/docs/src/migrate/vs-code.md index e77f460763..588f2ce46c 100644 --- a/docs/src/migrate/vs-code.md +++ b/docs/src/migrate/vs-code.md @@ -62,6 +62,7 @@ The following VS Code settings are automatically imported when you use **Import | `editor.hover.sticky` | `hover_popover_sticky` | | `editor.hover.hidingDelay` | `hover_popover_hiding_delay` | | `editor.parameterHints.enabled` | `auto_signature_help` | +| `workbench.editor.languageDetection` | `language_detection` | | `editor.multiCursorModifier` | `multi_cursor_modifier` | | `editor.selectionHighlight` | `selection_highlight` | | `editor.roundedSelection` | `rounded_selection` | diff --git a/docs/src/reference/all-settings.md b/docs/src/reference/all-settings.md index 9b53319a69..ecd9aca7d9 100644 --- a/docs/src/reference/all-settings.md +++ b/docs/src/reference/all-settings.md @@ -3130,6 +3130,16 @@ Unspecified values have a `false` value, hints won't be toggled if all the modif - `enabled`: Whether to enable automatic JSX tag closing +## Language Detection + +- Description: Whether to automatically detect the language of an untitled buffer from its contents. Languages explicitly selected from the language selector are not changed. +- Setting: `language_detection` +- Default: `true` + +**Options** + +`boolean` values + ## Languages - Description: Configuration for specific languages.