editor: Add language detection for untitled buffers (#61412)

# Objective

Closes #4868.

Reintroduce automatic language detection for user-created untitled
buffers without allowing it to override language choices made by users
or owning surfaces.

More details:
+ https://github.com/zed-industries/zed/pull/61201
+ https://github.com/zed-industries/zed/pull/61351

## Solution

VS Code distinguishes between inferred and explicitly selected
languages, and only continues automatic detection in the former state.
This change introduces a buffer-local opt-in flag with the same purpose.
It defaults to disabled and is enabled only for user-created untitled
buffers and restored untitled buffers that did not persist a language.
Choosing a language through the language picker disables detection,
while automatically detected languages remain eligible for later
redetection. Buffers whose language is assigned by their owning surface,
such as Channel Notes, never opt in.

Rather than adding persisted state or changing the database schema,
restoration reuses the existing optional serialized language. An
opted-in buffer that remains Plain Text is serialized without a language
and resumes detection after restoration. Once automatic detection has
selected a language, that language is serialized normally and
restoration leaves detection disabled, effectively treating it as an
explicit selection. This also matches VS Code's effective default after
restoration.

Candidate selection adopts VS Code's confidence-group filtering. Results
below the minimum confidence are discarded, and candidate groups are
accepted only after a clear confidence gap. The detector then selects
the first available Zed language without eagerly loading every
candidate. I then added a separate switching rule: Zed keeps the current
language unless another candidate has a clear confidence advantage,
reducing language changes as model scores fluctuate during editing.
Model inference runs off-thread on a bounded sample, and stale results
are discarded.

I also evaluated penalizing YAML and using stricter global switching
thresholds, but neither reliably resolves the ambiguity between Markdown
and YAML. In the reproduction from #61351, Betlang assigns YAML more
than 80% confidence during editing and peaks around 90%. This is
reasonable from the content alone: `#` denotes a YAML comment, while `-`
denotes a sequence item. Adding ordinary prose or a fenced code block
shifts the model strongly toward Markdown. A fixed penalty would overfit
this example and risk suppressing legitimate YAML detection. Since
known-language buffers do not opt in and manual language selection takes
precedence, I consider this level of ambiguity acceptable and prefer not
to introduce language-specific score tuning.

Model inference itself is fast. As noted in #61201:

> In release builds on Linux with an Intel Core i5-13600KF, the
`betlang::detect` call alone typically completes within 3 ms when
processing the maximum sampled input.

The main concern is the work surrounding inference. Without debouncing,
rapid typing creates a snapshot and launches a background detection task
on every keystroke, while the detected language may change before the
user has finished typing. This change therefore adds a 200 ms debounce
delay, allowing detection to run and update the language only after the
user pauses.

## Testing

Automated coverage verifies that short inputs remain Plain Text,
sufficient Rust content is detected as Rust, replacing it with Go
content triggers redetection, and manually selecting Plain Text prevents
subsequent automatic changes. Restoration coverage verifies that a
stored language keeps detection disabled, while an empty language-less
untitled buffer resumes detection.

Beyond the automated tests, I manually tested automatic language
detection while editing untitled buffers.

## Self-Review Checklist:

- [x] I've reviewed my own diff for quality, security, and reliability
- [x] Unsafe blocks (if any) have justifying comments
- [x] The content adheres to Zed's UI standards
([UX/UI](https://github.com/zed-industries/zed/blob/main/CONTRIBUTING.md#uiux-checklist)
and
[icon](https://github.com/zed-industries/zed/blob/main/crates/icons/README.md)
guidelines)
- [x] Tests cover the new/changed behavior
- [x] Performance impact has been considered and is acceptable

## Showcase


https://github.com/user-attachments/assets/103fc731-e12e-48d6-ae89-03978f1c9d74




---

Release Notes:

- Added automatic language detection for untitled buffers.

---------

Co-authored-by: Max Stevens <maxstevens2708@gmail.com>
Co-authored-by: Christopher Biscardi <chris@christopherbiscardi.com>

docs/src/migrate/vs-code.md

@@ -62,6 +62,7 @@ The following VS Code settings are automatically imported when you use **Import
 | `editor.hover.sticky`                       | `hover_popover_sticky`                         |
 | `editor.hover.hidingDelay`                  | `hover_popover_hiding_delay`                   |
 | `editor.parameterHints.enabled`             | `auto_signature_help`                          |
+| `workbench.editor.languageDetection`        | `language_detection`                           |
 | `editor.multiCursorModifier`                | `multi_cursor_modifier`                        |
 | `editor.selectionHighlight`                 | `selection_highlight`                          |
 | `editor.roundedSelection`                   | `rounded_selection`                            |

docs/src/reference/all-settings.md

@@ -3130,6 +3130,16 @@ Unspecified values have a `false` value, hints won't be toggled if all the modif
 
 - `enabled`: Whether to enable automatic JSX tag closing
 
+## Language Detection
+
+- Description: Whether to automatically detect the language of an untitled buffer from its contents. Languages explicitly selected from the language selector are not changed.
+- Setting: `language_detection`
+- Default: `true`
+
+**Options**
+
+`boolean` values
+
 ## Languages
 
 - Description: Configuration for specific languages.