language_core and grammars crates from language (#52238)
This extracts a `language_core` crate from the existing `language`
crate, and creates a `grammars` data crate. The goal is to separate
tree-sitter grammar infrastructure, language configuration, and LSP
adapter types from the heavier buffer/editor integration layer in
`language`.
## Motivation
The `language` crate pulls in `text`, `theme`, `settings`, `rpc`,
`task`, `fs`, `clock`, `sum_tree`, and `fuzzy` — all of which are needed
for buffer integration (`Buffer`, `SyntaxMap`, `Outline`,
`DiagnosticSet`) but not for grammar parsing or language configuration.
Extracting the core types lets downstream consumers depend on
`language_core` without pulling in the full integration stack.
## Dependency graph after extraction
```
language_core ← gpui, lsp, tree-sitter, util, collections
grammars ← language_core, rust_embed, tree-sitter-{rust,python,...}
language ← language_core, text, theme, settings, rpc, task, fs, ...
languages ← language, grammars
```
## What moved to `language_core`
- `Grammar`, `GrammarId`, and all query config/builder types
- `LanguageConfig`, `LanguageMatcher`, bracket/comment/indent config
types
- `HighlightMap`, `HighlightId` (theme-dependent free functions
`highlight_style` and `highlight_name` stay in `language`)
- `LanguageName`, `LanguageId`
- `LanguageQueries`, `QUERY_FILENAME_PREFIXES`
- `CodeLabel`, `CodeLabelBuilder`, `Symbol`
- `Diagnostic`, `DiagnosticSourceKind`
- `Toolchain`, `ToolchainScope`, `ToolchainList`, `ToolchainMetadata`
- `ManifestName`
- `SoftWrap`
- LSP data types: `BinaryStatus`, `ServerHealth`,
`LanguageServerStatusUpdate`, `PromptResponseContext`, `ToLspPosition`
## What stays in `language`
- `Buffer`, `BufferSnapshot`, `SyntaxMap`, `Outline`, `DiagnosticSet`,
`LanguageScope`
- `LspAdapter`, `CachedLspAdapter`, `LspAdapterDelegate` (reference
`Arc<Language>` and `WorktreeId`)
- `ToolchainLister`, `LanguageToolchainStore` (reference `task` and
`settings` types)
- `ManifestQuery`, `ManifestProvider`, `ManifestDelegate` (reference
`WorktreeId`)
- Parser/query cursor pools, `PLAIN_TEXT`, point conversion functions
## What the `grammars` crate provides
- Embedded `.scm` query files and `config.toml` files for all built-in
languages (via `rust_embed`)
- `load_queries(name)`, `load_config(name)`,
`load_config_for_feature(name, grammars_loaded)`, and `get_file(path)`
functions
- `native_grammars()` for tree-sitter grammar registration (behind
`load-grammars` feature)
## Pre-cleanup (also in this PR)
- Removed unused `Option<&Buffer>` from
`LspAdapter::process_diagnostics`
- Removed unused `&App` from `LspAdapter::retain_old_diagnostic`
- Removed `fs: &dyn Fs` from `ToolchainLister` trait methods
(`PythonToolchainProvider` captures `fs` at construction time instead)
- Moved `Diagnostic`/`DiagnosticSourceKind` out of `buffer.rs` into
their own module
## Backward compatibility
The `language` crate re-exports everything from `language_core`, so
existing `use language::Grammar` (etc.) continues to work unchanged. The
only downstream change required is importing `CodeLabelExt` where
`.fallback_for_completion()` is called on the now-foreign `CodeLabel`
type.
Release Notes:
- N/A
---------
Co-authored-by: Agus Zubiaga <agus@zed.dev>
Co-authored-by: Tom Houlé <tom@tomhoule.com>
Zed
Welcome to Zed, a high-performance, multiplayer code editor from the creators of Atom and Tree-sitter.
Installation
On macOS, Linux, and Windows you can download Zed directly or install Zed via your local package manager (macOS/Linux/Windows).
Other platforms are not yet available:
- Web (tracking issue)
Developing Zed
Contributing
See CONTRIBUTING.md for ways you can contribute to Zed.
Also... we're hiring! Check out our jobs page for open roles.
Licensing
License information for third party dependencies must be correctly provided for CI to pass.
We use cargo-about to automatically comply with open source licenses. If CI is failing, check the following:
- Is it showing a
no license specifiederror for a crate you've created? If so, addpublish = falseunder[package]in your crate's Cargo.toml. - Is the error
failed to satisfy license requirementsfor a dependency? If so, first determine what license the project has and whether this system is sufficient to comply with this license's requirements. If you're unsure, ask a lawyer. Once you've verified that this system is acceptable add the license's SPDX identifier to theacceptedarray inscript/licenses/zed-licenses.toml. - Is
cargo-aboutunable to find the license for a dependency? If so, add a clarification field at the end ofscript/licenses/zed-licenses.toml, as specified in the cargo-about book.
Sponsorship
Zed is developed by Zed Industries, Inc., a for-profit company.
If you’d like to financially support the project, you can do so via GitHub Sponsors. Sponsorships go directly to Zed Industries and are used as general company revenue. There are no perks or entitlements associated with sponsorship.