d2e52d12e1a88dac7718b4ae2c6e1d069ab2fa6a
2
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Xin Zhao
|
d2e52d12e1 |
grammars: Add semantic token rule for Python constants (#59811)
# Objective In Python, using variables in all uppercase letters signals that the variable is a constant. In Zed's built-in Python grammar, this convention is respected. Similarly, the `ty` language server has a rule for this: https://github.com/astral-sh/ruff/blob/e39ded7e89f503d892217364f924b38362fb9a66/crates/ty_ide/src/semantic_tokens.rs#L264-L267 https://github.com/astral-sh/ruff/blob/e39ded7e89f503d892217364f924b38362fb9a66/crates/ty_ide/src/semantic_tokens.rs#L356-L358 It returns a `"readonly"` token modifier if an item is determined to be a constant. However, this specific modifier is not currently respected in Zed. Consequently, when using `ty` as the language server (which is expected to be a common setup in the near future, similar to `ruff` and `uv`), and enable semantic token highlighting, these constants are highlighted as normal variables. ## Solution - Added a new semantic token rule in `crates/grammars/src/python/semantic_token_rules.json` to provide highlighting for constants. ## Testing - This change primarily affects the UI and has been tested locally. A before-and-after comparison is provided below in the Showcase section. ## Self-Review Checklist: - [x] I've reviewed my own diff for quality, security, and reliability - [x] Unsafe blocks (if any) have justifying comments - [x] The content adheres to Zed's UI standards ([UX/UI](https://github.com/zed-industries/zed/blob/main/CONTRIBUTING.md#uiux-checklist) and [icon](https://github.com/zed-industries/zed/blob/main/crates/icons/README.md) guidelines) - [ ] Tests cover the new/changed behavior - [x] Performance impact has been considered and is acceptable ## Showcase <details> <summary>Click to view showcase</summary> | Before | After | | :---: | :---: | | <img width="441" height="208" alt="highlight_before" src="https://github.com/user-attachments/assets/64cc28ba-32b6-4620-a212-8f7d43f54821" /> | <img width="441" height="208" alt="highlight_after" src="https://github.com/user-attachments/assets/133919fd-03c8-4f6c-9bf9-9efa2610f050" /> | | <img width="498" height="145" alt="highlight_tree_before" src="https://github.com/user-attachments/assets/b79f4036-65e4-4e4e-b0e9-e653529e114d" /> | <img width="498" height="145" alt="highlight_tree_after" src="https://github.com/user-attachments/assets/e0c3ae74-46b4-4d06-895b-51da69d410c5" /> | </details> --- Release Notes: - Improved semantic highlighting for Python constants when using the `ty` language server. |
||
|
|
ab3697f163 |
Extract language_core and grammars crates from language (#52238)
This extracts a `language_core` crate from the existing `language`
crate, and creates a `grammars` data crate. The goal is to separate
tree-sitter grammar infrastructure, language configuration, and LSP
adapter types from the heavier buffer/editor integration layer in
`language`.
## Motivation
The `language` crate pulls in `text`, `theme`, `settings`, `rpc`,
`task`, `fs`, `clock`, `sum_tree`, and `fuzzy` — all of which are needed
for buffer integration (`Buffer`, `SyntaxMap`, `Outline`,
`DiagnosticSet`) but not for grammar parsing or language configuration.
Extracting the core types lets downstream consumers depend on
`language_core` without pulling in the full integration stack.
## Dependency graph after extraction
```
language_core ← gpui, lsp, tree-sitter, util, collections
grammars ← language_core, rust_embed, tree-sitter-{rust,python,...}
language ← language_core, text, theme, settings, rpc, task, fs, ...
languages ← language, grammars
```
## What moved to `language_core`
- `Grammar`, `GrammarId`, and all query config/builder types
- `LanguageConfig`, `LanguageMatcher`, bracket/comment/indent config
types
- `HighlightMap`, `HighlightId` (theme-dependent free functions
`highlight_style` and `highlight_name` stay in `language`)
- `LanguageName`, `LanguageId`
- `LanguageQueries`, `QUERY_FILENAME_PREFIXES`
- `CodeLabel`, `CodeLabelBuilder`, `Symbol`
- `Diagnostic`, `DiagnosticSourceKind`
- `Toolchain`, `ToolchainScope`, `ToolchainList`, `ToolchainMetadata`
- `ManifestName`
- `SoftWrap`
- LSP data types: `BinaryStatus`, `ServerHealth`,
`LanguageServerStatusUpdate`, `PromptResponseContext`, `ToLspPosition`
## What stays in `language`
- `Buffer`, `BufferSnapshot`, `SyntaxMap`, `Outline`, `DiagnosticSet`,
`LanguageScope`
- `LspAdapter`, `CachedLspAdapter`, `LspAdapterDelegate` (reference
`Arc<Language>` and `WorktreeId`)
- `ToolchainLister`, `LanguageToolchainStore` (reference `task` and
`settings` types)
- `ManifestQuery`, `ManifestProvider`, `ManifestDelegate` (reference
`WorktreeId`)
- Parser/query cursor pools, `PLAIN_TEXT`, point conversion functions
## What the `grammars` crate provides
- Embedded `.scm` query files and `config.toml` files for all built-in
languages (via `rust_embed`)
- `load_queries(name)`, `load_config(name)`,
`load_config_for_feature(name, grammars_loaded)`, and `get_file(path)`
functions
- `native_grammars()` for tree-sitter grammar registration (behind
`load-grammars` feature)
## Pre-cleanup (also in this PR)
- Removed unused `Option<&Buffer>` from
`LspAdapter::process_diagnostics`
- Removed unused `&App` from `LspAdapter::retain_old_diagnostic`
- Removed `fs: &dyn Fs` from `ToolchainLister` trait methods
(`PythonToolchainProvider` captures `fs` at construction time instead)
- Moved `Diagnostic`/`DiagnosticSourceKind` out of `buffer.rs` into
their own module
## Backward compatibility
The `language` crate re-exports everything from `language_core`, so
existing `use language::Grammar` (etc.) continues to work unchanged. The
only downstream change required is importing `CodeLabelExt` where
`.fallback_for_completion()` is called on the now-foreign `CodeLabel`
type.
Release Notes:
- N/A
---------
Co-authored-by: Agus Zubiaga <agus@zed.dev>
Co-authored-by: Tom Houlé <tom@tomhoule.com>
|