This adds a dedicated `AvailableLanguages` struct in preparation for a
more cabable language matching based on a given language config. No
functional changes, just shuffling some code around for this round.
Also made the LanguageMatcher non-cloneable in favor of wrapping it in
an Arc, since cloning is rather expensive for this and having a
reference is sufficient in all cases right now.
Release Notes:
- N/A
---------
Co-authored-by: Ben Brandt <benjamin.j.brandt@gmail.com>
Closes #ISSUE
Before you mark this PR as ready for review, make sure that you have:
- [ ] Added a solid test coverage and/or screenshots from doing manual
testing
- [ ] Done a self-review taking into account security and performance
aspects
- [ ] Aligned any UI changes with the [UI
checklist](https://github.com/zed-industries/zed/blob/main/CONTRIBUTING.md#uiux-checklist)
Release Notes:
- N/A
This extracts a `language_core` crate from the existing `language`
crate, and creates a `grammars` data crate. The goal is to separate
tree-sitter grammar infrastructure, language configuration, and LSP
adapter types from the heavier buffer/editor integration layer in
`language`.
## Motivation
The `language` crate pulls in `text`, `theme`, `settings`, `rpc`,
`task`, `fs`, `clock`, `sum_tree`, and `fuzzy` — all of which are needed
for buffer integration (`Buffer`, `SyntaxMap`, `Outline`,
`DiagnosticSet`) but not for grammar parsing or language configuration.
Extracting the core types lets downstream consumers depend on
`language_core` without pulling in the full integration stack.
## Dependency graph after extraction
```
language_core ← gpui, lsp, tree-sitter, util, collections
grammars ← language_core, rust_embed, tree-sitter-{rust,python,...}
language ← language_core, text, theme, settings, rpc, task, fs, ...
languages ← language, grammars
```
## What moved to `language_core`
- `Grammar`, `GrammarId`, and all query config/builder types
- `LanguageConfig`, `LanguageMatcher`, bracket/comment/indent config
types
- `HighlightMap`, `HighlightId` (theme-dependent free functions
`highlight_style` and `highlight_name` stay in `language`)
- `LanguageName`, `LanguageId`
- `LanguageQueries`, `QUERY_FILENAME_PREFIXES`
- `CodeLabel`, `CodeLabelBuilder`, `Symbol`
- `Diagnostic`, `DiagnosticSourceKind`
- `Toolchain`, `ToolchainScope`, `ToolchainList`, `ToolchainMetadata`
- `ManifestName`
- `SoftWrap`
- LSP data types: `BinaryStatus`, `ServerHealth`,
`LanguageServerStatusUpdate`, `PromptResponseContext`, `ToLspPosition`
## What stays in `language`
- `Buffer`, `BufferSnapshot`, `SyntaxMap`, `Outline`, `DiagnosticSet`,
`LanguageScope`
- `LspAdapter`, `CachedLspAdapter`, `LspAdapterDelegate` (reference
`Arc<Language>` and `WorktreeId`)
- `ToolchainLister`, `LanguageToolchainStore` (reference `task` and
`settings` types)
- `ManifestQuery`, `ManifestProvider`, `ManifestDelegate` (reference
`WorktreeId`)
- Parser/query cursor pools, `PLAIN_TEXT`, point conversion functions
## What the `grammars` crate provides
- Embedded `.scm` query files and `config.toml` files for all built-in
languages (via `rust_embed`)
- `load_queries(name)`, `load_config(name)`,
`load_config_for_feature(name, grammars_loaded)`, and `get_file(path)`
functions
- `native_grammars()` for tree-sitter grammar registration (behind
`load-grammars` feature)
## Pre-cleanup (also in this PR)
- Removed unused `Option<&Buffer>` from
`LspAdapter::process_diagnostics`
- Removed unused `&App` from `LspAdapter::retain_old_diagnostic`
- Removed `fs: &dyn Fs` from `ToolchainLister` trait methods
(`PythonToolchainProvider` captures `fs` at construction time instead)
- Moved `Diagnostic`/`DiagnosticSourceKind` out of `buffer.rs` into
their own module
## Backward compatibility
The `language` crate re-exports everything from `language_core`, so
existing `use language::Grammar` (etc.) continues to work unchanged. The
only downstream change required is importing `CodeLabelExt` where
`.fallback_for_completion()` is called on the now-foreign `CodeLabel`
type.
Release Notes:
- N/A
---------
Co-authored-by: Agus Zubiaga <agus@zed.dev>
Co-authored-by: Tom Houlé <tom@tomhoule.com>
Gist is we only need to block the foreground thread for reparsing if
immediate language changes are useful to the user. That is usually only
the case when they edit the buffer
Release Notes:
- Improved performance of large project searches and project diffs
Co-authored by: David Kleingeld <david@zed.dev>
Adds debugger inline values support for JavaScript, TypeScript, and TSX languages.
Release Notes:
- debugger: Add inline value support for Javascript, TypeScript, and TSX
---------
Co-authored-by: Anthony <anthony@zed.dev>
Closes https://github.com/zed-industries/zed/issues/38690Closes#37353
### Background
On Windows, paths are normally separated by `\`, unlike mac and linux
where they are separated by `/`. When editing code in a project that
uses a different path style than your local system (e.g. remoting from
Windows to Linux, using WSL, and collaboration between windows and unix
users), the correct separator for a path may differ from the "native"
separator.
Previously, to work around this, Zed converted paths' separators in
numerous places. This was applied to both absolute and relative paths,
leading to incorrect conversions in some cases.
### Solution
Many code paths in Zed use paths that are *relative* to either a
worktree root or a git repository. This PR introduces a dedicated type
for these paths called `RelPath`, which stores the path in the same way
regardless of host platform, and offers `Path`-like manipulation APIs.
RelPath supports *displaying* the path using either separator, so that
we can display paths in a style that is determined at runtime based on
the current project.
The representation of absolute paths is left untouched, for now.
Absolute paths are different from relative paths because (except in
contexts where we know that the path refers to the local filesystem)
they should generally be treated as opaque strings. Currently we use a
mix of types for these paths (std::path::Path, String, SanitizedPath).
Release Notes:
- N/A
---------
Co-authored-by: Cole Miller <cole@zed.dev>
Co-authored-by: Piotr Osiewicz <24362066+osiewicz@users.noreply.github.com>
Co-authored-by: Peter Tripp <petertripp@gmail.com>
Co-authored-by: Smit Barmase <heysmitbarmase@gmail.com>
Co-authored-by: Lukas Wirth <me@lukaswirth.dev>
## Context
To support inline values a language will have to implement their own
provider trait that walks through tree sitter nodes. This is overly
complicated, hard to accurately implement for each language, and lacks
proper extension support.
This PR switches to a singular inline provider that uses a language's
`debugger.scm` query field to capture variables and scopes. The inline
provider is able to use this information to generate inlays that take
scope into account and work with any language that defines a debugger
query file.
### Todos
- [x] Implement a utility test function to easily test inline values
- [x] Generate inline values based on captures
- [x] Reimplement Python, Rust, and Go support
- [x] Take scope into account when iterating through variable captures
- [x] Add tests for Go inline values
- [x] Remove old inline provider code and trait implementations
Release Notes:
- debugger: Generate inline values based on a language debugger.scm file
## Context
This PR improves the accuracy of our inline values for Rust/Python. It
does this by only adding inline value hints to the last valid use of a
variable and checking whether variables are valid within a given scope
or not.
We also added tests for Rust/Python inline values and inline values
refreshing when stepping in a debug session.
### Future tasks
1. Handle functions that have inner functions defined within them.
2. Add inline values to variables that were used in inner scopes but not
defined in them.
3. Move the inline value provider trait and impls to the language trait
(or somewhere else).
4. Use Semantic tokens as the first inline value provider and fall back
to tree sitter
5. add let some variable statement, for loops, and function inline value
hints to Rust.
6. Make writing tests more streamlined.
6.1 We should be able to write a test by only passing in variables,
language, source file, expected result, and stop position to a function.
7. Write a test that has coverage for selecting different stack frames.
co-authored-by: Remco Smits \<djsmits12@gmail.com\>
Release Notes:
- N/A
---------
Co-authored-by: Remco Smits <djsmits12@gmail.com>
