Commit Graph
17 Commits
Author SHA1 Message Date
David AlecrimandOleksiy Syvokon 526112f857 fuzzy: Fix crash with Unicode chars whose lowercase expands to multiple codepoints (#52989)
Self-Review Checklist:

- [x] I've reviewed my own diff for quality, security, and reliability
- [x] Unsafe blocks (if any) have justifying comments
- [x] The content is consistent with the [UI/UX
checklist](https://github.com/zed-industries/zed/blob/main/CONTRIBUTING.md#uiux-checklist)
- [x] Tests cover the new/changed behavior
- [x] Performance impact has been considered and is acceptable

Closes #52973

## Problem

The file picker crashes with `highlight index N is not a valid UTF-8
boundary` when file paths contain Unicode characters whose lowercase
form expands to multiple codepoints. Turkish `İ` (U+0130) is the trigger
here: Rust's `char::to_lowercase()` turns it into `i` + combining dot
above (two codepoints). That expansion breaks the fuzzy matcher in two
ways:

1. The `j_regular` index mapping mixes the expanded lowercase index
space with the original character index space, so highlight positions
land on invalid byte boundaries.
2. The scoring matrices are allocated with the expanded length but
indexed with the original length as stride, so rows alias each other and
corrupt stored values.

Users with Turkish locale filenames were hitting this on v0.229.0 and
v0.230.0 stable.

## Fix

I went with simple 1:1 case mapping: a `simple_lowercase` helper in
`char_bag.rs` that takes only the first codepoint from `to_lowercase()`
and drops any trailing combining characters. For `İ` this gives `i`,
which is what anyone would actually type in a search query. The same
function is used in the matcher, the char bag pre-filter, and both
query-lowercasing call sites (`paths.rs` and `strings.rs`).

This gets rid of the `extra_lowercase_chars` BTreeMap, the `j_regular`
adjustment, and the matrix sizing discrepancy. The matcher now works
with a flat character array where `lowercase_candidate_chars.len() ==
candidate_chars.len()`, so there's no expanded-vs-original index space
to get wrong.

I also fixed `CharBag::insert`, which used `to_ascii_lowercase()` and
silently ignored non-ASCII characters. A file like `aİbİcdef.txt`
wouldn't show up when searching `ai` because `İ` was never registered as
`i` in the bag. It now goes through `simple_lowercase` too.

The alternative was keeping full case folding and fixing the index
tracking with a `Vec<usize>` mapping expanded positions back to
originals. That would work but keeps the dual-index-space complexity
that caused these bugs, plus adds a per-candidate allocation for the
mapping vector.

## Prior art

fzf uses Go's `unicode.To(unicode.LowerCase, r)`, which is simple case
mapping -- always one rune in, one rune out. `İ` maps to `i`, no
expansion.

VS Code's `String.toLowerCase()` does produce the expanded form, but the
scorer compares UTF-16 code units independently and sidesteps the
problem in practice.

Neither tool maintains a mapping between expanded and original index
spaces.

## Trade-off

Searching for the combining dot above (U+0307) won't match `İ` in a path
anymore. Nobody types combining characters in a file picker, and fzf
doesn't support it either.

## Screenshot
<img width="1282" height="458" alt="Screenshot 2026-04-02 at 09 56 34"
src="https://github.com/user-attachments/assets/720d327a-4855-4d4d-989e-cbd1c0657f97"
/>


Release Notes:
- Fixed a crash and improved matching and highlighting in the file
picker for paths with non-ASCII
  characters (e.g., Turkish İ, ß, fi).

---------

Co-authored-by: Oleksiy Syvokon <oleksiy.syvokon@gmail.com>
2026-04-03 11:00:13 +00:00
bae91cee47 Represent relative paths using a dedicated, separator-agnostic type (#38744)
Closes https://github.com/zed-industries/zed/issues/38690
Closes #37353

### Background

On Windows, paths are normally separated by `\`, unlike mac and linux
where they are separated by `/`. When editing code in a project that
uses a different path style than your local system (e.g. remoting from
Windows to Linux, using WSL, and collaboration between windows and unix
users), the correct separator for a path may differ from the "native"
separator.

Previously, to work around this, Zed converted paths' separators in
numerous places. This was applied to both absolute and relative paths,
leading to incorrect conversions in some cases.

### Solution

Many code paths in Zed use paths that are *relative* to either a
worktree root or a git repository. This PR introduces a dedicated type
for these paths called `RelPath`, which stores the path in the same way
regardless of host platform, and offers `Path`-like manipulation APIs.
RelPath supports *displaying* the path using either separator, so that
we can display paths in a style that is determined at runtime based on
the current project.

The representation of absolute paths is left untouched, for now.
Absolute paths are different from relative paths because (except in
contexts where we know that the path refers to the local filesystem)
they should generally be treated as opaque strings. Currently we use a
mix of types for these paths (std::path::Path, String, SanitizedPath).

Release Notes:

- N/A

---------

Co-authored-by: Cole Miller <cole@zed.dev>
Co-authored-by: Piotr Osiewicz <24362066+osiewicz@users.noreply.github.com>
Co-authored-by: Peter Tripp <petertripp@gmail.com>
Co-authored-by: Smit Barmase <heysmitbarmase@gmail.com>
Co-authored-by: Lukas Wirth <me@lukaswirth.dev>
2025-09-24 18:57:33 -04:00
Nia e9df4994eb fuzzy: Fixup atomic ordering (#38468)
Hopefully partially addresses some crashes that can be triggered in this
code.

Release Notes:

- N/A
2025-09-19 00:45:59 +00:00
Smit Barmase b4b4375cc4 editor: Improve code completion filtering to provide fewer and more accurate suggestions (#32928)
Closes #32756

- Uses `filter_text` from LSP source to filter items in completion list.
This fixes noisy lists like on typing `await` in Rust, it would suggest
`await.or`, `await.and`, etc., which are bad suggestions. Fallbacks to
label.
- Add `penalize_length` flag to fuzzy matcher, which was the default
behavior across. Now, this flag is set to `false` just for code
completion fuzzy matching. This fixes the case where if the query is
`unreac` and the completion items are `unreachable` and
`unreachable!()`, the item with a shorter length would have a larger
score than the other one, which is not right in the case of
auto-complete context. Now these two items will have the same fuzzy
score, and LSP `sort_text` will take over in finalizing its ranking.
- Updated test to be more utility based rather than example based. This
will help to iterate/verify logic faster on what's going on.

Before/After:

await: 
<img width="600" alt="before-await"
src="https://github.com/user-attachments/assets/384138dd-a90d-4942-a430-6ae15df37268"
/>
<img width="600" alt="after-await"
src="https://github.com/user-attachments/assets/d05a10fa-bae5-49bd-9fe7-9933ff215f29"
/>

iter:
<img width="600" alt="before-iter"
src="https://github.com/user-attachments/assets/6e57ffe9-007d-4b17-9cc2-d48fc0176c8e"
/>
<img width="600" alt="after-iter"
src="https://github.com/user-attachments/assets/a8577a9f-dcc8-4fd6-9ba0-b7590584ec31"
/>

opt:
<img width="600" alt="opt-before"
src="https://github.com/user-attachments/assets/d45b6c52-c9ee-4bf3-8552-d5e3fdbecbff"
/>
<img width="600" alt="opt-after"
src="https://github.com/user-attachments/assets/daac11a8-9699-48f8-b441-19fe9803848d"
/>

Release Notes:

- Improved code completion filtering to provide fewer and more accurate
suggestions.
2025-06-18 16:01:28 +05:30
Piotr Osiewicz d6520954b6 chore: Bump Rust edition to 2024 (#27800)
Follow-up to https://github.com/zed-industries/zed/pull/27791

Release Notes:

- N/A
2025-03-31 20:55:27 +02:00
张小白 7c8909225f workspace: Add trailing / to directories on completion when using OpenPathPrompt (#25430)
Closes #25045

With the setting `"use_system_path_prompts": false`, previously, if the
completion target was a directory, no separator would be added after it,
requiring us to manually append a `/` or `\`. Now, if the completion
target is a directory, a `/` or `\` will be automatically added. On
Windows, both `/` and `\` are considered valid path separators.



https://github.com/user-attachments/assets/0594ce27-9693-4a49-ae0e-3ed29f62526a



Release Notes:

- N/A
2025-03-04 14:01:08 +08:00
Piotr OsiewiczandAnthony Eid 51a82062b6 Rust 1.85 (#25272)
Closes #ISSUE

Release Notes:

- N/A *or* Added/Fixed/Improved ...

---------

Co-authored-by: Anthony Eid <hello@anthonyeid.me>
2025-02-28 18:33:35 +01:00
Michael Sloan a6a909cd0d Fuzzy match performance improvements redo (#22561)
Release Notes:

- N/A
2025-01-02 05:31:06 +00:00
Peter Tripp 71325da8d3 Fix for extension search crash (revert #22524; revert #22525) (#22543)
Revert "Improve fuzzy match performance and fix corner case that omits
results (#22524)"
This reverts commit e4d55e7e05.

Revert "Check cancel in multithreaded fuzzy matching (#22525)"
This reverts commit f4dd47de2a.

Fuzzy matching implemented in:
- https://github.com/zed-industries/zed/pull/22524
- https://github.com/zed-industries/zed/pull/22525

Caused a panic in the extension store search:
- Closes: https://github.com/zed-industries/zed/issues/22541

cc: @mgsloan 

Release Notes:

- N/A
2025-01-01 17:04:37 +00:00
Michael Sloan f4dd47de2a Check cancel in multithreaded fuzzy matching (#22525)
For both the strings and paths multithreaded matching it would still
aggregate the response even though it is unneeded. It now checks cancel.

In the paths matcher, cancel is now checked within the loop, since it
was calling `match_candidates` even though no further results would be
computed.

Release Notes:

- N/A
2024-12-31 22:37:41 +00:00
Michael Sloan e4d55e7e05 Improve fuzzy match performance and fix corner case that omits results (#22524)
* Removes `max_results` from the matcher interface as this is better
dealt with in consumers once all results are known. The current
implementation was quite inefficient as it was using binary search to
find insertion points and then doing an insert which copies the entire
suffix each time.

* There was a corner case where if the binary search found a match
candidate with the same score, it was dropped. Now fixed.

* Uses of `util::extend_sorted` when merging results from worker threads
also repeatedly uses binary search and insertion which copies the entire
suffix. A followup will remove that and its usage.

* Adds `util::truncate_to_bottom_n_sorted_by` which uses quickselect +
sort to efficiently get a sorted count limited result.

* Improves interface of Matcher::match_candidates by providing the match
positions to the build function. This allows for removal of the `Match`
trait. It also fixes a bug where the Match's own Ord wasn't being used,
which seems relevant to PathMatch for cases where scores are the same.

Release Notes:

- N/A
2024-12-31 20:56:23 +00:00
Michael Sloan 3e2fe3e1c9 Fix fuzzy string match invariant check (#22032)
Version in #21983 only handled out of range issues rather than utf-8
boundary issues (thanks to @s3bba for pointing this out)

Release Notes:

- N/A
2024-12-15 01:15:22 -07:00
Michael Sloan 051bdf5bb1 Improve StringMatchCandidate::new interface (#22011)
Release Notes:

- N/A
2024-12-14 13:35:36 -07:00
Michael Sloan 1823770243 Log invariant violations in fuzzy string match iterator (#21983)
Seeing frequent inscrutable panics here

Release Notes:

- N/A
2024-12-13 11:16:30 -07:00
Marshall Bowers ceab9f0906 Fix Clippy warnings in fuzzy crate (#8701)
This PR fixes a number of Clippy warnings in the `fuzzy` crate.

Release Notes:

- N/A
2024-03-02 01:02:34 -05:00
Max BrunsfeldandMikayla 8e704c8a47 Remove 2 suffix for lsp, language, fuzzy
Co-authored-by: Mikayla <mikayla@zed.dev>
2024-01-03 11:58:02 -08:00
Kay Simmons 9e5b4dce01 Add recent project picker 2022-12-14 15:59:50 -08:00