Commit Graph
121 Commits
Author SHA1 Message Date
Kyle Caverly ff90cd07a3 Leverage embeddings query to collapse syntax nodes if not selected (#3067)
Reverts zed-industries/zed#3049
2023-10-03 12:02:47 +03:00
KCaverly 62710360eb move collapsed only matches outside item parent in embedding.scm 2023-10-03 11:56:45 +03:00
KCaverly 8eacf279f1 add scheme for full parseable files in semantic index 2023-09-29 14:35:02 -04:00
KCaverly 05413774b2 leverage embeddings len returned in construction matrix multiplication 2023-09-27 10:33:04 -04:00
KCaverly 7f6e16e3bd removed blas and increase batch size for vector search 2023-09-27 09:43:23 -04:00
KCaverly 34e51dff43 fixed bug limiting number of results returned 2023-09-26 10:29:55 -04:00
KCaverly 44b407f00d ensure desc sort and cleanup unused imports 2023-09-26 09:53:49 -04:00
KCaverly b6f623e39c implement new search strategy 2023-09-25 13:44:19 -04:00
KCaverly 79fe1669de cargo fmt 2023-09-22 09:46:06 -04:00
KCaverly 338899567b move embedding provider to ai crate 2023-09-22 09:33:59 -04:00
Kyle Caverly e05a437a29 add semantic index status, for non authenticated users (#3005)
Update project search semantic ui to accommodate for users who have not
set the OPENAI_API_KEY in their environment variables.

Release Notes:

- Expand Semantic Index status to include status for non authenticated
users
- Update Search UI to illustrate this status.
2023-09-21 14:18:58 -04:00
KCaverly 79cac9d44c move api authentication to embedding provider 2023-09-21 14:00:00 -04:00
KCaverly 18b12dc1a3 add semantic index status, for non authenticated users 2023-09-21 13:40:01 -04:00
KCaverly 57f49a6530 remove conditional action registration for activate semantic search mode 2023-09-20 12:38:29 -04:00
Kyle Caverly 8a004641ee Semantic index eval (#2988)
v0 of the Semantic Index evaluate test suite

Release Notes:

- Added eval.rs as an example to the semantic-index crates
- Generates test metrics for two small projects, as a starting point to
systematically evaluate retrieval quality
2023-09-19 19:17:06 -04:00
KCaverly 0bb039bfb7 formatting 2023-09-19 12:27:33 -04:00
KCaverly 354db120f4 remove release channel flags in semantic_index 2023-09-19 12:20:59 -04:00
KCaverly 71862a72a8 fix Cargo.lock for merge 2023-09-19 11:44:51 -04:00
KCaverly d8cd7a4520 enable include based filtering for search inside open and modified buffers 2023-09-15 15:16:20 -04:00
KCaverly 44249374dc catchup with main 2023-09-15 09:31:33 -04:00
Antonio Scandurra c5dcfc4d05 Refactor semantic searching of modified buffers 2023-09-15 12:12:20 +02:00
KCaverly 54487b74fe update searching in modified buffers to accomodate for excluded paths 2023-09-14 19:42:06 -04:00
KCaverly c549532a98 add initial search inside modified buffers 2023-09-14 14:58:34 -04:00
Antonio Scandurra 6aa128353c WIP 2023-09-14 17:42:30 +02:00
Antonio ScandurraandKyle Caverly c45cab6c0c Make path optional when parsing file
Co-Authored-By: Kyle Caverly <kyle@zed.dev>
2023-09-14 17:09:08 +02:00
KCaverly 7e66b11ac8 semantic index eval, indexing appropriately 2023-09-13 20:02:15 -04:00
KCaverly 1c8e2961bb swap
SystemTime for Instant throughout rate_limit_expiry tracking
2023-09-11 10:26:14 -04:00
KCaverlyandAntonio e97cc35258 move cx notify observe for rate_limit_expiry into ProjectState in the semantic index
Co-authored-by: Antonio <antonio@zed.dev>
2023-09-11 10:11:40 -04:00
KCaverly 421d1f3ea7 updated notify to accomodate for updated countdown 2023-09-08 16:53:16 -04:00
KCaverly c250f31f18 updated semantic_index reset status to leverage target reset system time as opposed to duration 2023-09-08 15:04:50 -04:00
KCaverly 570061adb0 initial outline for rate limiting status updates 2023-09-08 12:35:15 -04:00
KCaverly 9558cc9647 update semantic search to go to no results if search query is blank 2023-09-07 14:43:41 -04:00
Antonio ScandurraandKyle Caverly 680aee026a Implement SemanticIndex::status and use it in project search
Co-Authored-By: Kyle Caverly <kyle@zed.dev>
2023-09-07 19:39:30 +02:00
Antonio ScandurraandKyle Caverly f9456ee00c Eagerly index project on workspace creation if it was indexed before
Co-Authored-By: Kyle Caverly <kyle@zed.dev>
2023-09-07 18:51:55 +02:00
Antonio Scandurra 5be66a655f 💄 2023-09-07 15:25:23 +02:00
Antonio ScandurraandKyle Caverly b663906981 Keep dropping the documents table if it exists
This is because we renamed `documents` to `spans`.

Co-Authored-By: Kyle Caverly <kyle@zed.dev>
2023-09-07 15:15:16 +02:00
Antonio Scandurra 4dfb90fcc1 Merge remote-tracking branch 'origin/main' into semantic-search-watch-worktrees 2023-09-07 15:07:46 +02:00
Antonio ScandurraandKyle Caverly 804e5f75a5 Remove unneeded logging
Co-Authored-By: Kyle Caverly <kyle@zed.dev>
2023-09-07 15:07:21 +02:00
KCaverly 59f2b062cb update request timeout for open ai embeddings 2023-09-06 15:09:46 -04:00
KCaverly 15cb416550 update token_count for OpenAIEmbeddings to accomodate for truncation 2023-09-06 15:09:15 -04:00
Antonio Scandurra 230f838f30 Rename Document to Span 2023-09-06 17:03:08 +02:00
Antonio Scandurra df73a67c6f Ensure SemanticIndex::search waits for indexing to complete 2023-09-06 11:40:59 +02:00
Antonio Scandurra ac570b6c1f Clip ranges returned by SemanticIndex::search
The files may have changed since the last time they were parsed, so the
ranges returned by `SemanticIndex::search` may be out of bounds.
2023-09-06 09:41:51 +02:00
Antonio ScandurraandKyle Caverly d271e8f54f Re-index project when a worktree is registered
Co-Authored-By: Kyle Caverly <kyle@zed.dev>
2023-09-05 17:17:58 +02:00
Antonio ScandurraandKyle Caverly 275a65eca0 Simplify SemanticIndex::index_project
Co-Authored-By: Kyle Caverly <kyle@zed.dev>
2023-09-05 16:54:48 +02:00
Antonio ScandurraandKyle Caverly 0269f9bbd8 Retrieve embeddings based on pending files
Co-Authored-By: Kyle Caverly <kyle@zed.dev>
2023-09-05 16:16:12 +02:00
Antonio ScandurraandKyle Caverly 381234cb74 Move retrieval of embeddings from the db into reindex_changed_files
Co-Authored-By: Kyle Caverly <kyle@zed.dev>
2023-09-05 16:09:24 +02:00
Antonio Scandurra 8ef079b287 🎨 2023-09-05 15:52:36 +02:00
KCaverly 73c4a76c9e update pragma settings for improved database performance 2023-09-01 17:07:20 -04:00
KCaverlyandMax 4d3955bbf9 updated embeddings background delay to 5 minutes
Co-authored-by: Max <max@zed.dev>
2023-09-01 13:04:09 -04:00