Convex Improved Search

Exact substring search for Convex with a bigram inverted index that handles CJK, full-width characters, and LIKE-style matching across any language.

Installation

npm install convex-improved-search

About Convex Improved Search

convex-improved-search provides a bigram inverted index stored in ordinary Convex tables, enabling exact substring search across any language including CJK scripts that Convex's built-in full-text search cannot handle. It applies NFKC plus lowercase folding so queries match across full-width, decomposed, and precomposed variants, and the result set is provably identical to a naive substring scan. The index stays transactional by syncing in the same mutation as your writes, and results are reactive from standard Convex queries.

Benefits

• Search Chinese, Japanese, Korean, and other non-Latin text that Convex's built-in full-text search cannot tokenize correctly
• Get provably exact substring match semantics verified against a naive-scan oracle, not approximations
• Keep the search index in sync within the same transaction as writes, eliminating any drift window
• Paginate search results reactively from standard Convex queries with multi-field filtering and highlight utilities included

Use cases

how to search CJK Chinese Japanese Korean text in Convex

convex-improved-search builds a bigram inverted index over NFKC-folded codepoints, so a two-character Chinese query like '停車' is a single index point-lookup. Install the component, declare a SearchIndex, and call searchDocs from a Convex query with any UTF-8 string. No tokenizer, dictionary, or language configuration is needed.

substring search LIKE query in Convex database

convex-improved-search provides exact LIKE '%...%' semantics for Convex tables. The bigram index accelerates substring lookups without approximation, and results are identical to scanning every document for containment. Call notesSearch.searchDocs with any query string and optional exact-match filters defined on the SearchIndex.

how to highlight search matches in Convex full text search results

convex-improved-search exports matchSegments(text, query) which splits original text into matched and unmatched segments using the same NFKC folding the index uses. It correctly maps folded matches back to original offsets, so a half-width query highlights its full-width occurrence. Use matchedFields from search results to decide which field to highlight.

paginated search results in Convex with cursor

convex-improved-search provides searchDocsPaginated which speaks Convex's standard paginationOpts and returns the standard page, isDone, continueCursor shape. For custom pagination, searchDocs accepts a cursor string and returns its own cursor and isDone flag. A non-null cursor with a short page means the scan budget ran out, not that the search is finished.

Frequently asked questions

Why can't Convex's built-in search handle CJK text?

Convex's built-in full-text search tokenizes on whitespace and punctuation, so a run of CJK text becomes a single token. Additionally, terms over 32 bytes, which is roughly 11 CJK characters, are silently dropped from the index entirely. convex-improved-search bypasses tokenization entirely by indexing overlapping bigrams of Unicode codepoints, making every two-character substring a direct index lookup.

Does convex-improved-search keep the index consistent with the source table?

Yes. The tableIndexer.sync method is called in the same Convex mutation as the write, so the index update is in the same transaction and can never drift. For automatic sync on every write, pair it with convex-helpers Triggers. For backfilling existing data, use @convex-dev/migrations with syncDoc, which runs the same mapper as live writes.

How does convex-improved-search handle large datasets?

For namespaces past roughly 100k entries, convex-improved-search provides ShardedSearchIndex which partitions the index by time using a period of month, quarter, or year derived from the sortKey. Each shard is an independent namespace. Searches can be limited to recent shards with shardsBetween to bound query cost, and empty periods cost nothing because finding the next shard requires only one index range.

Is convex-improved-search reactive? Will search results update live?

Yes. Because the bigram index lives in ordinary Convex tables, searches run from standard Convex query functions. Any change to the indexed data that triggers a sync will cause reactive query subscribers to re-run and receive updated results automatically, the same as any other Convex query.

How do I index multiple fields and know which field matched?

When creating a SearchIndex, pass a text object with named fields such as { title: note.title, body: note.body } in the tableIndexer mapper. Each search result includes a matchedFields array listing which fields contained the query. To restrict a search to specific fields, pass a fields array to the search call, and candidates matched only in other fields are dropped inside the component before document hydration.

Links

• npm package
• Source repository
• Homepage
• Live demo