Search Library for Go

Portable lexical search for Go, from an embedded index to a search service.
Features · Quick Start · Drivers · Installation · Performance · Examples · API
Features
| Feature | What you get | Example or API |
|---|---|---|
| Require every analyzed term, with explicit searchable fields. | Typed search example | |
| Combine Boolean logic, exact values, integer ranges, and array membership. | Filter example | |
| Order by up to two declared fields, followed by the document ID. | Sort | |
| Use bounded offsets and limits with explicit total-count precision. | Query | |
| Count matching documents before pagination, deduplicating repeated array terms. | Facet example | |
| Render untrusted text segments with explicit match markers. | Highlight example | |
| Retain input positions and honest per-item or aggregate outcomes. | Bulk example | |
| Wait for accepted writes to become searchable and retain recovery evidence. | Receipt and Outcome | |
| Use typed Go values while keeping document IDs separate from JSON bodies. | Typed search example | |
| Provision deliberately and detect schema drift without runtime migration. | Definition | |
| Observe classified events with panic isolation and optional bounded async delivery. | Observer · AsyncObserver | |
| Script results and inspect recorded calls without running a search engine. | Fake backend example |
Quick Start
Go 1.25+, with no external service required.
git clone https://github.com/goforj/search.git
cd search
go run ./examples/quickstart// Command quickstart demonstrates embedded indexing and search.
package main
import (
"context"
"fmt"
"github.com/goforj/search"
"github.com/goforj/search/driver/blevesearch"
)
// Product models a catalog item; its document ID is supplied separately to Upsert.
type Product struct {
Title string `json:"title"`
Description string `json:"description"`
Category string `json:"category"`
PriceCents int64 `json:"price_cents"`
InStock bool `json:"in_stock"`
}
// main demonstrates the complete embedded indexing and typed-query lifecycle.
func main() {
ctx := context.Background()
// Use an in-memory index so this example needs no files or search service.
driver, err := blevesearch.Open(ctx, blevesearch.Config{Memory: true})
if err != nil {
panic(err)
}
defer driver.Close()
// Search descriptive text and expose catalog fields for filtering and sorting.
definition := search.Definition{
Name: "products",
Fields: []search.Field{
{Name: "title", Type: search.TextField, Searchable: true},
{Name: "description", Type: search.TextField, Searchable: true},
{Name: "category", Type: search.KeywordField, Filterable: true},
{Name: "price_cents", Type: search.IntegerField, Required: true, Filterable: true, Sortable: true},
{Name: "in_stock", Type: search.BooleanField, Filterable: true},
},
}
// Provision the schema explicitly before creating an application handle.
if err := driver.CreateIndex(ctx, definition); err != nil {
panic(err)
}
index, err := search.NewIndex(driver, definition)
if err != nil {
panic(err)
}
receipt, err := search.Upsert(ctx, index, "mug-1", Product{
Title: "Blue mug",
Description: "Handcrafted ceramic mug with a matte glaze, 350 ml.",
Category: "kitchen",
PriceCents: 2495,
InStock: true,
})
if err != nil {
panic(err)
}
// A receipt acknowledges the write; Wait establishes search visibility.
outcome, err := index.Wait(ctx, receipt)
if err != nil {
panic(err)
}
if outcome.State != search.OutcomeSucceeded {
panic(outcome.Error)
}
// Require both search terms and decode hits directly into the application type.
page, err := search.SearchAs[Product](ctx, index, search.Query{Text: "blue mug"})
if err != nil {
panic(err)
}
hit := page.Hits[0]
fmt.Println(hit.ID, hit.Value.Title)
// mug-1 Blue mug
fmt.Printf("%s | $%.2f | in stock: %t\n", hit.Value.Category, float64(hit.Value.PriceCents)/100, hit.Value.InStock)
// kitchen | $24.95 | in stock: true
}Drivers
Choose an embedded index, an existing SQL pool, or a shared search service.
All listed engines pass the same literal all-terms, typed-filter, stable-sort, pagination, facet, highlight, replacement, receipt, bulk, cancellation, and concurrent-call fixtures. See executed driver evidence for exact scenario names and the source fingerprint.
Driver behavior and limits
Meilisearch
Meilisearch facets support at most 4096 distinct terms per requested field after filtering. Higher cardinality returns ErrUnsupported; it does not return an incorrectly chosen top bucket. The driver collects a bounded complete distribution because the engine does not preserve portable lexical ties at its native count cutoff.
Meilisearch accepts at most ten backend words. Queries in scripts with finer native segmentation use a conservative rune bound to prevent silent truncation. Its validated server search cutoff exceeds the enforced 30-second maximum HTTP deadline, so the client fails before the engine can return a silently timed-out response. As with schema validation, this assumes administrators do not change settings behind a running client.
Elasticsearch and OpenSearch
Elasticsearch and OpenSearch each have an independent driver and tested version matrix. Their writes wait for refresh and active shard copies before reporting visibility. Native bulk responses retain per-item failures even when HTTP succeeds. Queries reject timeouts and partial shard responses. Multi-shard facet counts retain the engine's estimated precision when native error bounds are nonzero. Keyword values are limited to 16382 UTF-8 bytes because their ordered encoding must fit Lucene's term limit. Provisioning defaults to one primary shard and no replicas; configure replicas explicitly for a production cluster.
PostgreSQL FTS and SQLite FTS5
PostgreSQL FTS and SQLite FTS5 accept an application-owned *sql.DB. The application selects and registers its SQL client, configures the pool, and closes it. The search driver creates dedicated tables, native full-text projections, and schema ownership metadata. Writes and their projections commit in one transaction. Queries compute hits, totals, and facets from one database snapshot. PostgreSQL uses the simple text-search configuration and byte-order collation. SQLite requires an FTS5-enabled provider. Driver Close preserves the injected pool.
Redis Search
Redis Search targets standalone Redis with its native Search module. It uses native literal matching and typed filtering, then collects at most 4096 matching documents to provide deterministic multiple sorts and exact array facets. More matches return ErrUnsupported. Mutations execute as one Lua command, and interrupted or failed scripts report uncertainty rather than implying rollback. Native query timeout exceeds the enforced maximum client deadline to avoid accepting partial timed-out results. Cluster routing is not supported by this driver.
Bleve
Bleve owns a single index path in one process. Its ownership lock refuses a second writer. Use a shared service when querying and indexing from separate processes. Close concrete drivers at the application lifecycle boundary.
Installation
The root package and service drivers require Go 1.24+ and have no third-party dependencies. The optional Bleve module requires Go 1.25+.
Install the root package and the drivers you need:
go get github.com/goforj/search
# Bleve (embedded, Go 1.25+)
go get github.com/goforj/search/driver/blevesearch
# Meilisearch
go get github.com/goforj/search/driver/meilisearch
# Typesense
go get github.com/goforj/search/driver/typesensesearch
# OpenSearch
go get github.com/goforj/search/driver/opensearch
# Elasticsearch
go get github.com/goforj/search/driver/elasticsearch
# PostgreSQL FTS
go get github.com/goforj/search/driver/postgressearch
# SQLite FTS5
go get github.com/goforj/search/driver/sqlitesearch
# Redis Search
go get github.com/goforj/search/driver/redissearchInstall the drivers you need. All drivers except Bleve are packages in the root module and are included when you install search; their commands above are provided for convenient copying. Bleve is a separate optional module. See module ownership and release order.
See search behavior and recovery for write visibility, query semantics, and receipt lifetime.
Performance
The same 1,000,000-document synthetic catalog and query sequence across all eight drivers, using the public search.Index API. Latency charts show client-observed p95 from 200 searches after 30 warm-up requests per workload. Latency axes are logarithmic; indexing throughput is linear.
Literal search
Filter and sort
Text and facets
Indexing throughput
| Driver | Load (s) | Literal p50 / p95 (ms) | Filter + sort p50 / p95 (ms) | Facets p50 / p95 (ms) |
|---|---|---|---|---|
| 45.1 | 7.984 / 14.845 | 0.868 / 2.105 | 0.874 / 1.331 | |
| 274.9 | 1.350 / 1.829 | 1.896 / 2.494 | 1.941 / 2.710 | |
| 67.5 | 5.973 / 6.675 | 1.387 / 1.573 | 1.770 / 3.031 | |
| 116.9 | 2.161 / 3.526 | 2.076 / 3.299 | 2.445 / 4.101 | |
| 110.4 | 2.179 / 5.343 | 1.886 / 4.407 | 1.899 / 5.216 | |
| 697.1 | 4.587 / 6.903 | 2.180 / 3.798 | 86.901 / 97.977 | |
| 204.3 | 17.569 / 22.071 | 0.686 / 5.012 | 922.885 / 1496.732 | |
| 30.1 | 6.256 / 7.844 | 5.970 / 7.508 | 8.029 / 10.369 |
SQL measurements include generated secondary indexes; broad-term facets remain comparatively expensive. The SQL benchmark audit records the original bottlenecks and subsequent improvements.
Measured on Apple M4 Max, 16 CPU cores, 48 GiB RAM; OrbStack 16 CPUs, 19.57 GiB reported RAM. Embedded engines run on the host; service engines run in Docker. These are warmed, single-client, selective searches returning at most 1,024 matches and 20 hits, without concurrent writes. Indexing includes document generation, validation, and waiting for visibility in batches of 1,000.
See methodology and reproduction for workload details, pinned versions, persistence settings, and limitations. The measurement snapshot contains corpus fingerprints, environment metadata, count precision, and every latency sample. Run make bench-drivers to measure or make bench-render to redraw the recorded results.
Testing
Use searchfake to queue search results and inspect recorded calls in application tests. It implements search.Backend without a running search engine. The fake backend example shows the complete workflow.
Driver authors can reuse searchtest to verify the portable behavior contract. Integration tests run against real Bleve indexes and SQLite FTS5 databases, plus Testcontainers with pinned Meilisearch, Typesense, OpenSearch, Elasticsearch, PostgreSQL, and Redis images. Each service container starts once per run; tests use separate index namespaces. OpenSearch and Elasticsearch run with authentication enabled and execute the shared contract with one and two primary shards.
See the executed driver evidence, unit test inventory, and integration test inventory for the scenarios that actually ran.
Run make bench for allocation-aware document validation benchmarks and embedded search over 1,000 documents.
Runnable examples
Run these examples from the repository root with Go 1.25+. Each program includes expected output, verified by automated tests.
| Example | Run |
|---|---|
| Inspect bulk visibility | go run ./examples/bulk |
| Count facets before pagination | go run ./examples/facets |
| Test an application service | go run ./examples/fake |
| Filter and sort results | go run ./examples/filters |
| Escape structured highlights | go run ./examples/highlights |
| Index and search a document | go run ./examples/quickstart |
| Search typed documents | go run ./examples/search |
| Construct service drivers | go run ./examples/services |
Development
make test
make test-race
make vet
make test-integration
make test-coverage
make generate
make check-generatedAPI descriptions come from GoDoc. Runnable examples and their expected output live in examples/<name>/main.go. A shared test runner executes those programs. Edit those sources and run make generate; generation refreshes the quick start, example index, feature badges, and API index. CI checks that regeneration produces no diff.
See development and release notes for focused driver tests, module boundaries, and publishing. Report problems or request features in GitHub issues.
API Index
| Group | Functions |
|---|---|
| Construction | Definition · NewIndex |
| Facets | TermFacet |
| Filters | And · Equal · Integer · Keyword |
| Highlights | Highlight |
| Mutations | DeleteMutation · Index.Bulk · Upsert |
| Queries | Index.Search · Query · SearchAs |
| Sorting | Sort |
| Testing | searchfake.Backend.Calls · searchfake.Backend.QueueSearch · searchfake.New |
| Visibility | Index.Wait · Outcome |
Find complete programs and setup in Runnable examples.
Construction
Definition · NewIndex
Index and search a document.
ctx := context.Background()
// Use an in-memory index so this example needs no files or search service.
driver, err := blevesearch.Open(ctx, blevesearch.Config{Memory: true})
if err != nil {
panic(err)
}
defer driver.Close()
// Search descriptive text and expose catalog fields for filtering and sorting.
definition := search.Definition{
Name: "products",
Fields: []search.Field{
{Name: "title", Type: search.TextField, Searchable: true},
{Name: "description", Type: search.TextField, Searchable: true},
{Name: "category", Type: search.KeywordField, Filterable: true},
{Name: "price_cents", Type: search.IntegerField, Required: true, Filterable: true, Sortable: true},
{Name: "in_stock", Type: search.BooleanField, Filterable: true},
},
}
// Provision the schema explicitly before creating an application handle.
if err := driver.CreateIndex(ctx, definition); err != nil {
panic(err)
}
index, err := search.NewIndex(driver, definition)
if err != nil {
panic(err)
}
receipt, err := search.Upsert(ctx, index, "mug-1", Product{
Title: "Blue mug",
Description: "Handcrafted ceramic mug with a matte glaze, 350 ml.",
Category: "kitchen",
PriceCents: 2495,
InStock: true,
})
if err != nil {
panic(err)
}
// A receipt acknowledges the write; Wait establishes search visibility.
outcome, err := index.Wait(ctx, receipt)
if err != nil {
panic(err)
}
if outcome.State != search.OutcomeSucceeded {
panic(outcome.Error)
}
// Require both search terms and decode hits directly into the application type.
page, err := search.SearchAs[Product](ctx, index, search.Query{Text: "blue mug"})
if err != nil {
panic(err)
}
hit := page.Hits[0]
fmt.Println(hit.ID, hit.Value.Title)
// mug-1 Blue mug
fmt.Printf("%s | $%.2f | in stock: %t\n", hit.Value.Category, float64(hit.Value.PriceCents)/100, hit.Value.InStock)
// kitchen | $24.95 | in stock: trueFacets
TermFacet
Count facets before pagination.
result, err := index.Search(ctx, search.Query{Limit: 1, Facets: []search.TermFacet{{Name: "categories", Field: "category"}}})
if err != nil {
panic(err)
}
for _, bucket := range result.Facets[0].Buckets {
value, _ := bucket.Value.Keyword()
fmt.Println(value, bucket.Count)
// kitchen 2
// books 1
}Filters
And · Equal · Integer · Keyword
Filter and sort results.
category, err := search.Keyword("kitchen")
if err != nil {
panic(err)
}
maximum, err := search.Integer(1500)
if err != nil {
panic(err)
}
result, err := index.Search(ctx, search.Query{
Filter: search.And(search.Equal("category", category), search.LessThan("price", maximum)),
Sort: []search.Sort{{Field: "price", Direction: search.Ascending}},
Limit: 1,
})
if err != nil {
panic(err)
}
fmt.Println(result.Hits[0].Document.ID)
// two
fmt.Println(result.Total.Value)
// 2Highlights
Highlight
Escape structured highlights.
result, err := index.Search(ctx, search.Query{Text: "blue", Highlight: []search.Highlight{{Field: "title"}}})
if err != nil {
panic(err)
}
for _, segment := range result.Hits[0].Highlights["title"][0].Segments {
text := html.EscapeString(segment.Text)
if segment.Matched {
fmt.Print("<mark>" + text + "</mark>")
} else {
fmt.Print(text)
}
}
fmt.Println()
// <mark>Blue</mark> mugMutations
DeleteMutation · Index.Bulk
Inspect bulk visibility.
first, err := search.DeleteMutation("one")
if err != nil {
panic(err)
}
second, err := search.DeleteMutation("two")
if err != nil {
panic(err)
}
batch, err := index.Bulk(ctx, []search.Mutation{first, second})
if err != nil {
panic(err)
}
outcome, err := index.Wait(ctx, batch.Receipt)
if err != nil {
panic(err)
}
for _, item := range outcome.Bulk.Items {
fmt.Println(item.ID, item.Status)
// one succeeded
// two succeeded
}Upsert
Index and search a document.
ctx := context.Background()
// Use an in-memory index so this example needs no files or search service.
driver, err := blevesearch.Open(ctx, blevesearch.Config{Memory: true})
if err != nil {
panic(err)
}
defer driver.Close()
// Search descriptive text and expose catalog fields for filtering and sorting.
definition := search.Definition{
Name: "products",
Fields: []search.Field{
{Name: "title", Type: search.TextField, Searchable: true},
{Name: "description", Type: search.TextField, Searchable: true},
{Name: "category", Type: search.KeywordField, Filterable: true},
{Name: "price_cents", Type: search.IntegerField, Required: true, Filterable: true, Sortable: true},
{Name: "in_stock", Type: search.BooleanField, Filterable: true},
},
}
// Provision the schema explicitly before creating an application handle.
if err := driver.CreateIndex(ctx, definition); err != nil {
panic(err)
}
index, err := search.NewIndex(driver, definition)
if err != nil {
panic(err)
}
receipt, err := search.Upsert(ctx, index, "mug-1", Product{
Title: "Blue mug",
Description: "Handcrafted ceramic mug with a matte glaze, 350 ml.",
Category: "kitchen",
PriceCents: 2495,
InStock: true,
})
if err != nil {
panic(err)
}
// A receipt acknowledges the write; Wait establishes search visibility.
outcome, err := index.Wait(ctx, receipt)
if err != nil {
panic(err)
}
if outcome.State != search.OutcomeSucceeded {
panic(outcome.Error)
}
// Require both search terms and decode hits directly into the application type.
page, err := search.SearchAs[Product](ctx, index, search.Query{Text: "blue mug"})
if err != nil {
panic(err)
}
hit := page.Hits[0]
fmt.Println(hit.ID, hit.Value.Title)
// mug-1 Blue mug
fmt.Printf("%s | $%.2f | in stock: %t\n", hit.Value.Category, float64(hit.Value.PriceCents)/100, hit.Value.InStock)
// kitchen | $24.95 | in stock: trueQueries
Query · SearchAs
Search typed documents.
page, err := search.SearchAs[struct {
Title string `json:"title"`
}](ctx, index, search.Query{Text: "blue mug"})
if err != nil {
panic(err)
}
fmt.Println(page.Hits[0].Value.Title)
// Blue mug
fmt.Println(page.Total.Value)
// 1Index.Search
Filter and sort results.
category, err := search.Keyword("kitchen")
if err != nil {
panic(err)
}
maximum, err := search.Integer(1500)
if err != nil {
panic(err)
}
result, err := index.Search(ctx, search.Query{
Filter: search.And(search.Equal("category", category), search.LessThan("price", maximum)),
Sort: []search.Sort{{Field: "price", Direction: search.Ascending}},
Limit: 1,
})
if err != nil {
panic(err)
}
fmt.Println(result.Hits[0].Document.ID)
// two
fmt.Println(result.Total.Value)
// 2Sorting
Sort
Filter and sort results.
category, err := search.Keyword("kitchen")
if err != nil {
panic(err)
}
maximum, err := search.Integer(1500)
if err != nil {
panic(err)
}
result, err := index.Search(ctx, search.Query{
Filter: search.And(search.Equal("category", category), search.LessThan("price", maximum)),
Sort: []search.Sort{{Field: "price", Direction: search.Ascending}},
Limit: 1,
})
if err != nil {
panic(err)
}
fmt.Println(result.Hits[0].Document.ID)
// two
fmt.Println(result.Total.Value)
// 2Testing
searchfake.Backend.Calls · searchfake.Backend.QueueSearch · searchfake.New
Test an application service.
backend := searchfake.New()
backend.QueueSearch(search.Result{Total: search.Total{Value: 3, Relation: search.TotalExact}}, nil)
index, err := search.NewIndex(backend, search.Definition{
Name: "products",
Fields: []search.Field{{Name: "title", Type: search.TextField, Searchable: true}},
})
if err != nil {
panic(err)
}
result, err := index.Search(context.Background(), search.Query{Text: "mug"})
if err != nil {
panic(err)
}
fmt.Println(result.Total.Value, backend.Calls()[0].Kind)
// 3 searchVisibility
Index.Wait · Outcome
Index and search a document.
ctx := context.Background()
// Use an in-memory index so this example needs no files or search service.
driver, err := blevesearch.Open(ctx, blevesearch.Config{Memory: true})
if err != nil {
panic(err)
}
defer driver.Close()
// Search descriptive text and expose catalog fields for filtering and sorting.
definition := search.Definition{
Name: "products",
Fields: []search.Field{
{Name: "title", Type: search.TextField, Searchable: true},
{Name: "description", Type: search.TextField, Searchable: true},
{Name: "category", Type: search.KeywordField, Filterable: true},
{Name: "price_cents", Type: search.IntegerField, Required: true, Filterable: true, Sortable: true},
{Name: "in_stock", Type: search.BooleanField, Filterable: true},
},
}
// Provision the schema explicitly before creating an application handle.
if err := driver.CreateIndex(ctx, definition); err != nil {
panic(err)
}
index, err := search.NewIndex(driver, definition)
if err != nil {
panic(err)
}
receipt, err := search.Upsert(ctx, index, "mug-1", Product{
Title: "Blue mug",
Description: "Handcrafted ceramic mug with a matte glaze, 350 ml.",
Category: "kitchen",
PriceCents: 2495,
InStock: true,
})
if err != nil {
panic(err)
}
// A receipt acknowledges the write; Wait establishes search visibility.
outcome, err := index.Wait(ctx, receipt)
if err != nil {
panic(err)
}
if outcome.State != search.OutcomeSucceeded {
panic(outcome.Error)
}
// Require both search terms and decode hits directly into the application type.
page, err := search.SearchAs[Product](ctx, index, search.Query{Text: "blue mug"})
if err != nil {
panic(err)
}
hit := page.Hits[0]
fmt.Println(hit.ID, hit.Value.Title)
// mug-1 Blue mug
fmt.Printf("%s | $%.2f | in stock: %t\n", hit.Value.Category, float64(hit.Value.PriceCents)/100, hit.Value.InStock)
// kitchen | $24.95 | in stock: true