AI ERP with agents inside the systems you run
AI phone and voice agents for real calls
LLM.coPrivate, self-hosted LLM deployments
Legal AI infrastructure for firms
AI RFP discovery and response drafting
Automatic.coBusiness process automation
CyberAttack.aiThreat detection and defense
Secure AI virtual data rooms10 business days
To start an Elasticsearch engagement
Scoping through first sprint
100%
Senior engineers, US-based
No offshore handoff on production search infrastructure
Every sprint
Working software on a preview URL
Not a status deck between milestones
100%
Cluster and index config you own
In your own environment, from the first commit
How Elasticsearch actually works
The concepts that determine whether search results are actually good
Elasticsearch's default behavior gets a demo working in an afternoon and a production search experience wrong in ways that don't show up until real users start querying it. These are the decisions that matter.
Documents, indices, and shards
A document is a single JSON record; an index is a collection of documents, roughly analogous to a database table; a shard is a horizontal slice of an index that lets Elasticsearch distribute data and queries across nodes. Shard count is set at index creation and is expensive to change later, which makes it one of the first decisions worth getting right rather than accepting a default.
Mapping defines what a field actually means
Elasticsearch will guess a field's type if you let it — dynamic mapping — and the guess is frequently wrong for what you actually need, like treating a numeric ID as a searchable text field. Defining mapping explicitly before indexing real data is the single highest-leverage step most teams skip.
Analyzers decide what "matching" means
An analyzer tokenizes and normalizes text at index time — lowercasing, stemming, removing stop words — and the same analyzer choices apply at query time. Two fields indexed with different analyzers will silently fail to match the way you expect, which is one of the most common causes of "search isn't finding things it should."
Relevance scoring is tunable, not fixed
Elasticsearch's default scoring (BM25) ranks results by term frequency and field length, and it's a starting point, not a finished relevance model. Boosting fields, applying function scores, and testing against real queries is what turns "technically matches" into "the result a user actually wanted first."
Aggregations are where analytics lives
Beyond search, Elasticsearch's aggregation framework computes metrics, histograms, and terms counts across matched documents in the same query — the mechanism behind most Kibana dashboards. Aggregation performance is shaped heavily by mapping decisions made at index time, which is another reason mapping isn't an afterthought.
Vector search and hybrid queries
Dense vector fields and approximate nearest-neighbor search let Elasticsearch find semantically similar content, not just keyword matches — the mechanism behind most retrieval-augmented generation (RAG) pipelines. Hybrid search, combining a traditional BM25 query with a vector similarity query, is often a better real-world result than either alone.
Deployment decisions
Elastic Cloud, self-hosted, or OpenSearch — decided on your actual constraints
In 2021, Elastic changed Elasticsearch and Kibana's license away from fully open source, and AWS led a fork of the pre-license codebase into OpenSearch. Both are now real, separately maintained options, and the choice between them is a genuine architecture decision, not a licensing footnote.
Elastic Cloud is the fastest path to a managed cluster
Elastic's own managed service, with the newest features first and direct support from the company that builds the product. The tradeoff is being on Elastic's licensing terms and pricing model, which matters more for some organizations than others depending on how the software is being used.
Self-hosted Elasticsearch is a real operational commitment
Running your own cluster gives full control over version, configuration, and infrastructure placement, at the cost of owning cluster operations yourself — node sizing, shard rebalancing, upgrade cadence. This earns its cost at real scale or under specific data-residency requirements, and is overhead without a clear benefit below that.
OpenSearch is the fork to evaluate on its own merits
OpenSearch tracks close to Elasticsearch's API for most common use cases and ships under an open-source license, with AWS offering it as a managed service. The gap between the two projects has grown since the fork — feature parity isn't guaranteed on newer capabilities like the latest vector search improvements — so the decision should be based on which specific features your use case needs, not on license preference alone.
Migrating between them is possible but not free
Reindexing between an Elasticsearch cluster and an OpenSearch cluster is a real data-migration project, not a configuration flag, because the two have diverged enough that not every feature or query syntax carries over cleanly. This is worth factoring into the initial choice rather than treating it as a reversible decision.
Data residency and compliance can decide it outright
Some organizations have data-residency or compliance requirements that rule out a specific managed offering regardless of feature comparison. When that's the case, it's worth surfacing early — it removes an otherwise open decision from the table entirely.
What's involved
Elasticsearch engagement types
What moves the scope is data volume, query complexity, and whether the goal is greenfield search, a migration, or fixing an existing cluster's relevance and performance problems.
| Engagement | Commitment | Timeline | What's included |
|---|---|---|---|
| Search and cluster audit | Fixed scope | 1 – 3 weeks | Mapping and analyzer review, shard strategy assessment, and a prioritized list of what's actually hurting relevance or performance. |
| Full-text search build | Fixed scope | 4 – 10 weeks | Index design, mapping, analyzer configuration, and relevance tuning against real query patterns, integrated into your application. |
| Vector or hybrid search build | Fixed scope | 6 – 12 weeks | Dense vector indexing and approximate nearest-neighbor search, combined with traditional search for a RAG pipeline or semantic search feature. |
| Log and event pipeline | Fixed scope | 4 – 8 weeks | Ingestion, index lifecycle management, and dashboards sized to actual retention needs rather than indexing everything indefinitely. |
| Ongoing cluster operations | Ongoing retainer | Ongoing | Standing ownership of cluster health, reindexing, and version upgrades through the next scale threshold. |
Ranges assume US-based senior engineers and include relevance testing against real queries, which a cheaper quote often skips in favor of shipping the default scoring as-is. The audit exists as its own first step because relevance problems are usually a mapping decision made at index time, not something a query-time fix can fully undo.
Where it actually earns its complexity
When Elasticsearch is the right tool, and when it's overkill
Elasticsearch is a genuinely powerful, genuinely complex piece of infrastructure. It earns that complexity at real scale and real query sophistication, and it's a poor first choice for a problem a simpler tool already solves.
Full-text search across large or unstructured content
Product catalogs, documentation, support tickets, or any corpus where users search with natural language rather than exact filters — this is Elasticsearch's core use case, and where its relevance tuning earns its keep.
Log and event analytics at real volume
The ELK stack (Elasticsearch, Logstash, Kibana) remains a common foundation for centralized logging, especially self-hosted. Our observability page covers the broader managed-versus-self-hosted decision for this use case in more depth.
Semantic and hybrid search for AI applications
Vector search combined with traditional keyword matching is increasingly the retrieval layer behind RAG systems and AI-powered search features, where pure keyword matching misses paraphrased or conceptually related queries.
Where a simpler tool is the honest answer
Postgres full-text search handles a large share of "search a few thousand rows" use cases without adding a second data store and a second consistency model to maintain. A dedicated search-as-a-service product (Algolia, Typesense) often ships faster for a product-catalog search box that doesn't need Elasticsearch's aggregation and analytics depth.
Where the volume doesn't justify the operational cost
A cluster sized for millions of documents, run for a dataset of a few thousand, is complexity without a corresponding benefit. This is the same overprovisioning pattern that shows up across infrastructure generally, and it's worth naming plainly during scoping rather than building the bigger thing because it sounds more capable.
Scale and lifecycle
What actually breaks an Elasticsearch cluster at scale
None of the following shows up in a proof-of-concept with a few thousand documents. All of it shows up within the first year of real production volume.
Shard sizing gets harder to fix the longer it's ignored
Too many small shards wastes cluster overhead on coordination; too few large shards slows recovery and rebalancing after a node failure. Reindexing to fix shard count on a live, growing index is a real project, not a config change — which is why the initial sizing decision matters more than it seems to at first.
Index lifecycle management is the difference between a bounded bill and an unbounded one
Without an ILM policy moving old indices through hot, warm, and cold tiers — or deleting them outright — log and event data accumulates indefinitely, and storage cost grows with it whether anyone's querying data from eight months ago or not.
Mapping explosion is a quiet cluster killer
Dynamic mapping on unstructured or inconsistent input data can silently create thousands of unique fields over time, degrading cluster performance in a way that's hard to diagnose after the fact. A strict or explicit mapping catches this before it becomes a cluster-wide problem.
Reindexing is sometimes the only way to fix a mapping mistake
Mapping is largely immutable once data is indexed — changing a field's type means creating a new index with the corrected mapping and reindexing into it. Planning for this as a routine operational capability, not a last resort, makes future schema changes far less painful.
Cluster health monitoring needs the same discipline as any production system
Shard allocation failures, JVM heap pressure, and node-level resource exhaustion all have real warning signs before a cluster goes red. Alerting on those signals, not just on the search API's uptime, is what catches a developing problem before users notice degraded results.
Hiring
What to look for when hiring an Elasticsearch developer
Someone who can write a query in Kibana's dev tools is different from someone who's designed a mapping strategy that held up as a dataset grew from thousands to millions of documents.
Mapping and analyzer design experience
Ask them to walk through a mapping decision they made and why — not just whether they've used dynamic mapping. This is where most relevance and performance problems actually originate.
Relevance tuning against real queries, not just syntax fluency
Knowing Query DSL syntax is table stakes. Knowing how to test and tune BM25 boosting and function scores against actual user queries — and iterate when the first pass ranks the wrong result first — is the harder, more valuable skill.
Cluster operations experience, if you're self-hosting
Shard rebalancing, node sizing, and upgrade planning are a different skill set from application-level query writing. If self-hosting is the plan, confirm the candidate has actually operated a cluster, not just queried one.
Awareness of the Elasticsearch/OpenSearch split
A candidate who can explain the licensing and feature divergence between the two, and when each is the better fit, is showing real current knowledge of the ecosystem rather than knowledge frozen at whatever point they first learned the tool.
Comfort recommending a simpler tool
The engineers worth hiring will tell you when Postgres full-text search or a search-as-a-service product actually fits your use case better than standing up an Elasticsearch cluster, instead of defaulting to the tool they know best.
Related
Related services
What Elasticsearch work usually connects to.
Questions
Common questions about Elasticsearch development
What teams ask before a first call.
A search and cluster audit is the smallest engagement here, and a full-text or vector search build is a larger one. An ongoing operations retainer is priced to the environment rather than sold as a fixed headcount.
What moves the number most is data volume, query complexity, and whether relevance tuning against real queries is part of the scope — a search feature that technically returns results and one that returns the right result first are very different amounts of work.
Designs the index mapping and analyzer configuration that determines what "matching" and "relevant" actually mean for your data, tunes scoring against real queries, and plans shard strategy and index lifecycle management so the cluster stays healthy as data grows.
A good Elasticsearch developer also knows when not to reach for it — recommending Postgres full-text search or a search-as-a-service product when the use case doesn't need Elasticsearch's scale or aggregation depth.
It depends on your specific feature needs and constraints, not on licensing preference alone. Elastic Cloud gets Elastic's newest features first, including on the vector search side, where the gap between the two projects has grown since OpenSearch's 2021 fork. OpenSearch ships under a fully open-source license with AWS offering it as a managed service, and it covers most common search and log-analytics use cases well.
Data residency or compliance requirements sometimes decide this outright by ruling one option out — worth surfacing early, since it removes what would otherwise be an open decision.
No — the correct spelling is "Elasticsearch," one word, lowercase s after the first letter. "ElasticSearch" with a capital S is a common but outdated spelling from the product's early years.
For a few thousand rows with straightforward search needs, Postgres's built-in full-text search often handles it without adding a second data store and a second consistency model to keep in sync. Elasticsearch earns its complexity at real scale, with more sophisticated relevance tuning, faceted search, or aggregation-heavy analytics needs.
We'll assess your actual data volume and query patterns honestly rather than defaulting to the more capable tool because it's more interesting to build.
Yes — Elasticsearch's dense vector fields and approximate nearest-neighbor search support the semantic retrieval step most RAG pipelines depend on, and hybrid search combining vector similarity with traditional keyword matching often outperforms either approach alone.
This is one of the faster-growing reasons we're asked to build on Elasticsearch specifically, alongside its established use for full-text and log search.
The most common cause is an analyzer mismatch — the text was indexed with different tokenization or normalization settings than the query is using, so terms that look identical to a human don't match at the index level. Dynamic mapping guessing a field's type incorrectly is the second most common cause.
Both are diagnosable in a focused audit and are usually fixable without a full reindex, though a reindex is sometimes the honest answer if the mapping itself needs to change.
Index lifecycle management policies that move older indices through hot, warm, and cold storage tiers — or delete them outright once they're past their useful retention window — are the main lever. Without an ILM policy, log and event data accumulates indefinitely regardless of whether anyone still queries it.
This is worth setting up before volume grows, not after storage costs prompt the conversation.