21 days ago
Poland (city not specified)Senior
Responsibilities
- Own prioritized competitive benchmarking initiatives across search, vector and hybrid retrieval, analytical, and observability workloads.
- Build and maintain automated benchmark harnesses, datasets, query sets, and realistic load profiles covering concurrent ingestion and querying.
- Design fair, documented comparisons covering versions, configurations, tuning, resource budgets, and functional differences.
- Measure latency distributions, throughput, resource consumption, cost efficiency, and retrieval quality at comparable relevance targets.
- Adapt tools such as Elasticsearch Rally, ClickBench, and vector search benchmark frameworks.
- Investigate performance differences using profiling, system metrics, and controlled experiments, and validate engineering improvements.
- Evaluate self-managed and managed cloud deployments, including serverless offerings, scaling behavior, and service constraints.
- Publish reproducible benchmark artifacts and technical reports with engineering and content partners.
- Translate benchmark findings into actionable product improvements with engineering and product management.
Requirements
- Solid software engineering skills and experience building maintainable automation and analysis tools in Python or another relevant programming language.
- Hands-on experience evaluating and diagnosing performance in distributed systems, databases, or search infrastructure.
- Depth in search, vector retrieval, analytical databases, or observability systems, with the ability to learn unfamiliar systems.
- Experience designing representative datasets and workloads involving query patterns, concurrency, and ingestion behavior.
- Sound experimental judgment involving repeated measurements, variability analysis, warm-up and cache effects, and measurement-noise assessment.
- Experience building reproducible test environments and documenting configurations, procedures, and results.
- Clear technical writing and communication skills for explaining complex findings and limitations.
- Strong collaboration skills and the ability to turn performance evidence into engineering recommendations.
- Preferred experience includes Elasticsearch, Lucene, Rally, ClickBench, vector search benchmarking, retrieval-quality measurement, cloud performance and cost analysis, autoscaling, serverless systems, public technical writing, open-source contributions, or published benchmark studies.
Benefits
- Base salary with no variable compensation component.
- Health coverage for employees and families in many locations.
- Flexible locations and schedules for many roles at the distributed company.
- Generous vacation allowance.
- Matching donations up to $2000 or local currency equivalent.
- Up to 40 hours annually for volunteer projects.
- At least 16 weeks of parental leave.
About Elastic
Elastic builds the Elasticsearch-based platform for enterprise search, observability (logging, metrics, APM), and security (SIEM/endpoint). It sells subscriptions and Elastic Cloud managed services with usage-based pricing, combining open source and proprietary features through Elasticsearch and Kibana. Founded in 2012 and headquartered in San Francisco, Elastic is publicly traded on the NYSE and is used by more than half of the Fortune 500.