Lead Data Engineer
HackerRank2 months ago
Bengaluru, IndiaStaff+
Responsibilities
- Own and evolve the data platform across StarRocks, Apache Hudi, Trino, Spark, and Apache Ranger.
- Build AI-optimized datasets supporting natural language querying and AI add-on features.
- Own customer-facing data features including exports, insights dashboards, interview analytics, and Custom Reports.
- Enable self-service pipelines for AI, analytics, and go-to-market teams.
- Implement data security, access controls, Apache Ranger policies, and confidence-scoring guardrails for AI outputs.
- Lead technical design reviews and define engineering standards for the data team.
- Partner with product managers and business stakeholders to identify and scope AI-enabled data use cases.
Requirements
- 6+ years of data engineering experience, including at least 2 years in a senior or lead capacity.
- Deep hands-on experience with OLAP databases such as StarRocks, ClickHouse, Druid, or similar.
- Strong experience with data lake technologies such as Apache Hudi, Iceberg, or Delta Lake.
- Proficiency with distributed query engines such as Trino or Presto and batch or streaming compute with Apache Spark.
- Understanding of data security, RBAC, and access-control tools such as Apache Ranger.
- Comfort working in hybrid AWS and open-source self-managed environments.
- Strong communication skills and the ability to explain technical decisions to non-technical stakeholders and independently drive cross-functional projects.
- Experience with AI/LLM-adjacent data work, confidence scoring, agentic pipelines, RAG architectures, or vector stores is preferred.
- Experience operationalizing agentic workflows, scaling data infrastructure at a SaaS or B2B product company, or building natural-language-querying interfaces and customer-facing data products is preferred.
Tech Stack
Categories
Data Engineering
About HackerRank
HackerRank builds a developer skills assessment and technical interview platform used by recruiters and hiring managers to evaluate engineers via coding tests, take-home challenges, and pair-programming sessions. It sells its software to enterprises on a subscription basis and maintains a large developer practice community; the privately held company was founded in 2012, is headquartered in Santa Clara, and counts organizations such as OpenAI, NVIDIA, and Amazon among its customers.