9 days ago
Pune, IndiaSenior
Responsibilities
- Design, build, and maintain scalable batch and incremental Databricks pipelines using PySpark, Spark SQL, and Delta Lake.
- Ingest data from multiple source systems into bronze, silver, and gold medallion architecture layers.
- Implement canonical data models and ensure figures reconcile consistently across product surfaces.
- Build production pipelines for deduplication, record linkage, and entity resolution across disparate systems.
- Profile source data and implement automated data quality checks, expectations, monitoring, and reporting.
- Orchestrate and monitor jobs with Databricks Workflows, troubleshoot pipeline failures, and optimize Spark performance and cost.
- Apply Unity Catalog governance, multi-tenant data isolation, anonymization, and aggregation rules for pooled analytics.
- Partner with data scientists, architects, engineering, and product stakeholders to productionize models and expose curated data.
- Use AI tools to accelerate development, debugging, and documentation while validating generated code before production use.
Requirements
- 5+ years of experience in data engineering or analytics engineering building production data pipelines in a cloud environment.
- Hands-on experience with Databricks, PySpark, Spark SQL, Delta Lake, Databricks Workflows or Jobs, and Unity Catalog.
- Strong SQL and Python skills, solid dimensional or canonical data modeling experience, and experience with lakehouse or medallion architecture.
- Practical experience with deduplication, record linkage, entity resolution, and diagnosing and remediating data quality problems at scale.
- Experience implementing automated data quality checks, pipeline monitoring, and Spark performance and cost optimization.
- Experience with cloud platforms, preferably AWS, Git-based version control, and data-pipeline CI/CD.
- Strong communication skills with data scientists, architects, and product stakeholders, including the ability to flag data limitations early.
- Working proficiency with Claude, Databricks Assistant, or GitHub Copilot and judgment to validate AI-generated code.
- Preferred: Databricks Data Engineer Associate or Professional certification.
- Preferred: experience with Auto Loader, Structured Streaming, or Delta Live Tables.
- Preferred: experience building data for benchmarks, scoring models, or externally consumed analytics.
- Preferred: experience with anonymization, aggregation thresholds, or privacy-preserving analytics.
- Preferred: background in enterprise B2B SaaS or legal technology.
Benefits
- Health coverage for employees and immediate family members.
- Flexible paid time off and 10 company-paid holidays annually.
- Paid leave for birth parents, non-birth parents, and caregivers, plus surrogacy and adoption reimbursement.
- 100% employer-paid life and disability insurance.
- Voluntary hospital indemnity, critical illness, and accident coverage.
- Tax-advantaged Flexi and NPS accounts.
- One paid volunteer day annually.
Tech Stack
Categories
Data Engineering
About Onit
Onit builds enterprise legal operations software used by corporate legal departments and law firms to manage contracts, matters, e‑billing, legal holds, and related workflows. Its products are delivered as SaaS with implementation and integration services, emphasizing process automation and AI‑assisted review and approval. The company is privately held, headquartered in Atlanta, Georgia, and operates globally with additional offices worldwide.