Lead Data Engineer
NXP Semiconductors3 months ago
Bengaluru, IndiaStaff+
Responsibilities
- Design and develop scalable data pipelines and Lakehouse solutions on Databricks.
- Tune Databricks workloads for performance and cost through cluster sizing, query optimization, and Delta Lake table design.
- Establish standards for partitioning, clustering, workload isolation, governance, and platform performance.
- Monitor performance trends, identify high-cost queries, and resolve long-running loads with source teams and end users.
- Design and operationalize Unity Catalog for access control, lineage, and security.
- Build monitoring and self-healing automation using Databricks-native AI and agentic capabilities.
- Drive CI/CD workflows and DevOps best practices for Databricks asset deployment and release management.
- Lead design reviews, mentor Data Engineers, and shape the AI-native operations roadmap.
- Coordinate with the Databricks vendor on support cases, escalations, and release adoption.
Requirements
- Bachelor's or master's degree in Computer Science, Information Technology, or equivalent relevant experience.
- At least 6 years of data engineering experience, including at least 2 years of hands-on Databricks experience in enterprise settings.
- Deep understanding of Databricks Lakehouse architecture, Delta Lake, Unity Catalog, and workflow orchestration.
- Proven ability to tune Spark workloads for cost and performance at production scale.
- Advanced Python, PySpark, and SQL skills.
- Working knowledge of CI/CD practices and DevOps principles applied to data workloads.
- Experience with observability tooling for Databricks.
- Preferred experience with Databricks-native AI capabilities and agentic frameworks.
- Preferred familiarity with Databricks Serverless Compute and DBSQL performance tuning.
- Databricks Certified Professional certification and exposure to Infrastructure-as-Code are pluses.
- Strong performance-engineering, platform-ownership, communication, and cross-functional influence skills.
Tech Stack
Categories
Data Engineering
About NXP Semiconductors
NXP Semiconductors designs and supplies microcontrollers, application processors (i.MX), analog/RF, secure connectivity, and automotive chips for OEMs in automotive, industrial/IoT, mobile, and communications infrastructure markets. A public company (NASDAQ: NXPI) founded in 2006 and headquartered in Eindhoven, it sells silicon, software, and reference designs, and supports customers with BSPs and development tools. Notable lines include i.MX processors, S32 automotive platforms, NFC and secure elements, radar, and power management devices.