Lead Data Engineer - ETL Java AI
JPMorgan Chase21 days ago
Responsibilities
- Lead architecture and hands-on delivery of large-scale ETL/ELT data pipelines across ingestion, transformation, validation, reconciliation, and publishing layers.
- Design and operationalize data lake patterns with partitioning, data quality controls, lineage, governance, and reusable datasets or data products.
- Build and optimize distributed processing workloads using Apache Spark and formats such as Parquet and Avro.
- Tune compute, storage, Spark workloads, and applicable data warehouses for performance.
- Build and maintain Java-based ingestion services, orchestration helpers, APIs, and data access layers supporting data workflows.
- Develop REST APIs and integration components for downstream consumption and platform interoperability.
- Implement code quality, automated testing, CI/CD, observability, security-by-design, and operational readiness practices.
- Lead adoption of enterprise-authorized AI-assisted engineering tools while establishing standards for validating correctness, performance, and security.
- Lead technical discussions and drive delivery across cross-functional teams.
Requirements
- Formal training or certification in software engineering concepts and 5+ years of applied experience.
- 10+ years of software engineering experience with strong depth in data engineering and big data platforms.
- Strong hands-on Java development experience and proficiency in at least one data-focused language such as Python.
- Experience designing robust ETL/ELT pipelines and data integration frameworks.
- Strong Apache Spark and distributed processing experience, including fault tolerance, partitioning, and performance tuning.
- Understanding of Parquet, Avro, data lake or lakehouse patterns, data quality, metadata, and governance.
- Experience with Snowflake or an equivalent cloud data warehouse.
- Experience leading responsible adoption of AI-assisted software development tools, including validation of AI outputs and secure handling of sensitive data.
- Ability to lead technical discussions, communicate with varied stakeholders, and drive cross-functional delivery.
- Preferred experience includes Airflow, Dagster, Control-M, Kafka, Spark Structured Streaming, CDC, upserts, Delta Lake, Apache Iceberg, Hudi, on-call operations, SLAs/SLOs, observability, incident response, regulated environments, FinOps, and agentic AI or generative AI solutions.
Benefits
- Competitive total rewards package with base salary determined by role, experience, skill set, and location, plus possible incentive compensation for eligible roles.
- Health care coverage, on-site health and wellness centers, retirement savings plan, backup childcare, tuition reimbursement, mental health support, and financial coaching.
Tech Stack
Categories
BackendData Engineering
About JPMorgan Chase
JPMorgan Chase provides consumer and commercial banking, payments, credit card, wealth management, and corporate and investment banking services to individuals, businesses, institutions, and governments. The public company (NYSE: JPM) earns revenue from interest, fees, trading, and asset management across operations in more than 100 markets. Headquartered in New York City with roots dating to 1799, it serves retail customers and prominent corporate and government clients through brands including Chase and J.P. Morgan.