Senior Lead Software Engineer - Data & AI Platform Engineer
JPMorgan Chase1 day ago
Base Salary
$157k - $215k/yr
Responsibilities
- Design, develop, and troubleshoot creative software solutions and production-grade data platform capabilities.
- Build and maintain scalable batch and real-time data pipelines and ETL/ELT workflows using Spark, Airflow, Kafka, and Flink.
- Develop data cataloging, data quality, semantic and metrics layers with governance, lineage, and compliance controls.
- Implement fact, dimensional, star-schema, snowflake, and wide-table data modeling strategies for analytics and downstream consumption.
- Partner with analytics teams, product managers, and business stakeholders to translate requirements into production solutions.
- Develop, review, debug, and maintain secure, high-quality production code.
- Automate recurring issue remediation and improve application and system operational stability.
- Evaluate external vendors, startups, and internal technologies and assess their architectural applicability.
- Lead development of an Agentic Autonomous Lakehouse for governed self-service pipeline provisioning and lakehouse health, cost, and performance operations.
- Lead software engineering communities of practice to promote new and emerging technologies.
Requirements
- Formal training or certification in software engineering concepts and 5+ years of applied experience.
- Professional experience in software engineering or data platform development, including system design, application development, testing, and operational stability.
- Advanced proficiency in one or more of Python, Java, and SQL.
- Hands-on experience with distributed data processing frameworks such as Apache Spark and Flink.
- Strong knowledge of data modeling techniques, including star schema and snowflake, and query optimization.
- Experience designing and operating Databricks data pipelines with orchestration tools such as Apache Airflow.
- Proficiency with cloud data services including AWS S3, Glue, Redshift, Athena, EMR, and Lake Formation, or equivalent services.
- Experience building production-grade data platforms on Kubernetes with open catalog integrations such as Apache Iceberg, Unity Catalog, or OpenMetadata.
- Experience developing Agentic AI, LLM, RAG, MCP, vector database, and embedding-based retrieval systems.
- Familiarity with data platform and transformation framework development, data mesh or data product architectures, infrastructure as code with Terraform, and containerized deployments with Docker and Kubernetes.
- Experience with data observability, data quality, metadata management, semantic layers, metrics stores, or BI platforms such as Tableau and dbt Metrics.
Benefits
- Benefits may include comprehensive health care coverage, on-site health and wellness centers, a retirement savings plan, backup childcare, tuition reimbursement, mental health support, and financial coaching.
- The employer offers an equal opportunity workplace and reasonable accommodations for religious practices, mental health needs, and physical disabilities.
Tech Stack
Amazon RedshiftApache AirflowApache FlinkApache KafkaApache SparkAWSDatabricksDockerJavaKubernetesPythonSQLTerraform
Categories
AI ApplicationsData Engineering
About JPMorgan Chase
JPMorgan Chase provides consumer and commercial banking, payments, credit card, wealth management, and corporate and investment banking services to individuals, businesses, institutions, and governments. The public company (NYSE: JPM) earns revenue from interest, fees, trading, and asset management across operations in more than 100 markets. Headquartered in New York City with roots dating to 1799, it serves retail customers and prominent corporate and government clients through brands including Chase and J.P. Morgan.