Software Engineer III - Python, Databricks and AWS
JPMorgan Chase7 hours ago
Base Salary
$138k - $185k/yr
Responsibilities
- Implement resilient batch, streaming, and CDC ingestion from AWS sources into Databricks.
- Build maintainable data pipelines using Delta Live Tables or Databricks Jobs with testing and documentation.
- Productionize workloads through Databricks Workflows and Jobs with retries, checkpointing, idempotency, and safe reruns.
- Implement data governance and security controls covering least privilege, PII classification, auditing, lineage, metadata, and controlled data sharing.
- Tune Spark and Delta workloads, right-size clusters, optimize storage layouts, and manage job and warehouse costs.
- Use Terraform for Databricks and AWS infrastructure and promote resources across environments.
- Create unit and integration tests, data quality checks, contract tests, replay procedures, and backfill procedures.
- Maintain monitoring and alerting, participate in incident response and root-cause analysis, and manage SLOs and SLAs.
- Use enterprise-authorized AI-assisted development tools for coding, testing, troubleshooting, and documentation while validating generated outputs.
- Maintain version control, code reviews, documentation, and operational runbooks.
Requirements
- Formal training or certification in software engineering concepts and at least 3 years of hands-on software development lifecycle experience.
- Experience building and maintaining data pipelines.
- Hands-on experience building and operating a Databricks Lakehouse hosted in AWS.
- Experience with Delta Lake, including ACID tables, partitioning, and schema evolution.
- Proven experience with Spark on Databricks, including performance tuning, cluster sizing, skew mitigation, joins, caching, and file sizing.
- Experience with streaming and batch pipelines, Structured Streaming, incremental processing, backfills, and late-arriving data.
- Strong AWS fundamentals for data platforms, including S3, IAM, KMS, VPCs, security groups, logging, and auditing.
- Experience implementing Databricks data governance and security controls such as Unity Catalog, table and column permissions, and credential passthrough patterns.
- Experience with monitoring, alerting, incident response, root-cause analysis, and SLO/SLA management.
- Understanding of responsible AI use in engineering workflows, including data sensitivity, secure input and output handling, resiliency, and security expectations.
- Hands-on experience with enterprise-authorized AI-assisted software development tools and the ability to evaluate, validate, and refine their outputs for correctness, performance, and security.
- Preferred: familiarity with modern front-end technologies and exposure to cloud technologies.
Benefits
- Competitive total rewards and benefits package, with additional compensation details provided during the hiring process.
- Eligible roles may receive commission-based pay or discretionary incentive compensation in cash and/or forfeitable equity.
- Benefits may include comprehensive health care coverage, on-site health and wellness centers, retirement savings, backup childcare, tuition reimbursement, mental health support, and financial coaching.
Tech Stack
Categories
Data Engineering
About JPMorgan Chase
JPMorgan Chase provides consumer and commercial banking, payments, credit card, wealth management, and corporate and investment banking services to individuals, businesses, institutions, and governments. The public company (NYSE: JPM) earns revenue from interest, fees, trading, and asset management across operations in more than 100 markets. Headquartered in New York City with roots dating to 1799, it serves retail customers and prominent corporate and government clients through brands including Chase and J.P. Morgan.