Lead Software Engineer - Machine Learning
JPMorgan Chase8 hours ago
Base Salary
$157k - $215k/yr
Responsibilities
- Design, build, and maintain an end-to-end ML training platform.
- Run and optimize single-node and distributed GPU training workloads for throughput, utilization, stability, reproducibility, and cost efficiency.
- Build and operate Kubernetes infrastructure for compute-intensive training workloads, including resource management and troubleshooting.
- Enable Gen AI and LLM training and fine-tuning workflows with evaluation harnesses, artifact and version governance, and scalable GPU execution.
- Implement observability through metrics, logs, dashboards, alerting, GPU telemetry, and operational runbooks.
- Partner with data engineering and platform teams to define interfaces, standards, security guardrails, access controls, and cost controls.
- Improve ML developer experience through standardized containers, CI/CD, templates, documentation, and self-service workflows.
- Lead adoption of approved AI-assisted software development tools and establish standards for validating their outputs for correctness, performance, and security.
- Coach engineers on responsible, secure, and compliant use of AI-assisted development practices.
Requirements
- Formal training or certification in software engineering concepts and at least 5 years of applied experience.
- Demonstrated experience running ML training in cloud environments and debugging infrastructure and code issues.
- Strong Python skills and sound engineering practices, including testing, code reviews, modular design, and dependency management.
- Experience building automation and CI workflows for ML codebases, including build, test, release, deployment, and promotion workflows.
- Hands-on experience with deep learning training workflows and PyTorch or TensorFlow.
- Understanding of data loading bottlenecks, mixed precision, checkpointing, reproducibility, and evaluation methodology.
- Experience with distributed training, DDP, FSDP, DeepSpeed concepts, collective communication, scaling, and bottleneck analysis.
- Ability to profile and optimize CPU/GPU utilization, memory, I/O throughput, networking, and scheduling.
- Experience with Kubernetes fundamentals and AWS services including EKS, ECR, S3, IAM, VPC/networking, CloudWatch, and EC2.
- Experience leading effective use of approved AI-assisted software development tools and setting expectations for validating AI outputs.
- Understanding of responsible AI use, data sensitivity, secure input and output handling, resiliency, and security expectations.
- Preferred experience across multiple cloud platforms, cloud-native networking and storage, large-scale training data pipelines, Spark, Ray, Dask, workflow orchestration, model registries, training cost optimization, and observability practices.
Benefits
- Competitive total rewards package with base salary determined by role, experience, skill set, and location; eligible roles may also receive commission or discretionary incentive compensation.
- Comprehensive health care coverage, on-site health and wellness centers, retirement savings plan, backup childcare, tuition reimbursement, mental health support, and financial coaching.
- Employment is contingent on review of criminal conviction history under Section 19 of the Federal Deposit Insurance Act.
Tech Stack
Categories
About JPMorgan Chase
JPMorgan Chase provides consumer and commercial banking, payments, credit card, wealth management, and corporate and investment banking services to individuals, businesses, institutions, and governments. The public company (NYSE: JPM) earns revenue from interest, fees, trading, and asset management across operations in more than 100 markets. Headquartered in New York City with roots dating to 1799, it serves retail customers and prominent corporate and government clients through brands including Chase and J.P. Morgan.