
Software Engineer, Data Infrastructure
Thinking Machines Lab2 days ago
Base Salary
$300k - $400k/yr
Responsibilities
- Design, build, and operate scalable, fault-tolerant distributed compute, data orchestration, and multimodal storage infrastructure for LLM research.
- Develop high-throughput systems for data ingestion, processing, transformation, training-data catalogs, deduplication, quality checks, and search.
- Build traceability, reproducibility, and quality-control systems across the data lifecycle.
- Implement and maintain monitoring and alerting for platform reliability and performance.
- Collaborate with research teams to improve data quality, unlock features, and accelerate training cycles.
- Own projects end-to-end and work across teams and technology stacks to deliver infrastructure.
Requirements
- Bachelor’s degree or equivalent experience in computer science, engineering, or a similar field.
- Proficiency in at least one backend language, specifically Python or Rust.
- Fluency with distributed compute frameworks such as Apache Spark or Ray.
- Deep familiarity with cloud infrastructure, data lake architectures, and batch and streaming pipelines.
- Comfort operating across the stack and owning projects end-to-end.
- Preferred: hands-on experience with Kafka, dbt, Terraform, and Airflow.
- Preferred: experience building a web crawler and scaling deduplication, data mining, and search systems.
- Preferred: strong knowledge of file formats and storage systems, including Parquet and Delta Lake.
- Preferred: proactive documentation, testing, and development of tools that empower teammates.
Benefits
- Based in San Francisco, California or New York, New York.
- Generous health, dental, and vision benefits.
- Unlimited paid time off.
- Paid parental leave.
- Relocation support as needed.
Tech Stack
Categories
Data Engineering
About Thinking Machines Lab
Thinking Machines Lab develops AI and generative AI software and conducts applied research to help organizations make data-driven decisions. The company builds products and data science solutions for enterprise use cases, pairing foundational models with practical tooling and services across industries. It is privately held and headquartered in San Francisco.