about 4 hours ago
San Francisco, CA, USA
Senior
H1B Sponsor
Base Salary
$185k - $327k/yr
Responsibilities
- Design, build, and operate scalable data replication and ingestion pipelines.
- Develop and enhance Kafka Iceberg connectors and data loading frameworks.
- Drive the modernization of Block's CDC platform with next-generation approaches.
- Build self-service tooling and observability features for internal teams.
- Collaborate with teams to define data contracts and improve service encapsulation.
- Contribute to the unification of Block's data ingestion architecture.
- Design and implement solutions for PII detection and privacy-compliant data handling.
- Establish best practices for data pipeline reliability and compliance.
Requirements
- 8+ years of experience in software engineering or data platform development.
- Strong programming proficiency in Java, Python, Scala, or Go.
- Hands-on experience with streaming data systems like Apache Kafka.
- Solid understanding of Change Data Capture (CDC) and database replication patterns.
- Experience with modern data storage formats like Apache Iceberg or Delta Lake.
- Experience with cloud-based data ecosystems and infrastructure-as-code tools.
Benefits
- Remote work flexibility.
- Medical insurance.
- Flexible time off.
- Retirement savings plans.
- Modern family planning options.
Tech Stack
Apache AirflowApache KafkaApache SparkAWSAzureDatabricksGoGoogle Cloud PlatformJavaPythonScalaSnowflakeTerraform
Categories
AI & MLData EngineeringData Science