about 4 hours ago
Base Salary
$140k - $220k/yr
Responsibilities
- Design, build, and operate large-scale data acquisition pipelines.
- Architect resilient ETL/ELT workflows for various data processing methods.
- Develop production Java services for data ingestion and orchestration.
- Utilize technologies such as Apache Airflow, Apache Beam, and Kafka.
- Define approaches for schema evolution and data validation.
- Enhance reliability, performance, and cost efficiency of data pipelines.
- Implement monitoring and alerting for pipeline health and data quality.
- Collaborate with product and data teams to meet business requirements.
- Contribute to technical designs and system modernization efforts.
Requirements
- 5+ years of professional software engineering experience focused on backend systems.
- Proven experience building and operating production data pipelines at scale.
- Deep proficiency in Java and object-oriented design.
- Hands-on expertise with data processing technologies like Apache Beam and Spark.
- Strong experience with streaming systems such as Apache Kafka.
- Understanding of batch processing, data modeling, and data quality management.
- Experience designing high-throughput, fault-tolerant backend services.
- Familiarity with cloud services, preferably GCP.
- Bachelor's degree in Computer Science or related field.
Tech Stack
Apache AirflowApache BeamApache KafkaApache SparkGoogle BigQueryGoogle Cloud PlatformJavaKubernetesSnowflakeTerraform
Categories
BackendData Engineering