about 9 hours ago
Responsibilities
- Write clean, maintainable, and efficient code while adhering to best practices.
- Design, develop, and maintain data pipelines and ETL workflows using Apache Spark and Apache Airflow.
- Optimize data storage, retrieval, and processing systems for reliability and performance.
- Develop and fine-tune complex queries and data processing jobs for large-scale datasets.
- Monitor, troubleshoot, and improve data systems for minimal downtime and maximum efficiency.
- Collaborate with data scientists and software engineers to deliver integrated solutions.
- Provide technical guidance and mentorship to junior engineers.
Requirements
- Bachelor's degree in Computer Science, Engineering, or a related field.
- 5+ years of experience in software and/or data engineering with expertise in big data technologies.
- Strong understanding of SOLID principles and distributed systems architecture.
- Proven experience in distributed data processing, data warehousing, and real-time data pipelines.
- Advanced SQL skills with expertise in query optimization for large datasets.
- Exceptional problem-solving abilities and capacity to work independently or collaboratively.
- Excellent verbal and written communication skills.
- Experience with cloud platforms such as AWS, GCP, or Azure, and containerization tools like Docker and Kubernetes (preferred).
- Familiarity with additional big data technologies, including Hadoop, Kafka, and Presto (preferred).
- Strong programming skills in Python, Java, or Scala (preferred).
- Knowledge of CI/CD pipelines, DevOps practices, and infrastructure-as-code tools (preferred).
- Expertise in data modeling, schema design, and data visualization tools (preferred).
- AI literacy and curiosity regarding Generative AI.
Benefits
- Comprehensive benefits including healthcare, life, accident, and disability insurance.
- Global access to mental health and financial wellness support.
- Flexible work arrangements with a hybrid work approach.
- Support for taking time off in accordance with local leave policies.