4 days ago
Taipei, TaiwanMid Level
Responsibilities
- Design, implement, and maintain cloud-based analytics and data processing services using Golang and AWS.
- Develop production ETL pipelines for structured and unstructured data at scale.
- Build scalable real-time and batch data ingestion and transformation systems.
- Collaborate with data engineers, backend developers, and product teams on data models and analytical workflows.
- Optimize data processing pipelines for performance, cost, and reliability.
- Ensure data accuracy, integrity, and security throughout the data lifecycle.
- Monitor data pipelines and cloud infrastructure, resolve incidents, and improve observability.
- Contribute to architecture decisions and engineering best practices for analytics systems.
Requirements
- At least 3 years of cloud or backend development experience using Golang or Python.
- Hands-on production experience building and maintaining ETL pipelines or data processing workflows.
- Strong familiarity with AWS Lambda, S3, Kinesis, Glue, Athena, and RDS.
- Understanding of distributed systems, data streaming, and cloud-native architecture.
- Proficiency in RESTful API development, microservices design, SQL, and analytics-oriented schema design.
- Knowledge of version control and CI/CD practices.
- Bachelor’s degree in Computer Science, Data Engineering, or a related technical field.
- Strong debugging, analytical thinking, and communication skills.
- Proficiency with AI development environments such as Cursor and Claude Code, and expertise in AI development techniques such as Spec-Driven Development.
- Preferred experience with real-time analytics systems and streaming platforms such as Apache Kafka or Amazon Kinesis.
- Preferred familiarity with data lakes, data warehousing, and metadata management.
- Preferred experience with data quality assurance, validation, and pipeline testing frameworks.
- Preferred understanding of data governance, compliance, and security best practices.
- Preferred experience optimizing cloud cost and performance in large-scale analytics environments.
- Preferred experience with ML feature pipelines or integrating data with machine learning workflows.
Tech Stack
Categories
Data Engineering
