Sr Big Data Engineer - Oozie and Pig (GCP)
Rackspace
3 months ago
Remote, United States
Senior
H1B Sponsor
Base Salary
$116k - $198k/yr
Responsibilities
- Design and develop scalable batch processing systems using Hadoop, Oozie, Pig, Hive, MapReduce, and HBase.
- Lead Jira Epics to manage project tasks and deliverables.
- Write clean, efficient, and production-ready code focusing on data structures and algorithms.
- Develop, manage, and optimize complex data workflows within the Apache Hadoop ecosystem.
- Leverage GCP tools like Dataproc, GCS, and Composer for cloud-native big data solutions.
- Implement DevOps and automation best practices, including CI/CD pipelines and infrastructure as code.
Requirements
- Bachelor's degree in Computer Science, software engineering, or a related field.
- 5+ years of experience in customer-facing software/technology or consulting.
- 5+ years of experience with on-premises to cloud migrations or IT transformations.
- 5+ years of experience building and operating solutions on GCP.
- Proficiency in Oozie, Pig, and MapReduce.
- Strong programming skills in Java (specifically Spark), Python, and SQL.
- Expertise in the Apache Hadoop ecosystem.
- Familiarity with BigTable and Redis.
- Experience in applying DevOps principles and using CI/CD tools.
Tech Stack
Apache AirflowApache HadoopApache HBaseApache HiveApache PigGoogle Cloud PlatformJavaPythonRedisSQLTerraform
Categories
Data EngineeringDevOps