Shop Your Way - Cloud Data/Systems Engineer
Shop Your Wayalmost 4 years ago
Remote, United StatesMid Level
Responsibilities
- Provide technical subject matter expertise from concept through operations for large-scale enterprise data solutions.
- Design and implement scalable data solution architectures, cloud infrastructure, and secure, reliable data-centric services on GCP.
- Use cloud infrastructure to analyze data, build models, and produce reports and visualizations.
- Integrate datasets from multiple sources for data modeling and design, build, and maintain data-processing pipelines.
- Design, build, implement, and manage APIs and API proxies, including internal and external API development processes.
- Troubleshoot and test product features, systems, and functionality.
- Automate delivery components to reduce development and production labor.
- Translate business problems into technical data problems in collaboration with product management.
- Create and maintain machine learning and statistical models, including recommender-system models.
- Extract, load, transform, clean, validate, query, and visualize data.
Requirements
- Bachelor’s, master’s, or PhD in Computer Science, Engineering, Economics, Business, or Mathematics.
- At least 3 years of experience designing and optimizing data models on GCP using data stores such as BigQuery and BigTable.
- At least 3 years of experience re-architecting and re-platforming on-premise data warehouses to GCP data platforms using GCP and third-party services.
- At least 3 years of experience architecting data lakes for analytics and BI integrations and implementing production-scale API solutions on GCP.
- At least 3 years of experience assessing current-state data platforms and defining transition paths to GCP.
- At least 3 years of experience designing and building production data pipelines using Java, Python, or Scala in hybrid big data architectures.
- Hands-on experience with Spark, Cloud Dataproc, Cloud Dataflow, Apache Beam, BigTable, Cloud BigQuery, Cloud Pub/Sub, and Cloud Functions.
- Experience with metadata management, data governance, and data platform security on GCP.
- Experience designing operations architecture and conducting performance engineering for large-scale production data lakes.
- Experience architecting and operating large production Hadoop or NoSQL clusters on-premise or in the cloud.
- Preferred qualifications include expertise in network, internet, and data-storage security; software engineering practices; source management; testing and operations processes; cross-functional communication; and troubleshooting.
- Google Cloud Platform certification is preferred.
Tech Stack
Categories
Data Engineering