3 days ago
Base Salary
$119k - $200k/yr
Responsibilities
- Develop distributed, scalable, reliable real-time data pipelines and perform offline analysis of large datasets.
- Create data environments and datasets for data scientists, data analysts, business analysts, and other users.
- Author ETL scripts and curate data for datalakes, data warehouses, and datamarts.
- Design, deploy, monitor, and maintain Hadoop, Spark, database, and big-data infrastructure in AWS.
- Troubleshoot complex data provenance, metadata, engineering, and system-performance issues.
- Investigate, evaluate, test, and recommend technical solutions and emerging big-data technologies.
- Own product features from development through production deployment.
- Develop tools and procedures to monitor and automate tasks on servers and clusters.
- Collaborate with technical teams on data tools supporting operational and product use cases.
- Lead complex data initiatives, advise management, and mentor junior engineers and colleagues.
Requirements
- Bachelor’s degree in computer science, computer engineering, or a related technical field, or 12+ years of related data software engineering experience in lieu of the degree.
- 8+ years of professional experience as a data software engineer.
- 2+ years of experience designing, provisioning, and tuning AWS cloud computing environments.
- Python experience is required, with advanced experience in Scala, Ruby, R, SQL, AWS RDS, Redshift, and MySQL.
- Advanced experience with Hadoop MapReduce, Elasticsearch, Pig, Hive, Cascading or Scalding, Apache Spark, and AWS EMR.
- Advanced experience with Kinesis, Kafka, MQTT, and relational or NoSQL databases including DynamoDB.
- Experience with Git, Jenkins, and AWS Code tools.
- Knowledge of database concepts, object and data modeling, database architectures, software, facilities, and performance tuning.
- Previous experience as a Data Engineer, Database Administrator, or Business Intelligence Analyst.
- Related AWS certification is preferred.
- Strong communication, technical judgment, complex problem-solving, project leadership, and mentoring abilities.
Benefits
- Unlimited PTO for exempt salaried employees, or accrued vacation, sick, and personal days for non-exempt employees; 11 paid holidays and a U.S. company-wide shutdown between Christmas and New Year.
- Medical, dental, and vision insurance with multiple coverage options.
- 401(k) with a 50% match on contributions up to 8%, fully vested from day one.
- Wellness, counseling, FSA, HSA, life, disability, critical illness, accident, legal assistance, pet, identity theft, dependent care, education assistance, commuter, employee purchase, and service award programs.
- The posting states that all candidates must have valid authorization to work in the U.S.
Tech Stack
Amazon DynamoDBAmazon RedshiftApache HadoopApache HiveApache KafkaApache PigApache SparkAWSElasticsearchGitJenkinsMySQLPythonRRubyScalaSQL
Categories
Data Engineering