Data Engineer
SumerSports LLC2 months ago
Remote, United StatesMid Level / Senior
Responsibilities
- Build and operate robust ingestion, cleaning, transformation, ETL/ELT, batch, and streaming data pipelines.
- Develop retrieval pipelines using RAG and vector search over structured sports statistics and unstructured sources.
- Model and maintain Delta, Parquet, and Iceberg data assets with reliability, versioning, and lineage tracking.
- Implement workflow orchestration, monitoring, dependency tracking, automated failure recovery, validation, schema enforcement, and audit logging.
- Partner with ML/AI, MLOps, sports data, product, and analytics teams to make datasets and tools discoverable and safe for AI applications and autonomous agents.
- Contribute to data platform evolution, tool evaluation, standards, developer experience, and performance and cost optimization across compute, storage, and orchestration systems.
Requirements
- 3–8 years of experience as a Data Engineer or ETL Developer in a production environment.
- Proficiency in Python and SQL, with strong familiarity with Databricks, Spark, or equivalent big-data frameworks.
- Experience with workflow orchestration tools such as Airflow, Dagster, Luigi, or Prefect.
- Strong understanding of data modeling, data warehousing, distributed data processing, and modern data lakehouse architectures.
- Familiarity with GitHub Actions, Infrastructure as Code, data pipeline testing frameworks, and CI/CD practices.
- Experience with cloud infrastructure such as AWS, GCP, or Azure and container orchestration using Docker or Kubernetes.
- Exposure to LLM-powered data tools, including text-to-SQL, RAG, agent/tool interfaces such as MCP, or natural-language analytics.
- Preferred experience with sports, telemetry, sensor data, streaming and event-driven processing, Kafka, Spark Structured Streaming, or Flink.
- Preferred familiarity with American football, the NFL, college football, data governance, lineage, observability tools, semantic layers, metric definitions, machine-learning model management, and MLOps.
Benefits
- Competitive salary and bonus plan, with compensation varying based on skills, experience, certifications, and work location; additional performance bonus or incentive compensation may apply.
- Comprehensive health insurance plan.
- 401(k) retirement savings plan with company match.
- Remote working environment.
- Flexible unlimited time off policy.
- 13 paid holidays, including the Monday after the Super Bowl.
Tech Stack
Apache AirflowApache FlinkApache KafkaAWSAzureDatabricksDockerGitHub ActionsGoogle Cloud PlatformKubernetesPythonSQL
Categories
Data Engineering