NielsenIQ

Principal Data Engineer

NielsenIQ
Apply
3 hours ago
Mexico City, MexicoStaff+

Responsibilities

  • Design, develop, and maintain scalable data pipelines for extracting, transforming, and loading data from multiple sources.
  • Build and extend automation for infrastructure provisioning, auto-scaling, code deployment, monitoring, alerting, reporting, and failover.
  • Collaborate on a resilient data platform with strong availability, reliability, and performance.
  • Use Apache Airflow to orchestrate and manage complex ETL workflows.
  • Optimize pipeline performance, troubleshoot issues, and ensure data quality through monitoring, testing, and validation.
  • Evaluate and introduce technologies that improve data infrastructure.
  • Design and implement data security and access-control policies.
  • Coordinate with DevOps teams on deployment and monitoring of data pipelines and workflows.

Requirements

  • Bachelor’s or master’s degree in computer science, data engineering, or a related field.
  • At least 7 years of experience as a data engineer, including ETL processes and data warehousing.
  • At least 4 years of experience using Apache Airflow for data-pipeline orchestration.
  • Experience with ETL tools and frameworks such as Apache Spark and Matillion.
  • Proficiency in Python, Java, or other relevant programming languages.
  • Knowledge of data modeling, data warehousing, and SQL.
  • Expertise working with structured and unstructured data.
  • Experience with cloud-based data solutions such as AWS, GCP, or Azure.
  • Understanding of data storage, transformation, and integration concepts.
  • Knowledge of data security and encryption best practices is preferred.
  • Strong problem-solving, communication, collaboration, organization, and prioritization skills.
  • Ability to work with distributed global teams; English is required.

Benefits

  • Flexible working environment.
  • Volunteer time off.
  • LinkedIn Learning.
  • Employee Assistance Program (EAP).
  • Annual compensation of up to MXN 1542000.
  • Global work environment with distributed teams.

Tech Stack

Apache AirflowApache SparkAWSAzureDatabricksdbtGoogle Cloud PlatformJavaPythonSnowflakeSQL

Categories

Data Engineering
Contact me