Lead Data Engineer II, IT
McCormick & Company, Incorporated2 months ago
Gurgaon, IndiaStaff+
Responsibilities
- Partner with data product managers to gather requirements and deliver data pipelines.
- Design, build, implement, and maintain scalable ETL and ELT pipelines from diverse source systems.
- Develop data models and perform advanced data cleaning, transformation, manipulation, and validation.
- Ensure data quality, security, availability, reliability, resiliency, and performance.
- Optimize large-dataset workflows using partitioning, indexing, clustering, and other performance strategies.
- Implement deployment automation and data validation processes.
- Collaborate with data science, AI, and data product teams to improve solution performance and cost effectiveness.
- Automate manual processes and improve the scalability and delivery of data products.
- Monitor and troubleshoot production data pipelines and resolve data errors.
- Provide expert-level support and guidance to enterprise data teams.
Requirements
- Bachelor’s degree in Mathematics, Statistics, Computer Science, Data Analytics/Science, or a related field.
- Eight or more years of data engineering experience.
- Microsoft Certified: Azure Data Engineer (DP203), Microsoft Certified: Fabric Data Engineer Associate, or a related cloud technology certification.
- Demonstrated coding ability in one or more languages, with PySpark preferred.
- Experience building data pipelines and owning production-grade pipelines.
- Ability to manage multiple priorities simultaneously.
- Knowledge graph experience is a plus.
- Fabric IQ or Databricks certifications are a plus.
Benefits
- 401(k), health insurance, and paid time off.
- Competitive compensation.
- Career growth opportunities.
- Flexibility and support for diverse life stages and choices.
Tech Stack
DatabricksSQL
Categories
Data Engineering
About McCormick & Company, Incorporated
McCormick & Company, Incorporated manufactures and markets spices, seasoning blends, condiments, and flavor solutions for consumers, food manufacturers, and foodservice operators. It sells both branded products for retail and custom flavor systems for B2B customers, distributing to more than 150 countries. Founded in 1889 and headquartered in Hunt Valley, Maryland, McCormick is a public company traded on the NYSE (MKC).