Data Engineer IV
PDI Technologies3 months ago
Chennai, IndiaStaff+
Responsibilities
- Design, develop, and optimize scalable cloud-based data pipelines for enterprise analytics, reporting, and business intelligence.
- Lead complex data integration solutions using AWS-native technologies and modern data engineering practices.
- Design data models, ingestion frameworks, and transformation processes that provide reliable, high-quality analytics.
- Apply data architecture, governance, metadata management, and data quality principles to scalable and maintainable solutions.
- Establish engineering standards, reusable frameworks, automation, infrastructure as code, and DevOps practices.
- Partner with business stakeholders to translate reporting and analytics requirements into technical solutions.
- Troubleshoot and resolve complex data platform, integration, and performance issues.
- Mentor data engineers through code reviews, knowledge sharing, and best-practice adoption.
- Evaluate emerging technologies and recommend improvements to platform performance, scalability, security, and operational efficiency.
Requirements
- Bachelor's degree in computer science, information technology, engineering, or a related field, or equivalent practical experience.
- 8+ years of experience in data engineering, cloud data platforms, or related software engineering roles.
- Expert experience building cloud-based data solutions with AWS services including S3, Glue, Redshift, AppFlow, Lake Formation, and Step Functions.
- Advanced proficiency with SQL, Python, and PySpark for scalable ETL/ELT solutions.
- Practical experience applying AI/ML techniques to data engineering problems such as automated data quality checks, anomaly detection, or intelligent pipeline monitoring.
- Experience developing semantic and contextual layers within a data lakehouse.
- Strong understanding of dimensional modeling, data warehousing, data lakehouse concepts, Apache Iceberg, Parquet, data governance, and metadata management.
- Experience integrating enterprise platforms such as Salesforce, NetSuite, and other SaaS applications.
- Experience with Terraform, Git, Azure DevOps, and CI/CD pipelines.
- Ability to mentor engineers, influence technical decisions, and drive engineering best practices without formal people leadership.
- Strong analytical, problem-solving, communication, and cross-functional collaboration skills.
Benefits
- Competitive salary, market-competitive benefits, and a quarterly perks program.
- Ample time off and hybrid working arrangements where appropriate.
- Continuous learning, professional certifications, and leadership development opportunities.
- Supportive global culture focused on diversity, inclusion, authenticity, trust, curiosity, and diversity of thought.
About PDI Technologies
PDI Technologies builds enterprise software and services for the convenience retail and petroleum wholesale ecosystem, spanning ERP, logistics/dispatch, fuel pricing, loyalty, payments, and cybersecurity. It sells cloud software and managed services to retailers, fuel marketers, and distributors worldwide, with embedded analytics and mobile workflows. Founded in 1983 as Professional Datasolutions, the privately held company is headquartered in Alpharetta, Georgia and has expanded through multiple acquisitions.