Senior Software Engineer II - Data Engineering and Platform
LG Ad Solutions3 months ago
Bengaluru, IndiaMid Level / Senior
Responsibilities
- Architect, develop, and deploy scalable ETL pipelines, data warehouses, data architectures, catalogs, and data delivery mechanisms.
- Define and execute the strategic vision for the data platform and oversee data storage, processing, and utilization systems.
- Collaborate with product managers, engineers, data scientists, and other stakeholders on data integration and hand-off mechanisms.
- Identify performance bottlenecks and improve platform performance, scalability, and reliability for petabyte-scale processing.
- Implement data governance frameworks and policies supporting data quality, security, compliance, and integrity.
- Provide technical leadership and mentorship to data engineers and guide complex technical challenges and deliveries.
- Lead code and architecture reviews and maintain organizational standards for code quality and system design.
- Evaluate emerging data engineering and platform technologies to improve platform capabilities and efficiency.
Requirements
- Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field.
- More than five years of software development experience, including at least two years in a technical leadership role overseeing data engineering or data platform teams.
- Strong proficiency with Apache Spark/PySpark, Apache Airflow, and Apache Kafka.
- Experience using Scala and Python to develop robust data pipelines.
- In-depth knowledge of relational and non-relational databases and complex query design and optimization.
- Hands-on experience with data visualization tools and techniques.
- Experience developing and maintaining CI/CD pipelines with tools such as Jenkins and GitHub Actions.
- Experience with infrastructure-as-code tools such as Pulumi and Terraform for cloud resource management.
- Experience writing and maintaining unit, integration, and end-to-end tests for data pipeline reliability and accuracy.
- Strong problem-solving, attention to detail, communication, and cross-functional collaboration skills.
- Familiarity with data governance and security practices.
- Preferred experience with Databricks, AWS cloud services, Agile development, Agile team leadership, and distributed graph databases.
Benefits
- Equal employment opportunity and anti-discrimination protections are provided.
- Reasonable accommodations are available throughout the hiring process for applicants with medical conditions or disabilities.
Tech Stack
Categories
Data Engineering
About LG Ad Solutions
LG Ad Solutions builds a connected TV advertising and analytics platform that helps brands plan, target, and measure campaigns across smart TVs and digital video. It combines premium LG smart TV ad inventory with TV viewership data and attribution tools for cross-screen activation and reporting. Founded in 2013 and headquartered in Mountain View, it operates as a subsidiary of LG Electronics.