2 months ago
Bengaluru, IndiaStaff+
Responsibilities
- Lead the technical design, architecture, and implementation of high-throughput data ingestion systems and ETL pipelines.
- Mentor and guide a team of engineers while promoting technical excellence and best practices.
- Write clean, scalable, and maintainable code primarily in Java and Python.
- Monitor and resolve high memory and CPU consumption, data-transfer bottlenecks, and queue-processing issues.
- Own the software development lifecycle from system design and data formatting or analytics through deployment and infrastructure management.
- Partner with product managers and scrum masters on sprint planning, Scrum ceremonies, and feature delivery.
Requirements
- Expert-level proficiency in Java and Python.
- Deep understanding of Apache Spark, ETL pipeline design, data formatting, and data analytics.
- Hands-on experience with SQL and NoSQL databases, including PostgreSQL, MySQL, DynamoDB, MongoDB, and Cassandra.
- Experience with message brokers and queues such as Kafka, AWS SQS, and RabbitMQ.
- Extensive experience with AWS cloud services, including S3, EC2, EMR, Lambda, and RDS.
- Experience managing deployment pipelines with GitLab or Jenkins.
- Experience deploying applications with Kubernetes and Docker.
- Familiarity with Terraform or AWS CloudFormation for infrastructure provisioning.
- Basic understanding of UI development with HTML, CSS, JavaScript, or modern frameworks.
- Seven to eight years of professional software engineering experience, including one to two years in a leadership, lead, or architectural capacity.
- Proven experience building and scaling high-volume data ingestion or streaming platforms.
- Strong understanding of system design, data structures, and algorithms, along with excellent communication skills.
Tech Stack
Amazon DynamoDBApache CassandraApache KafkaApache SparkAWSCSSDockerHTMLJavaJavaScriptJenkinsKubernetesMongoDBMySQLPostgreSQLPythonRabbitMQSQLTerraform
Categories
BackendData Engineering
About Nielsen
Nielsen provides audience measurement and analytics that quantify who watches, listens, and buys across TV, digital, radio, and streaming. It sells ratings, cross-platform datasets, and advertising effectiveness tools to media owners, advertisers, and agencies on subscription and consulting agreements. Founded in 1923 and headquartered in New York, it operates in 55+ countries and is known for U.S. TV ratings.