5 days ago
Bengaluru, IndiaStaff+
Responsibilities
- Lead the technical design, architecture, and implementation of high-throughput data ingestion systems and ETL pipelines.
- Mentor and guide a team of engineers while promoting technical excellence and best practices.
- Write clean, scalable, and maintainable code primarily in Java and Python.
- Monitor and resolve high memory and CPU consumption, data-transfer bottlenecks, and queue-processing issues.
- Own the software development lifecycle from system design and data formatting or analytics through deployment and infrastructure management.
- Partner with product managers and scrum masters on sprint planning, Scrum ceremonies, and feature delivery.
Requirements
- Expert-level proficiency in Java and Python.
- Deep understanding of Apache Spark, ETL pipeline design, data formatting, and data analytics.
- Hands-on experience with SQL and NoSQL databases, including PostgreSQL, MySQL, DynamoDB, MongoDB, and Cassandra.
- Experience with message brokers and queues such as Kafka, AWS SQS, and RabbitMQ.
- Extensive experience with AWS cloud services, including S3, EC2, EMR, Lambda, and RDS.
- Experience managing deployment pipelines with GitLab or Jenkins.
- Experience deploying applications with Kubernetes and Docker.
- Familiarity with Terraform or AWS CloudFormation for infrastructure provisioning.
- Basic understanding of UI development with HTML, CSS, JavaScript, or modern frameworks.
- Seven to eight years of professional software engineering experience, including one to two years in a leadership, lead, or architectural capacity.
- Proven experience building and scaling high-volume data ingestion or streaming platforms.
- Strong understanding of system design, data structures, and algorithms, along with excellent communication skills.
Tech Stack
Amazon DynamoDBApache CassandraApache KafkaApache SparkAWSCSSDockerHTMLJavaJavaScriptJenkinsKubernetesMongoDBMySQLPostgreSQLPythonRabbitMQSQLTerraform
Categories
BackendData Engineering
About Nielsen
Nielsen shapes the world’s media and content as a global leader in audience insights, data and analytics. Through our understanding of people and their behaviors across all channels and platforms, we empower our clients with independent and actionable intelligence so they can connect and engage with their audiences—now and into the future. Nielsen operates around the world in more than 55 countries. Learn more at http://nlsn.co/6006JMfty and connect with us on social media (LinkedIn, Twitter, Facebook and Instagram).