2 hours ago
Bengaluru, IndiaStaff+
Responsibilities
- Drive innovation by researching and applying emerging cloud technologies.
- Lead the design, development, and maintenance of data engineering infrastructure.
- Architect end-to-end data solutions covering collection, storage, modeling, transformation, and consumption.
- Act as a subject matter expert for migrating on-premises applications, multi-terabyte databases, and data warehouses to cloud-based platforms.
- Design conceptual and logical data models, flowcharts, source-to-target mappings, transformations, and data lineage.
- Design data-intensive applications with APIs and streaming data pipelines in partnership with application architects.
- Create data architecture, engineering, and migration patterns across relational, NoSQL, analytical, big data, key-value, and warehouse technologies.
- Establish best practices, coding standards, and technical documentation.
- Mentor and provide technical guidance to junior data engineers.
- Collaborate with data scientists, analysts, and business stakeholders to understand data requirements.
- Optimize data models and database designs for performance and reliability.
- Implement data quality controls and monitoring systems.
- Evaluate and recommend data technologies and tools.
- Deploy and manage cloud-based data solutions across AWS, Azure, and GCP.
- Ensure data security, compliance, and governance standards are met.
Requirements
- Bachelor's degree in Computer Science, Engineering, or a related technical field; a master's degree is preferred.
- 12+/15+ years of experience in data engineering roles and 5+ years in a technical leadership position.
- Experience with real-time, batch, NoSQL, and SQL technologies.
- Proficiency in data warehousing, data modeling, ETL design, and optimization.
- Deep understanding of data enrichment, transformation, security, movement, data architecture, golden records, and data integrity.
- Experience with Hadoop, Spark, Kafka, Trino, Hive, and Iceberg.
- Experience with data orchestration tools such as Airflow or Luigi.
- Hands-on GenAI experience from the last 1–3 years, including LLM integration, prompt engineering, RAG, MCP, agentic AI, orchestration frameworks, and inference optimization.
- Familiarity with AWS Database Migration Service and Server Migration Service.
- Programming experience with Python, SQL, and Java or Scala.
- Experience with cloud platforms, preferably AWS, and with data warehousing, ETL/ELT, databases, containers, orchestration, version control, and CI/CD tools listed in the posting.
- Knowledge of dimensional, relational, and NoSQL data modeling, database design and optimization, batch and real-time pipeline architecture, data mesh and data fabric architectures, and data lakehouse implementation.
- Understanding of API design and development and microservices architecture is a plus.
Tech Stack
Amazon DynamoDBAmazon RedshiftApache AirflowApache CassandraApache HadoopApache HiveApache KafkaApache SparkAWSAzureDatabricksdbtDockerGitGitHub ActionsGitLab CI/CDGoogle BigQueryGoogle Cloud PlatformInfluxDBInformaticaJavaJenkinsKubernetesMicrosoft SQL ServerMongoDBMySQLPostgreSQLPythonScalaSnowflakeSQLTalend
Categories
Data Engineering
