6 days ago
Bengaluru, IndiaSenior
Responsibilities
- Build, test, maintain, and operate data pipelines and data-product components using Spark, dbt, Airflow, or equivalent cloud-native services.
- Execute ingestion, transformation, and loading jobs from operational systems, IoT devices, and clinical sources into data platforms.
- Troubleshoot pipeline failures, data-quality anomalies, and performance bottlenecks, and implement fixes without service disruption.
- Validate pipelines against data contracts, schema standards, and access-control policies.
- Optimize pipeline performance and cloud infrastructure costs through runtime profiling, partitioning, and resource configuration.
- Document pipeline designs, data lineage, and operational runbooks.
- Coach less-experienced colleagues on data-engineering patterns, testing practices, and cloud-platform tooling.
Requirements
- Bachelor’s degree in Computer Science, Data Engineering, Information Technology, or a related field.
- At least 6 years of experience.
- Proficiency in SQL, Python, PySpark, Spark, or another programming language such as Java, Scala, or R.
- Experience building highly scalable, performant, resilient, and cost-optimized data pipelines.
- Experience with modern data-processing frameworks such as Snowflake, Databricks, or AWS SageMaker Unified Studio.
- Hands-on experience with AWS services including Amazon S3, IAM, Lake Formation, AWS Glue, and Lambda.
- Strong understanding of relational databases and lakehouse table formats including Apache Iceberg and Delta Lake.
- Expertise designing, building, maintaining, and documenting conceptual, logical, and physical data models.
- Understanding of data governance, access control, cataloging, lineage, encryption, auditability, and compliance requirements.
- Ability to implement secure cloud environments and role-based access controls using IAM and Lake Formation.
- Preferred: familiarity with RAG data pipelines, data hub/mesh/fabric architecture patterns, RESTful APIs, data products, and GxP or computerized system validation.
- Preferred: relevant GCP, AWS, or Azure certifications.
- Preferred: experience with Terraform, Git, GitHub, GitHub Actions, and CI/CD practices.
Benefits
- Equal opportunity employer committed to fairness and respect at work.
- Location: Bangalore, Karnataka, India.
Tech Stack
Apache AirflowApache SparkAWSAzureDatabricksdbtGitGitHub ActionsGoogle Cloud PlatformJavaPythonRScalaSnowflakeSQLTerraform
Categories
Data Engineering
About Bayer
Bayer is a global life-sciences company that develops and sells prescription pharmaceuticals, consumer health brands (including Aspirin), and agricultural seeds, traits, and crop-protection products, plus digital farming tools, for patients, consumers, and farmers. It is a public company founded in 1863 and headquartered in Leverkusen, Germany. Bayer acquired Monsanto in 2018, making Crop Science one of its three core businesses alongside Pharmaceuticals and Consumer Health.