Senior Software Engineer, Data Engineering
ValGenesis11 months ago
Chennai, IndiaMid Level / Senior
Responsibilities
- Design, develop, and maintain batch and real-time data ingestion, transformation, and orchestration pipelines.
- Build and optimize lakehouse architectures using Azure Synapse, Delta Lake, or similar frameworks.
- Integrate and manage structured and unstructured data from SQL/NoSQL systems, files, documents, and IoT streams.
- Develop and operationalize ETL/ELT pipelines using Azure Data Factory, Databricks, or Apache Spark.
- Prepare and serve ML-ready datasets for model training and inference in collaboration with data scientists.
- Implement data quality, lineage, governance, monitoring, observability, scalability, and automation across data workflows.
- Enable self-service analytics with Power BI, Superset, and Tableau.
- Deploy and maintain data APIs and ML models in production using Azure ML, Kubernetes, and CI/CD pipelines.
- Collaborate with engineering, product, and business teams to translate insights into action.
Requirements
- 4 to 8 years of experience in Data Engineering or equivalent roles.
- Strong Python and SQL skills plus at least one compiled language: C#, Java, or Scala.
- Experience with relational databases including SQL Server, PostgreSQL, and MySQL.
- Experience with NoSQL systems including MongoDB and Cosmos DB.
- Hands-on experience with Azure Data Lake and Databricks.
- Experience with Azure Data Factory, Apache Airflow, or dbt for ETL/ELT.
- Experience with Kafka, Event Hubs, or Service Bus for real-time processing.
- Familiarity with TensorFlow, PyTorch, and MLOps concepts.
- Experience with Power BI, Apache Superset, or Tableau.
- Experience with Azure, Docker, Kubernetes, and GitHub Actions or Azure DevOps.
- Understanding of data modeling, version control, and CI/CD for data systems.
- Preferred experience with knowledge graphs, semantic search, LLM-based RAG patterns, data mesh, data fabric, domain-oriented data architecture, MLflow, Delta Live Tables, or Databricks Unity Catalog.
- Strong analytical, problem-solving, communication, collaboration, and attention-to-detail skills.
Benefits
- Chennai, Hyderabad, and Bangalore office roles are onsite five days per week.
- Equal-opportunity employment with consideration based on merit and protected characteristics.
Tech Stack
Apache AirflowApache KafkaApache SparkApache SupersetAzureC#DatabricksdbtDockerGitHub ActionsJavaKubernetesMicrosoft SQL ServerMLflowMongoDBMySQLPostgreSQLPythonPyTorchScalaSQLTensorFlow
Categories
Data Engineering
About ValGenesis
ValGenesis builds enterprise software for digitizing validation and manufacturing workflows in the life sciences, centered on its Validation Lifecycle Management System (VLMS) and eLog tools. Customers use it to manage equipment, cleaning, process, and computer system validation with audit trails and compliance support. Founded in 2005 and headquartered in Santa Clara, the privately held firm says its products are used by 30 of the top 50 global pharma and biotech companies.