Sr. Principal Architect-Data
Eli Lilly and Company23 hours ago
Hyderābād, IndiaStaff+
Responsibilities
- Architect multi-agent AI systems on Databricks, including orchestration, tool use, memory, state management, and inter-agent communication.
- Design LLM, RAG, structured-data reasoning, and autonomous decision-making workflows.
- Own Databricks Lakehouse architecture, including Unity Catalog, medallion architecture, Delta Lake, governance, and multi-workspace standards.
- Design real-time and batch model serving, feature engineering and serving, vector search, embedding pipelines, and LLM fine-tuning workflows.
- Establish data governance, security, lineage, auditing, data quality, responsible AI, and regulated-workload architecture.
- Create architecture decision records, reference architectures, platform blueprints, and technical standards for engineering teams.
- Lead architecture reviews and proof-of-concept evaluations, mentor senior data and ML engineers, and influence vendors and enterprise technical direction.
Requirements
- Bachelor’s degree in Computer Science, Data Engineering, Artificial Intelligence, or equivalent; an advanced degree is preferred.
- 10+ years of experience in data engineering, data platform architecture, or AI/ML engineering.
- 4+ years of hands-on production Databricks experience, including Unity Catalog, Delta Lake, Model Serving, Workflows, Databricks SQL, cluster lifecycle management, and workspace administration.
- 5+ years designing and delivering AI/ML systems at scale, including model training, feature engineering, model serving, and monitoring.
- Production experience architecting agentic AI systems, multi-agent orchestration, tool-use patterns, LLM integration, RAG pipelines, and autonomous workflows.
- Expert SQL and PySpark skills with strong Python experience for AI/ML pipeline development.
- Deep knowledge of data modeling, medallion architecture, Delta Lake internals, and lakehouse design patterns.
- Experience leading architecture at a principal or distinguished level, setting standards, mentoring engineers, and influencing organization-wide technical direction.
- Preferred qualifications include Databricks professional certifications, experience with Mosaic AI, Databricks Apps, AI/BI Genie, DBRX, MLOps, LLMOps, Terraform, Databricks Asset Bundles, GitOps, vector databases, embedding model management, and responsible AI governance.
- Pharmaceutical, healthcare, or regulated-industry experience involving GxP, GDPR, HIPAA, Customer Master, Product Master, HCP/HCO data, or commercial alignment data is preferred.
Categories
About Eli Lilly and Company
Eli Lilly and Company researches, develops, manufactures, and markets prescription medicines and biologics for patients, sold through healthcare providers, pharmacies, and payers. Core therapeutic areas include diabetes, oncology, immunology, neuroscience, and obesity. Founded in 1876 and headquartered in Indianapolis, it is a public company (NYSE: LLY) with global operations and is building a new advanced manufacturing site for gene therapy and other modalities in Lebanon, Indiana.