Sr. Principal Architect-Data
Eli Lilly and Company22 hours ago
Hyderābād, IndiaStaff+
Responsibilities
- Architect multi-agent AI systems on Databricks, including orchestration, tool use, memory, state management, and inter-agent communication.
- Design LLM, RAG, structured-data reasoning, and autonomous workflow architectures.
- Own Databricks Lakehouse, Unity Catalog, Delta Lake, semantic layer, model serving, feature serving, and multi-workspace governance architectures.
- Define enterprise patterns for LLM integration, prompt and context management, vector search, embeddings, fine-tuning, MLflow lifecycle management, and responsible AI.
- Establish data governance, security, lineage, auditing, data quality, and regulated-workload standards across Databricks environments.
- Create architectural decision records, reference architectures, platform blueprints, and engineering standards.
- Lead architecture reviews, design critiques, proof-of-concept evaluations, mentoring, and vendor or industry engagement.
Requirements
- Bachelor’s degree in Computer Science, Data Engineering, Artificial Intelligence, or equivalent; an advanced degree is preferred.
- 10+ years of experience in data engineering, data platform architecture, or AI/ML engineering.
- 4+ years of hands-on production Databricks experience, including Unity Catalog, Delta Lake, Model Serving, Workflows, Databricks SQL, cluster lifecycle management, and workspace administration.
- 5+ years designing and delivering AI/ML systems at scale across training, feature engineering, serving, and monitoring.
- Demonstrated experience architecting agentic AI systems, multi-agent orchestration, tool-use patterns, LLM integration, RAG pipelines, and autonomous workflows.
- Expert-level SQL and PySpark plus strong Python for AI/ML pipeline development.
- Deep knowledge of data modeling, medallion architecture, Delta Lake internals, and lakehouse design patterns.
- Experience setting organization-wide technical standards, mentoring engineers, translating business requirements, and influencing technical direction at principal or distinguished level.
- Preferred qualifications include Databricks professional certifications, Mosaic AI, Databricks Apps, AI/BI Genie, DBRX, MLOps/LLMOps, Terraform, Databricks Asset Bundles, GitOps, Databricks Vector Search, Pinecone, Weaviate, embedding model management, and responsible AI governance.
- Pharmaceutical, healthcare, regulated-industry, GxP, GDPR, HIPAA, Customer Master, Product Master, HCP/HCO, or commercial alignment data experience is preferred.
Categories
About Eli Lilly and Company
Eli Lilly and Company researches, develops, manufactures, and markets prescription medicines and biologics for patients, sold through healthcare providers, pharmacies, and payers. Core therapeutic areas include diabetes, oncology, immunology, neuroscience, and obesity. Founded in 1876 and headquartered in Indianapolis, it is a public company (NYSE: LLY) with global operations and is building a new advanced manufacturing site for gene therapy and other modalities in Lebanon, Indiana.