1 day ago
Responsibilities
- Design, build, and maintain scalable real-time streaming pipelines using Apache Kafka, AWS Kinesis, Spark Streaming, or similar technologies.
- Implement data and AI pipelines for structured, semi-structured, and unstructured data supporting AI and agentic solutions.
- Design data domains, reusable data products, semantic layers, real-time analytics systems, and RAG architectures.
- Model domain entities and relationships in knowledge graphs using technologies such as Neo4j, Amazon Neptune, RDF, Cypher, and SPARQL.
- Develop synthetic data generation and automated testing workflows for agent evaluation.
- Integrate semantic layers with AI and LLM platforms for secure, context-rich, low-latency data access.
- Improve data engineering productivity through AI-driven pipeline generation, self-healing pipelines, and automated data quality frameworks.
- Ensure pipeline reliability, availability, scalability, monitoring, alerting, incident management, fault tolerance, and disaster recovery.
- Collaborate with DevOps, infrastructure, architects, and stakeholders on deployment, operations, and the vision for data and AI pipelines.
- Mentor junior team members, lead communities of practice, and promote reusable data engineering patterns and standards.
- Design and apply GenAI solutions for insurance-specific data use cases.
Requirements
- 6+ years of hands-on data engineering experience building large-scale, complex enterprise data ecosystems on AWS, Azure, or GCP.
- Deep expertise with streaming platforms such as Apache Kafka, AWS Kinesis, and Spark Streaming, plus distributed processing frameworks.
- Experience with RAG architectures, vector search systems, chunking and embedding techniques, and LLM- or agentic-AI data pipelines.
- Experience with graph databases including Neo4j and Amazon Neptune, and query languages such as Cypher, SPARQL, or Gremlin.
- Proficiency in domain-driven data design, dimensional modeling, semantic layer integration, and reusable data products.
- High proficiency in Python, Scala, or Java, along with SQL, DataOps, CI/CD, and containerized deployments.
- Financial industry experience with complex, multi-structured domain data is strongly preferred.
- Exceptional leadership, stakeholder communication, cross-functional collaboration, and mentoring skills.
Benefits
- Remote work location in the USA.
- Contract engagement structured as C2C.
- Opportunity to work with Fortune 500 companies, market disruptors, and current AI, machine learning, data, and cloud technologies.
- Opportunity to join a global AI-first digital transformation and engineering company.
Tech Stack
Categories
AI ApplicationsData Engineering
About Quantiphi
Quantiphi is an AI-first digital engineering and consulting firm that designs and implements machine learning, data, and cloud solutions for large enterprises across industries. Its teams build production systems—such as generative AI applications, computer vision, and predictive analytics—primarily on Google Cloud and other hyperscalers, delivered as professional services and managed solutions. Founded in 2013 and headquartered in Marlborough, Massachusetts, the company is privately held and a Google Cloud partner.