Senior Data Engineer
Credence Management Solutions, LLC1 month ago
Boston, MA, USA +2 moreSenior
Base Salary
$115k - $155k/yr
Responsibilities
- Build and optimize data pipelines that prepare, clean, and structure data for generative AI and LLM applications.
- Manage cloud-based data lakes and warehouses across AWS, Azure, and GCP using medallion architecture and data quality, lineage, and accessibility practices.
- Model, query, load, monitor, and tune large-scale SQL and NoSQL data stores.
- Collaborate with data engineers, software engineers, data scientists, and stakeholders to develop operational agentic AI systems.
- Automate AI model deployment workflows using infrastructure as code, CI/CD pipelines, and container orchestration tools.
- Monitor deployed AI systems, perform performance tuning, and improve reliability and scalability.
- Write clean, documented, reproducible code in accordance with industry and federal guidelines.
- Participate in technical design reviews, mentorship, and ongoing learning related to AI, machine learning, and data engineering.
Requirements
- U.S. citizenship and eligibility for a DoD Secret clearance are required.
- A bachelor’s or master’s degree in Computer Science, Data/AI/ML, or a related field is required.
- At least 7 years of hands-on experience delivering Data/AI/ML solutions is required.
- Experience with ETL, ELT, enterprise data and storage systems, and tools such as OpenSearch/Elasticsearch, Kafka, Bedrock, Glue, Databricks, Snowflake, Amazon S3, RDS, EBS, or Glacier is required.
- Experience with vector databases, embeddings, related data structures, file formats, services, and APIs is required, including technologies such as FAISS, PGVector, OpenSearch/Elasticsearch, Hugging Face, Pinecone, or Bedrock Knowledge Base.
- Experience with generative AI, LLMs, tool calls such as MCP, and agents such as A2A is required.
- Understanding of AI APIs including OpenAI, Anthropic, Gemini, Bedrock, and Vertex for LLM and RAG search use cases is required.
- Familiarity with CI/CD tools such as GitLab, GitHub, or Jenkins is required.
- Experience with VS Code and AI extensions such as Cline and Claude Code is required.
- Strong communication skills and a client-oriented mindset are required.
- Preferred qualifications include Python proficiency and experience with PySpark, Pandas, uv, Pydantic, FastAPI, CrewAI, LangChain, LangGraph, and Unstructured.
- Preferred qualifications include experience with Terraform, OpenTofu, AWS CDK, or CloudFormation.
- Preferred qualifications include experience with Agent2Agent Protocol, AWS Bedrock Agents, Mastra, CrewAI, Strands, or AgentCore.
- Exposure to data science, UI/UX, cloud engineering, and platform engineering is preferred.
- Knowledge of federal cybersecurity, RMF, FedRAMP, or regulatory frameworks is preferred.
Benefits
- Full-time, on-site position in McLean, United States.
- Health care coverage including medical, dental, and vision.
- Retirement plans including 401(k) and IRA options.
- Basic, voluntary, and AD&D life insurance.
- Paid vacation, sick leave, and public holidays.
- Maternity and paternity family leave.
- Short-term and long-term disability coverage.
- Training and development opportunities.
- Wellness resources.
Tech Stack
Apache KafkaAWSAzureDatabricksElasticsearchFastAPIGoogle Cloud PlatformJenkinsPandasPythonSnowflakeSQLTerraform
Categories
Data Engineering
About Credence Management Solutions, LLC
Credence Management Solutions, LLC provides IT consulting, engineering, and mission support to U.S. federal agencies, with emphasis on defense, intelligence, health, and humanitarian programs. It builds and operates secure AI, cloud, cyber, and DevSecOps solutions and delivers program management and systems engineering under government contracts. The privately held firm is headquartered in McLean, Virginia, and supports customers such as the U.S. Air Force at Wright-Patterson Air Force Base.