3 months ago
Remote, PeruMid Level / Senior
Responsibilities
- Design and build production data pipelines using Lakeflow Declarative Pipelines, Auto Loader, Structured Streaming, data quality expectations, and Declarative Automation Bundles.
- Architect and implement Databricks Lakehouse solutions using medallion architecture, Delta Lake, and Unity Catalog.
- Build and maintain transformation layers with DLT pipelines, PySpark notebooks, and dbt, including data quality constraints and SLAs.
- Design data and AI foundations using Unity Catalog, Feature Store, MLflow, and Model Serving for production ML, agent workflows, and AI-enabled products.
- Collaborate with product and backend engineers on data models, APIs, application data contracts, and data-to-application architectures.
- Consult with clients on data challenges, strategies, architecture, and sustainable implementation approaches.
- Work across AWS and Azure environments and provide Databricks-centered platform recommendations.
- Champion governance through access control, lineage, data quality policies, and compliance practices.
- Design architectures involving Lakebase-backed services and Databricks Apps.
- Contribute to Livefront’s Databricks practice through accelerators, internal enablement, certification goals, and partner go-to-market materials.
Requirements
- 3-5 years of data engineering experience, including at least 2 years in production Databricks environments.
- Production experience with Lakeflow Declarative Pipelines, Auto Loader, Structured Streaming, Lakeflow Jobs, and Unity Catalog, including fine-grained access control and lineage.
- Experience designing production-scale Lakehouse architectures using medallion patterns, Delta Lake table design, partitioning, Z-ordering, and query optimization.
- Working knowledge of AWS and Azure services relevant to Databricks deployments, including storage, networking, IAM, and compute; GCP familiarity is a plus.
- Experience with data pipeline testing, observability, data quality frameworks, and version-controlled deployments using Git and Declarative Automation Bundles.
- Strong proficiency in SQL and Python for clean, performant, maintainable code.
- Understanding of data modeling, schema design, and query optimization.
- Strong communication, problem-solving, organization, peer-review, and stakeholder collaboration skills.
- Experience with Kafka or Kinesis is a bonus.
- Experience with alternative cloud data platforms, healthcare or fintech, MLOps or LLMOps on Databricks, MLflow experiment tracking, model registries, Model Serving, Vector Search, Databricks Apps, or Lakebase is a bonus.
- Java, Go, or Scala experience is a bonus.
- Databricks or Apache Spark certification, including Data Engineer Associate, Professional, or Apache Spark Developer, is a bonus.
- Public speaking, open-source data project contributions, technical diagramming, or public education about data engineering are bonus qualifications.
Benefits
- Remote work from Peru.
- Collaborative, respectful, and trust-based work environment.
- Opportunities to work on high-impact products and accounts.
- Opportunities for mentoring, education, professional growth, and community contribution.
- Opportunity to help build a Databricks data practice from the ground up.
- Hiring process may include a phone interview, video interviews, and a take-home exercise with up to one week for completion.
Tech Stack
Categories
Data Engineering
About Livefront
Livefront is a digital product consultancy that designs and builds mobile, web, and AI-enabled experiences for enterprises and startups. The privately held firm, founded in 2001 and headquartered in Minneapolis, provides strategy, design, and software engineering services on a client-services model. Notable clients have included CVS, Samsung, General Mills, Optum, and startups such as HomeSpotter and Credly.