25 days ago
Base Salary
$180k - $240k/yr
Responsibilities
- Design scalable data pipelines for multi-million-file workloads, including parsing, ingestion, storage, partitioning, and indexing.
- Build fault-tolerant queue-based asynchronous processing frameworks with retry, idempotency, dead-letter handling, and elastic capacity.
- Design relational, object, search, and vector storage and indexing architectures with consistency, backfill, and reconciliation mechanisms.
- Design multi-tenant data architectures with tenant isolation, per-tenant resource controls, and workload management.
- Establish pipeline observability and service-level objectives for throughput, latency, queue depth, errors, and data quality.
- Plan and execute modernization and migration strategies while preserving production availability and data integrity.
- Produce technical design documentation, participate in architectural reviews, collaborate across engineering, product, and security, and strengthen testing, CI, code-quality, and operational-readiness standards.
Requirements
- 5+ years of experience designing, building, and operating data-intensive or distributed backend systems in production.
- Experience with large-scale data-processing pipelines and operational responsibility for them.
- Production experience with message brokers and queue-based architectures, including scaling, delivery and ordering guarantees, and failure handling.
- Strong relational database skills involving query optimization, indexing, partitioning, and zero-downtime schema migration.
- Experience with workflow-orchestration frameworks such as Temporal, Airflow, or Dagster.
- Experience with search or vector-index technologies and maintaining consistency between indexes and systems of record.
- Proficiency in Python or a comparable language with the ability to work primarily in Python.
- Experience with Prometheus, Grafana, OpenTelemetry, or comparable observability tooling.
- Experience operating containerized cloud environments using Azure or AWS, Kubernetes, and Docker.
- Strong system-design and architecture experience, including producing technical design documents.
- Demonstrated proficiency using AI tools in software development.
- Preferred: experience with document processing, text extraction, OCR, format conversion, archive or email extraction, production modernization, Terraform, multi-region deployments, data residency or sovereignty, AI/ML data infrastructure, retrieval technologies such as Solr or Qdrant, or legal technology and other regulated industries.
Benefits
- The role is based in the office three to four days per week.
- The compensation range is $180,000-$240,000 annually, with potential eligibility for an annual bonus.
- Epiq offers enterprise-wide learning and mobility opportunities, workplace flexibility, and benefits described through its benefits program.
Tech Stack
Categories
BackendData Engineering
