8 months ago
Base Salary
$230k - $265k/yr
Responsibilities
- Design and build scalable data pipelines and lakehouse infrastructure using PySpark, Databricks, Airflow, and AWS.
- Create abstractions, self-service tooling, and documentation to improve the data platform experience for Engineering, Data Science, and Product teams.
- Own core data pipelines and models supporting internal dashboards, ML models, and customer-facing products.
- Administer Databricks workspaces, including access management, performance monitoring, configuration optimization, and pipeline availability.
- Lead initiatives involving data quality, testing, observability, cost efficiency, ingestion scaling, data lineage, and migrations such as Databricks SQL pipelines to dbt.
- Serve as the primary engineering partner for Data Science and collaborate with backend engineers on performant data models and product development.
- Share best practices and mentor engineers working on data-centric systems.
Requirements
- At least 4 years of software engineering experience with a strong background in data infrastructure, pipelines, and distributed systems.
- Advanced proficiency in Python and SQL.
- Hands-on Spark development experience.
- Expertise with AWS, including S3 and RDS, Databricks, Airflow, and lakehouse architectures.
- Hands-on experience with Hadoop, Hive, Kafka or similar streaming platforms, Delta Lake or Iceberg, and Trino or Presto.
- Familiarity with ingestion frameworks, developer-experience tooling, data versioning, lineage, partitioning, and clustering.
- Strong problem-solving skills, ownership mindset, and communication and collaboration abilities.
- Experience with AWS infrastructure using Terraform, observability tools such as Datadog, cloud cost tracking, financial systems, ML infrastructure, feature stores such as Tecton, or internal and open-source data tooling is beneficial.
Benefits
- Medical, dental, and vision insurance
- Work-from-home flexibility
- Unlimited PTO
- Commuter benefits
- Free lunches
- Paid parental leave
- 401(k)
- Employee assistance program
- Equity grant
Tech Stack
Apache AirflowApache HadoopApache HiveApache KafkaApache SparkAWSDatabricksDatadogdbtPrestoPythonSQLTerraform
Categories
Data EngineeringDevOps
About Parafin
Parafin builds embedded financial infrastructure for marketplaces, vertical SaaS, and payment platforms to offer small-business sellers capital, spend management, and savings tools via a single integration. It underwrites, funds, services, and manages compliance on behalf of its partners, effectively providing white-label financing programs. Founded in 2020 and headquartered in San Francisco, Parafin is privately held, Series C–stage, and counts DoorDash, Amazon, Worldpay, and Mindbody among its distribution partners.