4 months ago
Remote, IndiaSenior
Responsibilities
- Design and implement internal SDKs and self-service frameworks for autonomous data ingestion and transformation
- Create reusable patterns for batch and real-time event processing as part of a platform-engineering approach
- Tune Spark execution plans, optimize shuffle partitions, and implement auto-scaling strategies to manage Databricks Unit consumption
- Balance latency, throughput, performance, and cloud spend as data volume grows
- Implement schema-on-write validation and data contracts before data reaches the Bronze layer
- Enforce data privacy, security standards, and metadata lineage in partnership with data architects and stewards
- Champion AI-assisted development tools and mentor engineers on distributed-computing best practices
- Conduct deep-dive code reviews focused on scalability and maintainability
Requirements
- 5+ years of experience building and operating production-grade data systems at massive scale
- Deep hands-on expertise with the Databricks and Spark ecosystem, including Delta Lake, DLT, Spark UI debugging, and performance tuning
- Proven experience building production real-time or streaming architectures with Spark Structured Streaming, Kafka, or Kinesis
- Experience managing and optimizing cloud costs in a high-growth environment
- Experience building APIs, tools, or frameworks used by internal engineering teams
Benefits
- Remote-first work environment with flexibility and resources
- Generous paid parental leave
- Flexible time off
- Spending accounts
- Medical, dental, and vision insurance
- Sabbatical after 5 years
- Eligible for an annual bonus for non-sales roles
Tech Stack
Categories
BackendData Engineering
About G-P
G-P (Globalization Partners) builds a SaaS-based global employment platform with Employer of Record services that let companies hire, onboard, and pay talent in 180+ countries while handling payroll, benefits, and compliance. The remote-first, privately held company earns revenue through platform subscriptions and service fees for EOR, immigration, and compliance support. Founded in 2012, it serves organizations expanding internationally across a range of industries and sizes.