2 days ago
Bengaluru, IndiaStaff+
Responsibilities
- Define the 3–5 year technical roadmap and architecture for the enterprise data lakehouse.
- Build internal SDKs, CLI tools, control planes, custom operators, and automated orchestration frameworks.
- Develop high-performance systems, abstraction layers, APIs, and self-service capabilities for data engineers.
- Prototype and benchmark emerging data technologies and specialized Spark extensions.
- Automate data lineage, PII masking, governance, and fine-grained access control across petabyte-scale environments.
- Lead architecture reviews and serve as the escalation point for complex outages and performance bottlenecks.
- Influence technology adoption, engineering standards, hiring expectations, and professional development across the data engineering organization.
- Architect infrastructure supporting AI agent development, including vector database integrations.
Requirements
- 15+ years of software engineering and distributed-systems experience, including at least 4 years leading platform-scale initiatives at a principal or staff level.
- Strong foundations in software engineering, system architecture, algorithms, data structures, and scalable production applications.
- Professional experience with Java/J2EE, Spring Boot, and Python, with an emphasis on clean, testable, high-performance code.
- Experience building data platforms, framework engines, and APIs that power data movement rather than only ETL/ELT pipelines.
- Deep knowledge of Apache Iceberg, Hudi, or Delta Lake, including metadata management, manifest files, and compaction strategies.
- Experience contributing to or deeply customizing open-source data projects such as Spark or dbt.
- Extensive cloud architecture experience with AWS and Azure, including services such as S3, EMR, Kubernetes, and Lambda.
- Experience with development automation, Git Actions, Terraform, and modern development tools.
- Experience using AI code-generation platforms for code generation, reviews, unit tests, and root-cause analysis.
- Ability to architect platform infrastructure for AI agent development and integrate vector databases.
- Demonstrated ability to lead by influence across autonomous teams and communicate architectural trade-offs to technical and executive audiences.
- Bachelor’s or master’s degree in computer science with a distributed-systems focus is preferred, or equivalent deep industry experience.
Tech Stack
Categories
BackendData Engineering
About WEX
WEX is a public financial technology company that provides corporate payment solutions to fleets, healthcare/benefits administrators, and travel businesses. Its products include fuel and mobility cards, virtual card payments, and platforms for HSAs/FSAs/HRAs, with revenue from payment processing and software/service fees. Founded in 1983 and headquartered in Portland, Maine, WEX trades on the NYSE under WEX and was formerly known as Wright Express.