2 months ago
Helsinki, FinlandSenior
Responsibilities
- Define technical designs, measurement frameworks, evaluation systems, datasets, correctness criteria, and regression gates for AI products.
- Run continuous end-user evaluations of APIs, SDKs, data representations, and reference applications, then identify and drive fixes for gaps.
- Own MVP delivery against a technical design while balancing technical quality with useful incremental releases.
- Build ingestion, conflation, entity resolution, quality-check, batch-processing, and streaming data pipelines.
- Evaluate external datasets, models, benchmarks, research, and open-source releases and decide whether to adopt or build.
- Design feedback loops, failure instrumentation, and recurring evaluation cases.
- Design model-tool boundaries and build or improve model harnesses for tool calling and agent orchestration.
- Optimize request latency and cost through streaming, partial results, caching, model routing, and prompt structure.
- Build internal CLI, MCP, and related engineering tools and share reusable components across teams.
- Participate in code and design reviews, mentor engineers on evaluation practices, and contribute to unresolved technical questions.
- Participate in a 24/7 on-call rotation, including possible immediate responses outside normal working hours and on weekends.
Requirements
- Bachelor’s degree in a STEM discipline and 5+ years of software engineering experience with production ownership of services, pipelines, or SDKs.
- 2+ years of experience shipping LLM-backed features to real users in systems with error budgets, on-call rotations, and customer-visible regressions.
- Direct experience or deep understanding of evaluation design for non-deterministic systems, including building datasets and identifying failures.
- Data engineering depth with SQL, at least one distributed processing framework, and production pipelines where data correctness is critical.
- Fluency with tool calling and agent orchestration, including stale context, hallucinated arguments, silent partial success, and unbounded-loop failure modes.
- Working knowledge of more than one agent harness.
- Strong Python or TypeScript skills and the ability to read code in other languages.
- Experience diagnosing latency in distributed request paths.
- Clear written communication and comfort working through ambiguity.
- Preferred experience includes geospatial data, routing, geocoding, POI or address data, OpenStreetMap, public API or SDK design, MCP or similar tool transports, CI evaluations, automotive platforms, voice pipelines, constrained compute or offline systems, and first external integrations.
Benefits
- Hybrid or remote work options, including working from home or periodically from an applicable office.
- Country-specific private health, dental, and income protection coverage.
- Family care support and maternity and paternity leave.
- Lifestyle spending account contributions for health, wellness, and personal growth.
- Mental health support for employees and dependents.
- Paid time away, company holidays, and generous absence policies.
- Dedicated paid volunteering time in addition to standard paid time off.
Tech Stack
Categories
AI ApplicationsData Engineering
About Mapbox
Mapbox builds a real-time location platform with APIs, SDKs, and data for maps, navigation, search, and traffic used by developers and enterprises across mobile, web, and automotive. It sells usage-based SaaS for custom maps, routing, and telemetry that support logistics, e-commerce, and fleet operations. Founded in 2011 and headquartered in Washington, DC, the privately held company serves millions of developers and powers navigation for people, packages, and vehicles.