Senior Software Engineer - Platform
Cognite - AI for Industry2 months ago
Bengaluru, IndiaSenior
Responsibilities
- Design, build, and operate the core serverless execution engine and Workflows orchestration layer supporting AI and automation capabilities.
- Own uptime, latency SLOs, incident response, deterministic function execution, and reliable workflow progression without data loss or silent failures.
- Architect multi-tenant and multi-cloud systems for high-throughput workloads, including scheduling, queueing, and retry mechanisms.
- Define and evolve versioned API-first and event-driven services for internal engineering teams and external customers.
- Implement distributed tracing, structured logging, alerting, and observability using OpenTelemetry, Prometheus, Grafana, and Honeycomb.
- Develop unit, integration, and smoke-test automation and maintain production deployment pipelines.
- Profile and resolve execution-throughput, cold-start, and cross-service performance bottlenecks.
- Model compute and storage costs and implement optimizations that reduce cloud spend without sacrificing reliability.
- Build platform primitives for ML compute scheduling, environment management, secrets handling, and reliable training, fine-tuning, and batch inference workflows.
- Support long-running, GPU-aware, data-intensive contextualization workflows and related vector or embedding infrastructure.
Requirements
- 6–8 years of experience building and operating production backend services at scale.
- Deep expertise in JVM languages, preferably Kotlin, with Java acceptable, plus Python and FastAPI.
- Experience with distributed systems patterns and cloud-native service design across Kubernetes, Azure, GCP, AWS, and private cloud environments.
- Hands-on experience with workflow engines such as Conductor or Apache Airflow and event-driven architectures using Kafka or Pub/Sub.
- Experience with relational and non-relational databases, object storage, data lakes, and caching layers such as Redis in multi-tenant environments.
- Practical experience with OpenTelemetry, Prometheus, and Grafana for instrumentation and operational insight.
- Experience supporting production ML workloads and notebooks through scheduling, resource management, experiment tracking, or model-serving infrastructure.
- Familiarity with industrial knowledge graphs, entity resolution, NLP/CV pipelines, React, or TypeScript is beneficial.
- Familiarity with vector embedding storage or serving, semantic search, RAG use cases, model versioning, and A/B experiment tracking is beneficial.
Tech Stack
Apache AirflowApache KafkaAWSAzureFastAPIGoogle Cloud PlatformGrafanaJavaKotlinKubernetesPostgreSQLPrometheusPythonReactRedisTypeScript
Categories
About Cognite - AI for Industry
Cognite builds an industrial data and AI platform used by energy, manufacturing, and power & utilities companies to integrate OT/IT data and deploy AI at scale. Its flagship product, Cognite Data Fusion, is sold as enterprise SaaS with services for implementation and Industrial AI agents. Founded in 2016 and privately held, the company focuses on asset-intensive operations and Industrial DataOps.