Senior Software Engineer
Cognite - AI for Industry4 days ago
India (city not specified)Senior / Staff+
Responsibilities
- Build high-performance data pipelines using Spark, Flink, and Kafka for terabyte-scale industrial datasets.
- Develop low-latency REST and gRPC APIs and services supporting thousands of concurrent users and sub-second response times.
- Optimize time-series, sensor, operational data, storage, retrieval, and OLAP query performance at scale.
- Design distributed processing and real-time streaming systems handling millions of events per second.
- Design and evolve cost-efficient data lake architectures using S3/GCS and Parquet/ORC.
- Build fault-tolerant and highly available systems with circuit breakers, retries, distributed tracing, monitoring, and observability.
- Drive technical innovation through research, prototyping, and open-source contributions.
- Collaborate with product, architecture, and platform teams to deliver scalable production-grade solutions.
Requirements
- 6–12 years of experience in distributed data systems, backend engineering, or platform engineering.
- Strong programming skills in Scala, Java, or Python.
- Deep expertise in Spark internals, including Catalyst optimizer, partitioning, and performance tuning.
- Hands-on experience with Flink and Kafka for streaming and batch workloads.
- Experience scaling systems to 10K+ QPS, including JVM performance, garbage collection tuning, and memory management.
- Experience building high-throughput REST/gRPC services and advanced caching with Redis or in-memory data structures.
- Strong understanding of Prometheus, Grafana, and OpenTelemetry for monitoring and observability.
- Experience with AWS, GCP, or Azure and data lake/object storage.
- Experience with large-scale data, OLAP systems, or industrial/IoT data is preferred.
- Open-source contributions or experience with industrial data/AI platforms is an advantage.
- Ability to design distributed, secure, and resilient platforms and collaborate across engineering, product, and platform teams.
Benefits
- Opportunity to build Industrial DataOps systems for global energy, manufacturing, and industrial AI customers.
- Work in a fast-growing, high-ownership environment at Cognite’s India R&D Center.
- Work on distributed systems, streaming architectures, AI-ready data platforms, and industrial-scale challenges.
- Culture supporting innovation, autonomy, continuous learning, and open-source contribution.
- Competitive compensation, equity, and benefits aligned with Cognite’s global standards.
Tech Stack
Apache FlinkApache KafkaApache SparkAWSAzureClickHouseGoogle Cloud PlatformGrafanagRPCJavaPrometheusPythonRedisScala
Categories
BackendData Engineering
About Cognite - AI for Industry
Cognite builds an industrial data and AI platform used by energy, manufacturing, and power & utilities companies to integrate OT/IT data and deploy AI at scale. Its flagship product, Cognite Data Fusion, is sold as enterprise SaaS with services for implementation and Industrial AI agents. Founded in 2016 and privately held, the company focuses on asset-intensive operations and Industrial DataOps.