
Senior Backend AI Engineer
BrightEdge6 months ago
Responsibilities
- Architect Python-based agentic pipelines using LLMs and AI agents to discover, parse, and map unstructured data.
- Develop self-healing infrastructure that diagnoses upstream failures, adapts to API schema changes, and performs real-time data validation.
- Design and manage low-latency data flows into ClickHouse and BigQuery using high-performance OLAP architecture.
- Build scalable integrations for APIs, streaming sources, and S3 that handle diverse, non-standardized data.
- Lead AI-assisted testing and observability practices and mentor the engineering team.
- Own high-impact initiatives for the global data platform.
Requirements
- 5+ years of experience and a bachelor’s degree in Computer Science or Software Engineering.
- Deep production-level Python expertise and experience integrating AI models or LLMs into backend services or data workflows.
- Proven experience with ClickHouse, BigQuery, or Snowflake, including schema optimization and materialized views for large-scale datasets.
- Strong distributed systems experience in GCP or AWS and with real-time streaming tools such as Kafka, Flink, or Spark.
- Experience leading high-impact initiatives and owning major platform work.
Tech Stack
Apache FlinkApache KafkaApache SparkAWSClickHouseGoogle BigQueryGoogle Cloud PlatformPythonSnowflake