5 days ago
Responsibilities
- Lead the architecture, design, deployment, and scaling of high-throughput Big Data systems.
- Build and manage vector, NoSQL, and document database systems supporting AI infrastructure.
- Develop end-to-end ETL/ELT pipelines, API services, data ingestion frameworks, and feature-rich context pipelines.
- Design storage and processing layers including data lakes, data warehouses, distributed file systems, and real-time streaming architectures.
- Optimize distributed queries and data transformations for performance and low latency.
- Implement data quality, integrity, reliability, and governance frameworks across data assets.
- Collaborate with analytics, product, and platform teams to develop semantic data models for customer metrics, hierarchies, and relationships.
- Evaluate modern data stack tools, distributed computing frameworks, and database technologies.
- Lead technical projects and mentor engineering teams.
Requirements
- Extensive data engineering experience with hands-on delivery in complex data environments.
- Deep practical knowledge of vector databases, NoSQL databases, and document stores.
- Experience building, scaling, and shipping large-scale production data platforms.
- Deep experience with distributed data processing frameworks such as Apache Spark, Flink, or Hadoop.
- Strong expertise with message brokers and event-streaming platforms such as Apache Kafka or Kinesis.
- Extensive experience with workflow orchestration tools such as Apache Airflow or Dagster.
- Hands-on expertise with cloud data warehouses such as Snowflake, BigQuery, or Redshift.
- Experience with data lake architectures and technologies such as Databricks, Delta Lake, or Apache Iceberg.
- Advanced SQL skills and proficiency in Python.
- Strong background in software development practices including testing, code review, CI/CD, and Infrastructure as Code.
- Experience with cloud-native infrastructure such as AWS, GCP, or Azure is desirable.
- Experience implementing data observability, monitoring, and alerting frameworks at scale is desirable.
- Familiarity with Anaplan or similar enterprise planning platforms is desirable.
Tech Stack
Amazon RedshiftApache AirflowApache FlinkApache HadoopApache KafkaApache SparkAWSAzureDatabricksGoogle BigQueryGoogle Cloud PlatformPythonSnowflakeSQL
Categories
Data Engineering
About Anaplan
Anaplan is a leading AI-driven scenario planning and analysis platform designed to optimize decision-making in today’s complex business environment so that enterprises can outpace their competition and the market. By building connections and collaboration across organizational silos, our platform intelligently surfaces key insights — so businesses can make the right decisions, right now. More than 2,500 global brands plan with Anaplan.