4 months ago
Base Salary
$225k - $275k/yr
Responsibilities
- Design and build scalable pipelines to ingest, transform, and deliver structured and unstructured data to SchonAI using Prefect.
- Develop ETL/ELT processes and real-time and batch workflows for market data, research documents, internal databases, and third-party APIs.
- Build and maintain AI/ML data infrastructure, including vector databases, semantic search systems, schemas, and storage solutions for LLM applications.
- Implement data versioning, lineage tracking, observability, access controls, encryption, governance, and compliance measures.
- Optimize pipeline performance, reliability, cost efficiency, low-latency AI interactions, and high-throughput batch processing.
- Integrate with risk platforms, trading systems, portfolio management tools, and research databases.
- Partner with AI engineers, software developers, data scientists, infrastructure teams, and business stakeholders to prioritize data needs and enhancements.
- Document data flows, transformations, dependencies, and pipeline operations.
Requirements
- Bachelor's or Master's degree in Computer Science, Data Engineering, or a related technical field.
- 5+ years of experience building production data pipelines with tools such as Apache Airflow, Prefect, or Dagster.
- Strong Python proficiency, SQL experience, and experience with at least one additional language such as Java, Scala, Go, or Rust.
- Hands-on experience with distributed computing frameworks such as Spark or Flink and modern data platforms.
- Proficiency with AWS services such as S3 and Kubernetes, or equivalent GCP services.
- Experience with SQL databases including PostgreSQL and MySQL and NoSQL databases including MongoDB, DynamoDB, and Elasticsearch.
- Understanding of AI/ML data requirements, including vector databases such as Pinecone, Weaviate, or Qdrant and embedding pipelines.
- Preferred experience with LLM applications, RAG systems, financial data sources, streaming technologies, OLAP or warehouse databases, Docker, Kubernetes, MLOps, and data quality frameworks.
- Strong problem-solving, communication, attention-to-detail, collaboration, and prioritization skills.
Benefits
- Learning and educational offerings, opportunities to make an impact, internal networks, external partnerships, and service initiatives promoting inclusion and purpose.
- The role may be eligible for a performance bonus and competitive benefits package.
Tech Stack
Amazon DynamoDBAmazon RedshiftApache AirflowApache FlinkApache KafkaApache SparkAWSClickHouseDockerElasticsearchGoGoogle Cloud PlatformJavaKubernetesMongoDBMySQLPostgreSQLPythonRustScalaSingleStoreSQL
Categories
Data Engineering
About Schonfeld
Schonfeld is a privately held, multi-manager hedge fund that allocates capital to internal and external portfolio managers across quantitative, fundamental equity, tactical trading, and discretionary macro/fixed-income strategies. The firm manages money for institutional and other sophisticated investors and supports teams with proprietary technology, infrastructure, and risk analytics. Founded in 1988 and headquartered in New York, it operates globally with an investing footprint spanning five continents.