Core Data Engineer
Flow Tradersabout 3 hours ago
Hong Kong, Hong KongMid Level / Senior
H1B Sponsor
Responsibilities
- Deliver core data products with a focus on low-latency analytics.
- Develop and maintain data pipelines and automate manual processes.
- Define and manage project plans for data product ownership.
- Create a common processing model for data products.
- Investigate and maximize value from existing data sources.
- Explore and onboard new data sources as needed.
- Collaborate with teams to automate workflows and ensure data architecture standards.
Requirements
- Excellent programming skills in Python.
- Proficiency with Linux, SQL, and relational databases.
- Experience with real-time OLAP databases like ClickHouse and Apache Pinot.
- Familiarity with Airflow, Kafka, and GCP is highly desired.
- Experience with Pandas, Spark, and Apache Beam is a plus.
- Strong attention to detail and a focus on quality.
- Ability to manage multiple projects and operational concerns.
- Strong interpersonal skills for collaboration with technical and non-technical stakeholders.
Benefits
- Access to Flow Academy for continuous learning.
- Comprehensive health insurance coverage.
- In-house lounge with recreational facilities.
- Daily catered meals and healthy snacks.
- In-house hairdresser and massage therapist.
- Personal trainers and subsidized gym membership.
- Annual company trip and various events throughout the year.
Tech Stack
Apache AirflowApache BeamApache KafkaApache SparkClickHouseGoogle Cloud PlatformLinuxPandasPythonSQL
Categories
Data Engineering