Senior Data Engineer
Hitachi, Ltd.28 days ago
Ho Chi Minh City, VietnamSenior
Responsibilities
- Build and maintain data pipelines within Databricks.
- Extract, transform, validate, and prepare data from multiple sources.
- Develop reusable datasets and analytical tables for Data Science projects.
- Develop data processing solutions using PySpark, SQL, Python, and Pandas.
- Translate Data Scientist and business requirements into efficient, maintainable, production-ready code.
- Support feature engineering, machine learning workflows, and model development and deployment.
- Investigate data anomalies and support root-cause analysis.
- Optimize queries and processing logic for performance and scalability.
- Collaborate with Data Scientists and engineering stakeholders on operational data sources and requirements.
- Contribute to code reviews, technical discussions, documentation, coding standards, and continuous improvement.
Requirements
- Experience developing analytical solutions using Databricks.
- Strong proficiency in PySpark and SQL, including complex query writing and optimization.
- Experience using Python and Pandas for data analysis and transformation.
- Familiarity with Airflow and Kafka.
- Experience working with large-scale datasets.
- Ability to translate business requirements into technical solutions.
- Strong problem-solving and debugging skills.
- Good communication and stakeholder engagement skills.
- Experience working in collaborative Agile teams.
- Experience with Docker and Kubernetes is a plus.
- Experience with stream-processing technologies such as Flink is a plus.
- Exposure to machine learning workflows and feature engineering is desirable.
- Experience with Delta Lake and Spark optimization is desirable.
- Familiarity with Azure Data Platform technologies is desirable.
- Knowledge of Power BI or Grafana and experience using Git/version control are desirable.
- Industry experience in engineering, rail, transportation, manufacturing, industrial, or asset-intensive sectors is desirable.
Benefits
- Industry-leading benefits, support, and services focused on holistic health and wellbeing.
- Flexible work arrangements, depending on the role and location.
- A culture offering belonging, autonomy, freedom, ownership, collaboration, and knowledge sharing.
- Equal opportunity employment and reasonable accommodation support during recruitment.
Tech Stack
Apache AirflowApache FlinkApache KafkaApache SparkDatabricksDockerGitGrafanaKubernetesPandasPythonSQL
Categories
Data Engineering
About Hitachi, Ltd.
Hitachi, Ltd. builds industrial, energy, rail, and digital systems for governments and enterprises, combining operational technology with IT, cloud, and data infrastructure services. Its portfolio spans power grids, factory automation, data centers, rail systems, and consulting delivered through units such as Hitachi Vantara, Hitachi Rail, and Hitachi Energy. Founded in 1910 and headquartered in Tokyo, it is listed on the Tokyo Stock Exchange and operates globally across critical infrastructure markets.