Data Engineer
American Bureau of Shipping1 month ago
Knoxville, TN, USASenior
Responsibilities
- Design, build, maintain, and optimize scalable data pipelines, data models, lakehouse structures, and data warehouse architectures.
- Develop and support ETL/ELT workflows for structured, semi-structured, and unstructured data from multiple sources.
- Prepare and manage datasets for feature engineering, model training, inference, and other AI/ML applications.
- Implement data quality checks, validation, monitoring, observability, and performance optimization practices.
- Manage data integration across internal and external systems, including batch and near-real-time data flows.
- Support data governance through lineage, cataloging, metadata management, access management, and documentation.
- Develop and deploy cloud-based data engineering solutions using distributed processing and containerized environments.
- Collaborate with technical and business teams to translate requirements into enterprise data solutions.
- Troubleshoot and continuously improve data infrastructure, workflows, and related tools.
- Comply with ABS Health, Safety, Quality, and Environmental Management System requirements and internal policies.
Requirements
- Technical degree or equivalent combination of education and experience.
- 8+ years of relevant experience in data engineering or a closely related discipline.
- Significant experience building and maintaining cloud-based data pipelines and architectures.
- Strong knowledge of data modeling, data integration, ETL/ELT design, and pipeline orchestration.
- Strong Python and SQL programming skills for production-grade data workflows.
- Experience with cloud-based data platforms such as AWS, Azure, or Google Cloud.
- Experience with data warehouse or lakehouse architectures such as Databricks, Snowflake, or Redshift.
- Knowledge of distributed data processing frameworks such as Spark.
- Experience implementing data quality, validation, monitoring, and performance optimization practices.
- Knowledge of data governance, lineage, metadata management, and cataloging.
- Familiarity with data infrastructure supporting AI/ML workflows, feature engineering, data preparation, and embedding pipelines.
- Preferred experience with real-time or streaming technologies such as Kafka or Kinesis.
- Preferred familiarity with Docker, Kubernetes, or similar containerization and deployment tools.
- Databricks certification is preferred.
- Ability to solve complex data problems, collaborate across technical and business teams, manage priorities, meet deadlines, and maintain quality.
- Working knowledge of the ABS Health, Safety, Quality, and Environmental Management System.
Tech Stack
Amazon RedshiftApache KafkaApache SparkAWSAzureDatabricksDockerGoogle CloudKubernetesPythonSnowflakeSQL
Categories
Data Engineering
About American Bureau of Shipping
American Bureau of Shipping (ABS) is a nonprofit classification society that sets technical standards and provides surveys, certification, and advisory services for ships, offshore units, and marine energy projects. It earns fees from classification, compliance audits, and engineering consulting for shipowners, shipyards, and energy companies. Founded in 1862 and headquartered in Spring, Texas, ABS operates worldwide and supports areas like cybersecurity, LNG, and condition-based monitoring.