Data Engineer
Southern Company10 days ago
Birmingham, AL, USASenior
Responsibilities
- Design and develop batch and real-time processing methods for structured, semi-structured, and unstructured data.
- Build, operationalize, maintain, and optimize scalable data pipelines and integrations.
- Develop release pipelines for automated build, version control, and deployment tasks.
- Create reusable templates, code libraries, semantic models, and modular data products.
- Establish production data promotion gatekeeping and guardrail processes.
- Define and enforce data quality, integrity, governance, taxonomy, and curation standards.
- Integrate data from multiple systems, databases, APIs, and business applications.
- Standardize and develop Databricks applications, data integrations, and automated business processes.
- Collaborate with data, business, and technical stakeholders to deliver business solutions.
- Provide technical guidance, knowledge sharing, and mentoring to data engineers and business users.
- Optimize performance through parallel processing and minimal data movement.
- Stay current with data technologies and share relevant insights and best practices.
Requirements
- Bachelor’s degree in Computer Science, Engineering, Machine Learning, or a related field.
- 9+ years of experience in data engineering or related roles.
- Experience with batch and real-time data processing frameworks.
- Experience with Hadoop, HIVE, Spark, and other big data technologies.
- Experience developing AI/ML solutions in Python.
- At least 4 years of experience with on-premises tools such as MSBI, Informatica, Oracle Golden Gate, and SQL Server.
- At least 2 years of experience with Microsoft Azure tools such as Data Lake, Data Factory, Databricks, Synapse, and Power BI.
- Experience with containerization using Docker or OpenShift.
- Hands-on expertise with Databricks, SQL, Python, and Apache Spark.
- Proficiency in data modeling for analytics and machine learning, data access, storage, quality tools, performance optimization, and data governance.
- Familiarity with Agile, DevOps, and CI/CD methodologies.
- Ability to lead and mentor others and communicate and collaborate effectively.
Benefits
- The role is based in Atlanta, Georgia, or Birmingham, Alabama.
- The position requires onsite work four days per week and allows one remote day per week.
Tech Stack
Apache HadoopApache HiveApache SparkAzureDatabricksDockerInformaticaMicrosoft SQL ServerOpenShiftPythonSQL
Categories
Data Engineering
About Southern Company
Southern Company is an investor-owned utility holding company headquartered in Atlanta and traded on the NYSE (SO). Through subsidiaries it provides electric service in Alabama, Georgia, and Mississippi and natural gas distribution in Georgia, Illinois, Tennessee, and Virginia, serving about 9 million customers. Its portfolio spans regulated utilities and competitive generation (Southern Power), with energy from natural gas, renewables, and nuclear (including Vogtle units in Georgia).