about 9 hours ago
Responsibilities
- Design, develop, and maintain automated data validation and quality assurance frameworks.
- Build and implement automated integrity checks for data accuracy and reliability.
- Identify gaps in data workflows and engineer scalable automation solutions.
- Write and optimize complex SQL queries for data analysis and integrity checks.
- Transform and process large datasets using PySpark or Snowpark.
- Collaborate with cross-functional teams to define quality standards and testing strategies.
- Advise clients on automation and data engineering best practices.
- Contribute to the continuous improvement of data pipeline tooling and processes.
- Support agile delivery practices by communicating technical risks and progress.
- Perform other duties as assigned.
Requirements
- 8+ years of experience in backend engineering, automation engineering, or data quality roles.
- Strong proficiency in Python and experience with data analytics frameworks like Pandas.
- Experience with distributed data transformation frameworks such as PySpark or Snowpark.
- Strong command of SQL and data querying patterns across databases.
- Proven ability to design and implement automated data validation and quality checks.
- Solid understanding of data ingestion patterns and backend system design.
- Experience working directly with clients to provide strategic technical guidance.
- Excellent problem-solving, analytical, and communication skills.
- Nice to have: experience with MongoDB or other document-based databases.
- Nice to have: familiarity with dbt for data transformation and modeling.
- Nice to have: experience with Snowflake as a cloud data platform.
- Nice to have: knowledge of CI/CD pipelines and DevOps practices.
- Nice to have: experience with RPA for automation tasks.