9 days ago
Sydney, AustraliaMid Level
Base Salary
$80k - $90k/yr
Responsibilities
- Build and maintain scalable cloud data pipelines for ingestion, transformation, and storage.
- Integrate customer data from multiple sources into unified profiles.
- Create metadata-driven frameworks for data quality, consistency, governance, and lineage.
- Write PySpark and Spark SQL scripts for complex, efficient, and scalable data transformations.
- Fix ETL errors, improve resiliency, and maintain data integrity across reports and dashboards.
- Monitor data platforms, respond to alerts, perform root cause analysis, and optimize efficiency and cost.
- Coordinate design reviews, implementation walkthroughs, production deployments, and delivery activities.
- Collaborate with stakeholders, architects, onsite teams, and offshore teams to capture requirements and deliver solutions.
- Apply security, compliance, and governance practices to data platforms.
- Use Jira, Confluence, and Bitbucket for project management, documentation, and collaboration.
- Lead projects through planning, prioritization, and team coordination.
Requirements
- Expertise in data engineering with Azure Data Lake Storage, Azure Blob Storage, Hadoop, HDFS, Hive, PySpark, and SQL Server.
- Experience with Azure and AWS cloud platforms.
- Proficiency with Python, SQL, Shell Scripting, Git, Bitbucket, PyCharm, and VS Code.
- Experience with Control-M and Sterling B2B Integrator for automation and scheduling.
- Knowledge of Postman and API testing.
- Ability to monitor platforms, analyze Spark logs, and perform root cause analysis.
- Understanding of security best practices, compliance requirements, and data governance.
- Banking or BFSI domain knowledge.
- Databricks Certified Data Engineer Associate certification.
Tech Stack
Categories
Data Engineering
About Cognizant
Cognizant is a public IT services and consulting firm that designs, builds, and runs enterprise technology, including digital engineering, cloud modernization, data/AI, and managed services. It sells consulting, systems integration, and outsourcing on multi-year engagements to large enterprises in healthcare, banking, retail, communications, and manufacturing. Founded in 1994 and headquartered in Teaneck, New Jersey, Cognizant is NASDAQ-listed (CTSH) and a Fortune 500 company.