10 hours ago
Sydney, AustraliaMid Level
Base Salary
$80k - $90k/yr
Responsibilities
- Design, develop, and optimize scalable data ingestion, transformation, aggregation, and processing pipelines using PySpark and Spark SQL.
- Build and maintain frameworks leveraging Spark DataFrames, Hive, HDFS, ADLS2, and Azure data platforms.
- Develop complex Spark SQL and HiveQL queries and perform Spark performance tuning.
- Investigate production issues through log analysis and root-cause identification, and provide production and release support.
- Configure, schedule, and manage enterprise data workflows using BMC Control-M.
- Implement monitoring, reliability improvements, change requests, deployment procedures, and operational documentation.
- Manage source code collaboration through Git and Bitbucket and support platform upgrades while maintaining service continuity.
- Collaborate with cross-functional stakeholders to deliver business-critical data engineering solutions.
Requirements
- Proficiency with Apache Spark, PySpark, Spark SQL, HDFS, ADLS2, Azure Synapse Analytics, and Databricks.
- Experience with Python, HiveQL, Hive, and MySQL for data processing and database work.
- Experience with Bitbucket, Jenkins, CI/CD pipeline implementation, Git, and enterprise workflow scheduling tools.
- Experience with ServiceNow, Spark log analysis, production support, troubleshooting, and root-cause analysis.
- Knowledge of data governance, security best practices, change management, and banking and financial-services environments.
- Listed qualifications include DP-203 Data Engineering, Data Engineering with Databricks, IBM Skill Badge – Artificial Intelligence, and AI Security & Governance by Security.
Tech Stack
Categories
Data Engineering
About Cognizant
Cognizant is a public IT services and consulting firm that designs, builds, and runs enterprise technology, including digital engineering, cloud modernization, data/AI, and managed services. It sells consulting, systems integration, and outsourcing on multi-year engagements to large enterprises in healthcare, banking, retail, communications, and manufacturing. Founded in 1994 and headquartered in Teaneck, New Jersey, Cognizant is NASDAQ-listed (CTSH) and a Fortune 500 company.
