1 day ago
Sydney, AustraliaSenior
Responsibilities
- Design, develop, and maintain scalable data pipelines using PySpark, Spark SQL, Python, SQL, and Azure Synapse Analytics.
- Build and optimize ETL/ELT workflows for structured, semi-structured, and unstructured data.
- Develop and support Azure Synapse pipelines, notebooks, SQL scripts, stored procedures, views, and transformation logic.
- Work with Azure Data Lake Storage Gen2, Azure Data Factory, Azure Databricks, Azure SQL Database, and related Azure data services.
- Develop and support Big Data solutions using Apache Spark, Hadoop, HDFS, Hive, Sqoop, Kafka, HBase, and related distributed-processing components.
- Translate business requirements and functional specifications into technical designs and reusable data-engineering components.
- Implement data ingestion, transformation, validation, reconciliation, and publishing processes.
- Tune PySpark, SQL, Big Data jobs, distributed workloads, Spark configurations, partitions, memory usage, storage formats, and cluster resources.
- Develop frameworks for metadata-driven ingestion, incremental loads, error handling, audit logging, and data quality checks.
- Support unit testing, system testing, UAT, deployment, production monitoring, incident resolution, and post-production support.
- Collaborate with architects, business analysts, QA teams, DevOps teams, platform teams, and client stakeholders.
- Ensure compliance with data governance, security, lineage, audit, access-control, and operational standards.
- Mentor junior data engineers, review code, provide technical guidance, and promote engineering best practices.
Requirements
- Experience with data engineering, Big Data platforms, distributed data processing, cloud data solutions, and enterprise data processing.
- Hands-on experience with PySpark, Spark SQL, Python, SQL, and data pipeline development.
- Hands-on experience with Big Data ecosystem tools such as Spark, Hadoop, HDFS, Hive, Kafka, Sqoop, or HBase.
- At least 3 years of practical experience with Azure Synapse Analytics and Azure cloud data services.
- Experience designing, developing, testing, deploying, and supporting production-grade data engineering solutions.
- Ability to independently own technical design, development, defect resolution, deployment, and production support activities.
Benefits
- Opportunities for career development, innovative career tracks, education, and dedicated training.
- Compensation, benefits, and wellness packages are provided.
- Collaborative work with experienced technology professionals on emerging industry trends and cutting-edge technologies.
Tech Stack
Categories
Data Engineering
About Cognizant
Cognizant is a public IT services and consulting firm that designs, builds, and runs enterprise technology, including digital engineering, cloud modernization, data/AI, and managed services. It sells consulting, systems integration, and outsourcing on multi-year engagements to large enterprises in healthcare, banking, retail, communications, and manufacturing. Founded in 1994 and headquartered in Teaneck, New Jersey, Cognizant is NASDAQ-listed (CTSH) and a Fortune 500 company.