24 days ago
Canberra, Australia or Melbourne, AustraliaMid Level
Responsibilities
- Build and support real-time pipelines using Amazon Kinesis Data Streams and Kinesis Data Firehose.
- Develop streaming solutions in Databricks using Spark Structured Streaming.
- Ingest and manage streaming data in Delta tables.
- Optimise pipelines for performance, latency, reliability and cost.
- Build monitoring and alerting using AWS CloudWatch.
- Integrate streaming and batch pipelines with batch data engineers.
- Develop clean, testable Python and SQL code.
- Troubleshoot pipeline and data-processing issues and contribute to continuous improvement.
Requirements
- Production experience with Amazon Kinesis Data Streams and/or Kinesis Data Firehose is essential.
- Strong Python and SQL skills are required.
- Experience with Apache Spark Structured Streaming is required.
- Hands-on Databricks experience, ideally including Delta Lake and Unity Catalog.
- Good knowledge of AWS services, particularly IAM, S3 and CloudWatch.
- Understanding of scalable and reliable data-pipeline design.
- Strong troubleshooting and problem-solving skills.
- Experience with Kafka is beneficial.
- Australian citizenship and an active security clearance are required.
Benefits
- 18 weeks of paid parental leave.
- Long- and short-term career break opportunities.
- Structured career development program.
- Local and international career opportunities.
- Flexible work arrangements.
- Certified Family Inclusive Workplace.
- The role is based in Australia and requires Australian citizenship and an active security clearance.
Tech Stack
Categories
Data Engineering
About Accenture
Accenture is a global professional services firm providing management consulting, systems integration and technology, cybersecurity, and business process outsourcing for enterprises and governments. It operates a services-driven model delivering projects and managed services, often with major cloud and software partners, across industries worldwide. Headquartered in Dublin and publicly traded on the NYSE (ACN), it originated as Andersen Consulting and adopted the Accenture name in 2001.