1 day ago
Bengaluru, IndiaSenior
Responsibilities
- Design, develop, test, and support data pipelines and applications on Palantir Foundry.
- Configure and customize Workshop workflows and ontologies.
- Ingest data from external source systems using data connections and synchronization.
- Build ETL/ELT pipelines for data migration, refinement, and data quality checks.
- Optimize Spark jobs, application performance, and code efficiency.
- Manage structured and unstructured data and scalable architectures for large datasets.
- Collaborate with stakeholders and technical teams on requirements, design, deployment, and troubleshooting.
- Implement data loading and change-data-capture strategies.
- Assist junior team members with efficiency and productivity improvements.
- Apply code readability, maintainability, and quality best practices.
Requirements
- Minimum five years of professional experience is required; the description also specifies minimum six years for the Lead Data Engineer role.
- Completion of 15 years of full-time education is required.
- Proficiency in PySpark, Python, and SQL, including writing and optimizing SQL and Spark jobs.
- Hands-on experience with Palantir Foundry services including Data Connection, Code Repository, Contour, Data Lineage, and Health Checks.
- Experience with Workshop, ontology, Slate, data pipelines, data ingestion, data refinement, and data quality checks in Palantir Foundry.
- Knowledge of Spark architecture, performance tuning, code optimization, scalable architecture, large datasets, and change-data-capture strategies.
- Experience handling CSV, JSON, Parquet, and ORC file formats and both structured and unstructured data.
- Experience with Git and hosting services such as Azure DevOps, GitHub, Bitbucket, and GitLab.
- Organizational, project management, stakeholder communication, and junior-team support experience are beneficial.
- Test-driven development and CI/CD workflow experience are nice to have.
Tech Stack
Categories
Data Engineering
About Accenture
Accenture is a global professional services firm providing management consulting, systems integration and technology, cybersecurity, and business process outsourcing for enterprises and governments. It operates a services-driven model delivering projects and managed services, often with major cloud and software partners, across industries worldwide. Headquartered in Dublin and publicly traded on the NYSE (ACN), it originated as Andersen Consulting and adopted the Accenture name in 2001.