28 days ago
Bengaluru, IndiaSenior
Responsibilities
- Develop, maintain, and optimize scalable data pipelines, workflows, and Feature Store solutions.
- Design, implement, and architect data engineering pipelines with attention to performance, scalability, storage, and processing.
- Build data ingestion, transformation, and storage solutions for diverse semi-structured and unstructured data sources.
- Implement advanced data transformations, data quality checks, data models, schemas, governance, and data management practices.
- Integrate data using AWS S3 and potentially Snowflake as a SQL data warehouse.
- Perform web crawling and scraping and work with API- and stream-based data extraction processes.
- Collaborate with technology providers, engineering teams, business teams, and stakeholders to support digital experiences and automation.
Requirements
- At least 5 years of data analysis and engineering experience.
- Bachelor’s degree in computer science, statistics, informatics, information systems, or another quantitative field.
- Working knowledge of API- or stream-based data extraction, including Salesforce API and Bulk API.
- Hands-on experience with web crawling and advanced web-crawling and scraping methods.
- Experience building end-to-end data engineering pipelines for text, tables, images, video, and audio data.
- Advanced proficiency with Python, PySpark, SQL, RDBMS, ETL/ELT, and SQL data warehouses such as Snowflake.
- Experience or familiarity with Databricks, AWS services, Kafka, Spark, Kafka Streaming, workflow automation, and root-cause analysis.
- Ability to collaborate cross-functionally, handle sensitive data ethically and accurately, solve complex technical problems, and work independently in ambiguous environments.
Benefits
- Hybrid work arrangement requiring an average of two days per week from an HPE office.
- Comprehensive health, financial, and emotional wellbeing benefits for employees and their loved ones.
- Personal and professional development programs supporting career growth and movement across divisions.
- Inclusive and accessible workplace with reasonable-accommodation support for applicants with disabilities.
Tech Stack
Categories
Data Engineering
About HPE
Hewlett Packard Enterprise (HPE) builds enterprise IT infrastructure and services, including servers, storage, networking (Aruba), and hybrid edge-to-cloud platforms like HPE GreenLake for businesses and public-sector organizations. Formed in 2015 after Hewlett-Packard’s split, HPE is headquartered in Houston, Texas and trades on the NYSE. It sells hardware, software, and subscription-based managed cloud services to help customers run workloads across data centers, colocation, and public clouds.