4 months ago
Hyderābād, IndiaSenior
Responsibilities
- Design, develop, maintain, and optimize scalable data solutions, pipelines, ETL/ELT processes, and data integration workflows.
- Use Databricks, PySpark, Spark, and SQL to ingest, transform, and process large-scale structured and unstructured datasets.
- Implement automated data ingestion, transformation, deployment, monitoring, and scheduling using Databricks Jobs and notebooks.
- Apply Spark performance optimization techniques including job tuning, caching, partitioning, and indexing.
- Build reliable integrations with SQL databases, APIs, cloud storage platforms, and other data sources.
- Develop data models, data dictionaries, documentation, and data quality processes.
- Implement data security, privacy, governance, and compliance measures for sensitive data.
- Own data pipeline projects from inception through deployment, managing scope, timelines, and risks.
- Collaborate with Data Architects, Data Scientists, Business SMEs, product teams, and global cross-functional teams.
- Identify and resolve complex data challenges and contribute to technical planning, estimation, testing, and reusable code development.
Requirements
- Bachelor’s degree in Computer Science, IT, or a related field, or a master’s degree, plus 5 to 8 years of relevant experience.
- Hands-on experience with Databricks, Apache Spark, PySpark, SparkSQL, workflow orchestration, and big data performance tuning.
- Proficiency with SQL and experience using data analysis and data visualization tools.
- Strong understanding of data architecture, data modeling, data warehousing, data integration, ETL, and data governance.
- Knowledge of Python or R and Python packages used for data processing and machine learning model development.
- Experience with automated data pipeline orchestration and monitoring using Databricks Jobs, Apache Airflow, or similar tools.
- Knowledge of data protection regulations and compliance requirements such as GDPR and CCPA is preferred.
- Knowledge of AWS, SageMaker, and cloud data platforms is preferred.
- A Certified Data Engineer or Data Analyst certification, particularly for Databricks or cloud environments, is preferred.
- Strong problem-solving, critical-thinking, communication, collaboration, and presentation skills.
Benefits
- The position requires a later shift and may include second- or third-shift evening or night schedules based on business requirements.
- The employer provides reasonable accommodation for individuals with disabilities during the application, interview, and employment process.
Tech Stack
Categories
Data Engineering
About Amgen
Amgen is a public biotechnology company that discovers, develops, manufactures, and sells biologic and biosimilar therapeutics for patients with serious diseases. Its portfolio and pipeline span oncology, cardiovascular disease, osteoporosis, inflammation, and rare diseases, marketed globally through direct sales and partnerships. Founded in 1980 and headquartered in Thousand Oaks, California (NASDAQ: AMGN), Amgen is a component of the Dow Jones Industrial Average and completed the acquisition of Horizon Therapeutics in 2023.