Senior Data Engineer
Tiger Analytics Inc.29 days ago
Remote, United States or Jersey City, NJ, USASenior
Responsibilities
- Design, develop, and maintain scalable data pipelines using Azure data services, Python, Scala, and Apache Spark.
- Develop and optimize ETL/ELT processes for data ingestion, transformation, and processing across diverse data sources.
- Build and manage pipelines using Azure Data Factory, Azure Databricks, and Azure Data Lake Storage.
- Develop and optimize Spark applications for processing performance, scalability, and reliability.
- Work with Hadoop and related big data technologies.
- Monitor pipeline execution, troubleshoot failures, and resolve data processing issues.
- Implement data validation, error handling, and data quality checks.
- Collaborate with data architects, analysts, and cross-functional teams to deliver reliable data solutions.
- Follow coding standards, testing practices, version control, and CI/CD procedures.
- Support production deployments, incident resolution, performance tuning, and ongoing maintenance.
Requirements
- Bachelor’s degree in Computer Science, Information Technology, Computer Engineering, or a related field.
- 10+ years of overall data engineering experience.
- Strong hands-on experience with Microsoft Azure data services, particularly Azure Data Factory, Azure Databricks, and Azure Data Lake Storage.
- Proficiency in Python and Scala for data processing and application development.
- Strong experience with Apache Spark and distributed data processing.
- Hands-on experience with Hadoop and big data technologies.
- Strong understanding of ETL/ELT processes, data transformations, and data pipeline architecture.
- Strong SQL skills and experience with performance tuning, debugging, troubleshooting, and production support.
- Experience with Control-M for batch scheduling, job monitoring, workflow orchestration, and dependency management.
- Familiarity with version control, CI/CD, and deployment practices.
- Excellent analytical, problem-solving, and collaboration skills.
Benefits
- Remote workplace arrangement.
- Medical, dental, and vision benefits.
- Retirement accounts.
- Long- and short-term disability insurance.
- Life insurance.
- HSA/FSA accounts for USA employees.
- Discretionary time off.
- Continuous learning, AI upskilling, and exposure to emerging technologies.
- Opportunities to work on enterprise-wide AI adoption, cloud-native data modernization, and strategic client engagements.
Tech Stack
Categories
Data Engineering
About Tiger Analytics Inc.
Tiger Analytics Inc. provides data and AI consulting services that deliver machine learning, analytics, and cloud data engineering solutions for large enterprises across industries, including life sciences. The company engages on strategy and implementation projects, from modern BI and data platforms to MLOps, helping Fortune 100 clients turn data into business outcomes. Founded in 2011 and headquartered in Santa Clara, it is privately held with operations across North America, Europe, and Asia-Pacific.