Software Engineer II
MiQ Digital1 month ago
Bengaluru, IndiaMid Level
Responsibilities
- Modernize data ingestion by replacing manual and brittle processes with declarative, scalable pipelines.
- Evaluate and integrate appropriate data technologies across workloads rather than relying on a single vendor.
- Improve dataset observability, discoverability, lineage, governance, and pipeline health.
- Embed automated data quality, reliability, and contract-enforced governance standards into data pipelines.
- Design storage and compute infrastructure with cost efficiency in mind.
- Ship a modern ingestion platform that enables self-service onboarding and retires legacy infrastructure.
- Contribute to MiQ’s Data and AI maturity initiatives through high-impact platform engineering work.
Requirements
- 2–4 years of professional software engineering experience, including at least 1 year building data pipelines or platform infrastructure.
- Strong proficiency in Python and SQL, with production experience using PySpark or Spark SQL.
- Understanding of distributed computing fundamentals, including Spark execution, partitioning, shuffles, joins, and memory management.
- Experience with a lakehouse or data warehouse platform such as Databricks, Snowflake, BigQuery, or an equivalent.
- Working knowledge of Delta Lake or Apache Iceberg, including table formats, schema evolution, time travel, and merge operations.
- Familiarity with AWS infrastructure, including S3, IAM, and EC2/EMR basics.
- Experience with Git, CI/CD pipelines, and collaborative software development workflows.
- Bachelor’s or Master’s degree in Computer Science, Engineering, or a related discipline.
- Preferred experience with Databricks features, data quality frameworks, Unity Catalog or metadata management, event-driven or streaming architectures, and Java, Kotlin, or Scala.
Benefits
- Hybrid work environment.
- New hire orientation with job-specific onboarding and training.
- Internal and global mobility opportunities.
- Competitive healthcare benefits.
- Bonus and performance incentives.
- Generous annual PTO, paid parental leave, and two additional paid days for holidays, cultural events, or inclusion initiatives.
- Employee resource groups supporting connection, action, and community initiatives.
Categories
Data Engineering