Software Engineer II
MiQ Digital20 hours ago
Bengaluru, IndiaMid Level
Responsibilities
- Build and productionize scalable data ingestion pipelines using Databricks, PySpark notebooks, SDP, Autoloader, and Delta technologies.
- Orchestrate ingestion pipelines through scheduling and ingestion services and contribute to the Unified Ingestion Service and Self Serve Ingestion vision.
- Onboard DSPs, regions, and third-party feeds while expanding reporting granularity and coverage.
- Own ingestion pipelines end to end, including partner onboarding, productionization, monitoring, cost optimization, and incident response.
- Strengthen data health, lineage, compliance, governance, release management, and incident management workflows.
- Manage platform hygiene and remediation using Bitbucket or GitHub, Terraform-managed IAM, Snyk, and Akto.
- Collaborate with internal business and technical stakeholders and external DSP, SSP, and measurement partners.
Requirements
- 2–3 years of experience in data engineering and data integration, with hands-on experience building and managing cloud data pipelines.
- Extensive hands-on experience developing PySpark notebooks, preferably on Databricks, and using advanced SQL for high-throughput data pipelines.
- Strong knowledge of Delta Lake, Delta Live Tables, Autoloader, and Unity Catalog; Hive and Spark experience is a plus.
- Understanding of data modeling, schema evolution, data transformation, data quality, reliability, and integrity.
- Experience with ingestion through APIs, S3, SFTP/FTP, email drops, event streams, and CDC.
- Working knowledge of SDLC, release management, incident management, Bitbucket or GitHub, JIRA, and change-management workflows.
- Basic software engineering fundamentals, including version control, code review, testing, and CI/CD.
- Understanding of clusters, nodes, compute, and storage; Terraform, infrastructure as code, and IAM exposure are bonuses.
- Familiarity with Grafana, job-level metrics, data health checks, and vulnerability remediation using tools such as Snyk or Akto.
- Strong problem-solving, analytical, stakeholder management, and communication skills.
Benefits
- Hybrid work environment.
- New-hire orientation with job-specific onboarding and training.
- Internal and global mobility opportunities.
- Competitive healthcare benefits.
- Bonus and performance incentives.
- Generous annual PTO, paid parental leave, and two additional paid days for holidays, cultural events, or inclusion initiatives.
- Employee resource groups connecting MiQ regions and supporting communities.
Tech Stack
Categories
Data Engineering
About MiQ Digital
MiQ Digital provides programmatic media and analytics services for brands and advertising agencies, using an agnostic approach across major buying platforms and channels including CTV and digital. It sells managed-service campaign planning, activation, and measurement, plus data science and media analytics, typically priced on media spend and service fees. Founded in 2010 and headquartered in London, it operates globally as a privately held company.