Senior Staff - IT Data Engineering and Analytics
StoneX Group Inc.2 years ago
Pune, IndiaStaff+
Responsibilities
- Run discovery sessions and translate stakeholder requests into requirements, data contracts, and acceptance criteria.
- Create source-to-target mappings covering lineage, transformation logic, keys, defaults, SCD handling, and data quality rules.
- Design and build batch and streaming pipelines using Databricks, PySpark, SQL, Delta Lake, Unity Catalog, and Lakeflow.
- Use AI coding agents to profile sources, draft mappings, generate pipelines and tests, and create reusable AI-enabled data engineering frameworks.
- Validate and reconcile outputs with business users, own data quality and monitoring, and drive adoption in Power BI and downstream systems.
- Own production reliability, CI/CD, alerting, incident response, root-cause analysis, and cost and performance tuning.
- Communicate with business and technology leaders, document decisions in Confluence and Jira, and manage changing scope and timelines.
- Coach engineers on business engagement and AI-assisted delivery and review AI-generated code for correctness, security, and performance.
Requirements
- 6–9 years of hands-on data engineering experience, including at least 3 years building production pipelines on Spark or Databricks.
- At least 2 years in a business-facing delivery role such as forward-deployed engineering, consulting, solutions, or embedded analytics engineering.
- Expertise in Python, PySpark, and SQL, with strong experience in Delta Lake, Unity Catalog, and Databricks Workflows or Lakeflow on Microsoft Azure.
- Proven ability to create source-to-target mappings and dimensional or medallion data models from ambiguous requirements.
- Recent demonstrable use of AI coding agents for production work, with the judgment to verify, test, and reject generated output.
- Excellent verbal and written communication skills, including the ability to run workshops and explain trade-offs to senior business stakeholders.
- Ability to work at pace with shifting priorities, tight deadlines, and occasional overlap with US and UK business hours.
- Bachelor’s degree in Computer Science, Engineering, Mathematics, or a related field, or equivalent demonstrable experience.
- Databricks Certified Data Engineer Professional is strongly preferred, or Databricks Certified Data Engineer Associate.
- Azure Data Engineer Associate or equivalent certification.
- Generative AI certifications such as Databricks Generative AI Engineer Associate are preferred.
- Capital markets, commodities, or trade-lifecycle data experience is a preferred qualification.
- Experience building reusable frameworks or internal tools, streaming with Kafka or RedPanda, APIs, Profisee MDM, Microsoft Dynamics, Power BI semantic models, or Databricks AI/BI Genie is preferred.
Benefits
- Hybrid work arrangement in Bengaluru under the company’s RTO policy.
- Collaboration with teams across the US, UK, India, and Poland, with regular overlap with US and UK hours.
- Occasional travel for discovery workshops and go-lives.
Tech Stack
Apache AirflowApache KafkaApache SparkAzureDatabricksDockerGitHub ActionsJenkinsKubernetesMicrosoft SQL ServerPythonReactSQL
Categories
Forward Deployed
About StoneX Group Inc.
StoneX Group Inc. is a public financial services firm that connects institutional, commercial, and retail clients to global markets through brokerage, execution, clearing, risk management, and cross-border payments. It offers trading and hedging across commodities, equities, fixed income, futures/options, and FX, and operates retail platforms via GAIN Capital’s FOREX.com and City Index (acquired 2020). Founded in 1924 and headquartered in New York, it lists on Nasdaq under SNEX.