Sr. Data Engineer-Hadoop,Hive,Spark,Trino, Star Rocks, Clickhouse,Data engineering-3+yrs AI exposure
Visa14 days ago
Bengaluru, IndiaSenior
Responsibilities
- Collaborate with product, business, cybersecurity, and other stakeholders to clarify requirements and identify data-product needs.
- Translate functional requirements into architecture designs and communicate technical solutions to data engineering teams.
- Implement maintainable, reusable, and extensible code and participate in code reviews.
- Own data products and data pipelines end to end, including technical-debt mitigation, documentation, and issue resolution.
- Build automation for transforming, managing, accessing, deploying, and monitoring batch and real-time data processes.
- Write queries and implement orchestration to extract and compile data across end-to-end pipelines.
- Maintain data governance, data quality, security, privacy, and retention practices.
- Develop unit, integration, and end-to-end testing and troubleshoot advanced production problems.
- Manage applicable software upgrades and server patches for security remediation.
Requirements
- At least 3 years of relevant work experience with a bachelor’s degree, or at least 5 years of relevant work experience.
- Experience solving data problems with data technologies and designing large-scale data pipelines and processing jobs.
- Experience with Hadoop, Hive, HBase, Spark, Trino, or comparable distributed data systems.
- Experience writing, modifying, reviewing, and deploying high-quality, testable, efficient code using Java or Python and SQL.
- Experience with data pipelines, ETL/ELT, data-quality frameworks, Git, production troubleshooting, and automated testing.
- Experience building and maintaining CI/CD pipelines and deploying applications with containerization and orchestration tools.
- Experience monitoring production applications and troubleshooting with observability tools.
- Experience integrating AI or ML services into applications and analyzing metrics for optimization.
- Experience partnering with stakeholders, applying secure coding practices, and complying with regulatory standards.
- Preferred: open-source contribution experience, AI-agent and agentic-pattern experience, enterprise high-availability and low-latency systems, storage/compute separation, query and table formats, streaming, metadata, governance, interoperability, and real-time systems.
Benefits
- Visa requires at least 3 days in the office, with the specific in-office expectations confirmed by the Hiring Manager.
Tech Stack
Categories
Data Engineering
About Visa
Visa builds and operates a global digital payments network that connects consumers, merchants, financial institutions, and governments. It earns fees for processing card transactions and offering risk, data, and value‑added services through VisaNet and products like tokenization and tap‑to‑pay. Founded in 1958 and headquartered in Foster City, California, Visa is a public company (NYSE: V) serving clients in more than 200 countries and territories.