2 months ago
Base Salary
$126k - $189k/yr
Responsibilities
- Design, build, and maintain scalable ETL/ELT pipelines using PySpark, Spark SQL, and Delta Lake on Databricks.
- Implement and manage cloud data solutions across AWS, Google Cloud Platform, and Azure, including cloud-native storage, processing, and analytics services.
- Process and optimize petabyte-scale datasets using Databricks, Snowflake, Apache Iceberg, and Spark.
- Tune Spark workloads and Databricks clusters through job optimization, partitioning, caching, and autoscaling.
- Implement lakehouse architecture with Delta Lake, data quality controls, schema evolution, governance, and Unity Catalog.
- Design and architect Starburst-based data solutions and data federation across data lakes, relational databases, NoSQL databases, and cloud storage.
- Identify and resolve performance bottlenecks in data pipelines, queries, storage, and processing.
- Develop governed data pipelines with strong data quality, lineage, security, and compliance controls.
- Design data models and architectures supporting business intelligence, analytics, machine learning, and operational use cases.
- Provide data infrastructure and pipeline support for AI and machine learning projects, including RAG and Agentic AI systems.
- Participate in Agile and Scrum activities including sprint planning, daily stand-ups, and retrospectives.
- Provide technical leadership, make architectural decisions, mentor junior engineers, and guide project workstreams.
- Serve as a key stakeholder and client contact, communicating progress and translating business requirements into technical tasks.
Requirements
- Bachelor’s degree or equivalent experience.
- 6–10 years of hands-on experience in data engineering, preferably in a large-scale enterprise or financial services environment.
- Expert-level Python experience, including Pandas, NumPy, Dask, production-grade data processing, automation, and API development.
- Extensive PySpark and Spark experience, including the DataFrame API, Spark SQL, distributed processing, and performance tuning.
- Hands-on Databricks Lakehouse experience, including Delta Lake, structured streaming, and Spark optimization.
- Practical Ab Initio experience with GDE, Co>Operating System, and Conduct>It for enterprise ETL workflows.
- Snowflake experience covering data warehousing, data modeling, RBAC security, performance tuning, Snowpipe, and Time Travel.
- Starburst or Trino experience with federated queries and connections to diverse data systems.
- Familiarity or experience with Apache Iceberg and open table formats.
- Multi-year experience with at least one major cloud provider: AWS, Google Cloud Platform, or Azure.
- Experience with cloud data services such as AWS Glue, Lambda, S3, Redshift, Azure Data Factory, Synapse Analytics, Google Cloud Composer, Dataflow, and BigQuery.
- Understanding of the data lifecycle and experience building pipelines to support machine learning and AI/ML models.
- Experience or strong interest in preparing data for vector databases, RAG, and Agentic AI systems.
- Deep familiarity with Agile and Scrum methodologies.
- Demonstrated technical leadership, architectural influence, project leadership, mentoring, stakeholder management, and client engagement.
- Experience with Docker and Kubernetes is preferred.
- Deep understanding of data governance, data quality, and data security principles.
- Relevant certifications such as AWS Certified Big Data, Google Professional Data Engineer, or Snowflake SnowPro are preferred.
- Strong analytical, problem-solving, written, verbal communication, project management, and people management skills.
- Experience as an Applications Development Manager or in a senior-level Applications Development role is preferred.
Benefits
- Full-time role based in Irving, Texas, United States.
- Medical, dental, and vision coverage for eligible employees.
- 401(k) benefits.
- Life, accident, and disability insurance.
- Wellness programs.
- Paid planned time off, sick leave, and holidays.
- Additional discretionary and formulaic incentive and retention awards may be available to eligible employees.
Tech Stack
Amazon RedshiftApache SparkAWSAzureDatabricksDockerGoogle BigQueryGoogle Cloud PlatformKubernetesNumPyPandasPythonSnowflake
Categories
Data Engineering
About Citi
Citi is a public financial-services company offering consumer and institutional banking, credit cards, wealth management, treasury and trade solutions, and capital-markets services. It serves individuals, corporations, financial institutions, and governments in more than 160 countries and jurisdictions, earning interest and fee income from lending, payments, trading, and advisory. Founded in 1812 and headquartered in New York, it trades on the NYSE under the ticker C.