5 hours ago
Campinas, BrazilMid Level
Responsibilities
- Design, implement, and maintain reliable, reprocessable, and cost-efficient end-to-end ELT/ETL data pipelines.
- Orchestrate data loads with Airflow and AWS Lambda, and support EKS deployments when needed.
- Model bronze, silver, and gold data layers in Snowflake and develop dbt models, macros, tests, and documentation.
- Build Python and FastAPI integrations and services for exposing, consuming, and automating data processes.
- Consume, normalize, version, and distribute historical market data through primarily batch workflows.
- Write and optimize SQL for Snowflake and Postgres, including partitioning, clustering, and query tuning.
- Use Pandas for targeted data processing, validation, and transformation prototyping.
- Implement data quality checks, observability, access controls, masking, documentation, SLAs, data contracts, and governance standards.
- Collaborate with business, analytics, and product teams on data standards and governance.
Requirements
- Strong SQL experience, including query optimization and relational and analytical data modeling.
- Practical experience with Snowflake warehousing, roles, tasks, performance, and cost management.
- Experience building Airflow pipelines with DAGs, sensors, retries, and SLAs, as well as AWS Lambda workloads.
- Knowledge of Postgres, including ingestion, basic replication or CDC, and database routines.
- Experience using Python and Pandas for data work and FastAPI for API development.
- Experience with batch-oriented ELT/ETL, Git versioning, and safe CI/CD deployment of pipelines and dbt models.
- Knowledge of data security and governance, including access control, lineage, documentation, and sensitive data handling.
- Technical English proficiency for reading documentation.
- Preferred: EKS/Kubernetes, market-data integrations, Prometheus, Grafana, CloudWatch, Great Expectations, Soda, Pydantic, JSON Schema, data contracts, Snowflake performance optimization, AWS/Snowflake FinOps, Debezium, DMS, SQS, SNS, and Kafka.
Benefits
- Health and dental insurance.
- Meal and food allowances.
- Childcare assistance.
- Extended parental leave.
- Wellhub, Gympass, and TotalPass partnerships for fitness, health, and wellness services.
- Profit-sharing program (PLR).
- Life insurance.
- CI&T University and other continuous-learning platforms.
- Discount club and online physical, mental, and wellness programs.
- Pregnancy and responsible-parenting courses.
- Online language-learning and course-platform partnerships.
- Candidates residing in the Campinas Metropolitan Region must work from the company's Campinas offices according to the applicable attendance policy.
Tech Stack
Apache AirflowApache KafkaAWSdbtFastAPIGitGitHub ActionsGitLab CI/CDGrafanaKubernetesPandasPostgreSQLPrometheusPythonSnowflakeSQLTerraform
Categories
Data Engineering
About CI&T
CI&T is a public IT services firm that builds and runs digital products for large enterprises, combining consulting with end-to-end software engineering. It delivers AI deployment, cloud modernization, data platforms, mobile/web development, and systems integration through project-based and managed engagements. Founded in 1995 and headquartered in New York, it operates globally and is listed on the NYSE (CINT).