9 days ago
Atlanta, GA, USASenior
Base Salary
$107k - $160k/yr
Responsibilities
- Build and maintain data infrastructure for efficient data storage and retrieval.
- Develop scalable, sustainable, and robust data products and analytical solutions using cloud-based technologies.
- Develop and maintain batch and streaming data pipelines for data ingestion, transformation, and movement into data lakes, warehouses, and other stores.
- Review and optimize existing data systems and architectures.
- Collaborate with data and advanced analytics teams to gather requirements and deliver solutions meeting functional and non-functional needs.
- Build prototypes and implement data engineering frameworks and architectures supporting advanced analytics.
- Develop automated deployment pipelines and code deployment governance.
- Perform complex data modeling aligned with datastore technologies for performance and accessibility.
Requirements
- At least 4 years of relevant work experience; the role typically reflects 5 or more years.
- Experience developing data systems on AWS, GCP, or Azure.
- Hands-on experience with modern data architectures including data lakes, data lakehouses, data hubs, ingestion, governance, modeling, and observability.
- Experience with Kafka, AWS Glue, Iceberg, and Parquet for data ingestion and storage.
- Experience developing streaming data pipelines with Kafka and Flink.
- Experience with SQL-based transformation and modeling frameworks and orchestration tools including dbt, AWS Glue, and Airflow.
- Knowledge of SCD and schema evolution concepts.
- Experience using Spark for streaming, performance tuning, and debugging with Spark UI.
- Advanced programming skills in Python, Java, Scala, or similar languages.
- Expert-level SQL skills for data manipulation and optimization.
- Experience with DevOps practices, code management, CI/CD, and deployment strategies.
- Strong understanding of data governance, data quality, privacy, and security considerations.
- Previous Cargill experience or Animal Health Nutrition experience is preferred.
Benefits
- Expected salary range is $107,000-$160,000, plus eligibility for a discretionary incentive award; compensation varies by location, certifications, education, experience, and other factors.
- Minnesota Sick and Safe Leave accrues at one hour per 30 hours worked, up to 48 hours per calendar year unless otherwise provided by law.
- The business will not sponsor work visas for applicants for this position.
Tech Stack
Apache AirflowApache FlinkApache KafkaApache SparkAWSAzuredbtGoogle Cloud PlatformJavaPythonScalaSQL
Categories
Data Engineering
About Cargill
Cargill builds and operates global food and agriculture supply chains, selling commodities, food ingredients, animal nutrition, and risk-management services to food manufacturers, retailers, and farmers. It trades and processes grains, oilseeds, cocoa, and proteins, and provides logistics and biofuel solutions. Founded in 1865 and headquartered in Wayzata, Minnesota, Cargill is privately held and family-owned.