4 months ago
Bengaluru, IndiaMid Level
Responsibilities
- Design and develop data pipelines integrating data from multiple sources into the G2 Data Platform.
- Optimize pipelines for cost effectiveness, scalability, reliability, and near-real-time data availability.
- Develop scalable code and contribute to the advancement of G2’s data platform and infrastructure architecture.
- Service data requests and contribute to data modeling, design reviews, and data architecture improvements.
- Implement and drive data quality and data governance initiatives to completion.
- Ensure the data platform follows privacy, security, and compliance standards.
- Document data architecture, data models, and data consumption workflows.
- Guide junior engineers through technical support, best practices, and constructive feedback.
- Collaborate with peers in knowledge sharing and seek mentorship from senior team members.
Requirements
- At least 4 years of experience as a data engineer or ETL developer.
- At least 2 years of development experience with data modeling, optimization, and database architecture.
- Experience designing and developing data pipelines with cloud and open-source tools.
- Proficiency writing and debugging SQL queries.
- Good programming skills in Python or Java.
- Knowledge of performance tuning, optimization, and debugging of data pipelines.
- Working knowledge of AWS data services including DynamoDB, RDS, Data Pipeline, EMR, Lambda, Glue, and ECS, plus Snowflake.
- Experience handling structured and unstructured data.
- Proficiency with AWS Glue, Step Functions, Data Pipelines, Airflow, Airbyte, and dbt.
- Familiarity with distributed computing and Apache Spark, Hadoop, and Apache Kafka.
- Familiarity with software engineering principles and best practices.
- Preferred experience with Docker, Kubernetes, Snowflake schema design and optimization, startup environments, database administration, and cloud data governance.
- Preferred knowledge of Agile process methodology, CI/CD automation, Test Driven Development, security and compliance standards, and reporting tools such as Tableau, QlikView, Looker, and Power BI.
Benefits
- G2 offers a global, diverse, inclusive, and value-driven work environment with employee resource groups and philanthropic programs.
- Applicants may opt out of AI-assisted application review without disadvantage; final hiring decisions remain under human oversight.
Tech Stack
AirbyteAmazon DynamoDBApache AirflowApache HadoopApache KafkaApache SparkAWSdbtDockerJavaKubernetesPythonSnowflakeSQL
Categories
Data Engineering
About G2
G2 builds a B2B software marketplace where buyers research, compare, and review SaaS and enterprise tools, and vendors manage profiles and reputation. Revenue comes from subscriptions for seller solutions such as advertising, buyer intent data, and market intelligence. Founded in 2012 and headquartered in Chicago, the privately held company raised a Series D in 2021.