Senior Data Engineer
CoStar Group2 months ago
Richmond, VA, USASenior
Base Salary
$133k - $172k/yr
Responsibilities
- Design, build, and maintain scalable data pipelines and data platforms.
- Develop and optimize ETL and ELT processes for large, complex datasets.
- Build repeatable, customizable, and recurring data migration and transformation processes from multiple heterogeneous external sources.
- Normalize inconsistent third-party schemas into standardized enterprise data models, including source-to-target mapping, deduplication, and entity resolution.
- Standardize the data environment across billing systems to improve visibility into business patterns, errors, and optimizations.
- Collaborate with engineering, product, DevOps, and DBA teams to deliver production systems.
- Contribute to technical design decisions and mentor other engineers.
- Implement data quality, validation, and reconciliation controls for externally sourced data.
- Use AI-assisted or agentic engineering workflows to accelerate software delivery, code review, and root-cause analysis.
Requirements
- A bachelor's degree from an accredited, not-for-profit, in-person college or university is required.
- 5+ years of data engineering experience, including setting technical direction on projects and influencing design across a team.
- Hands-on experience building batch and incremental ingestion pipelines from REST or GraphQL APIs, flat files, SFTP feeds, third-party data vendors, and database replication or CDC.
- Advanced SQL and strong data modeling skills, including dimensional/star schema, normalized, or Data Vault models.
- Proficiency in Python, Scala, Java, or C#/.NET.
- Production experience with a distributed processing engine, cloud data warehouse or lakehouse platform, and pipeline orchestration tooling.
- Experience with version control and CI/CD in AWS, Azure, or GCP.
- Experience with agentic engineering using AI-assisted development tools such as Claude Code or similar.
- Preferred experience with Databricks, Delta Lake, Unity Catalog, Delta Live Tables, Databricks SQL, Workflows, Apache Iceberg, or Apache Hudi.
- Preferred experience with dbt, SQLMesh, Fivetran, Airbyte, Qlik Replicate, AWS DMS, Debezium, Kafka Connect, Kafka, Kinesis, Spark Structured Streaming, or Flink.
- Preferred experience with data quality, observability, cataloging, lineage, governance, master data management, entity resolution, record-linkage, address, or geospatial standardization tools.
- Preferred experience with AWS data services, Terraform, C#/.NET backend data services, and AI/ML or feature-engineering pipelines.
- A track record of commitment to prior employers is required.
Benefits
- Comprehensive medical, vision, dental, prescription drug, life, legal, and supplementary insurance.
- Virtual and in-person mental health counseling for individuals and families.
- Commuter and parking benefits.
- 401(k) retirement plan with matching contributions.
- Employee stock purchase plan and paid time off.
- Tuition reimbursement, internal training, and professional and academic growth opportunities.
- On-site fitness center or reimbursed fitness center membership costs, with yoga, Pelotons, personal training, and group exercise classes depending on location.
- Employee Resource Groups and complimentary coffee, tea, hot chocolate, fresh fruit, and other healthy snacks.
- Richmond, Virginia location with Monday through Thursday in the office and Friday working from home.
Tech Stack
AirbyteAmazon RedshiftApache AirflowApache FlinkApache KafkaApache SparkAWSAzureC#DatabricksdbtGoogle BigQueryGoogle Cloud PlatformGraphQLJava.NETPythonScalaSnowflakeSQLTerraform
Categories
Data Engineering
About CoStar Group
CoStar Group builds subscription databases, analytics, and online marketplaces for commercial and residential real estate professionals, investors, and advertisers. Its portfolio includes CoStar (research and data), LoopNet (commercial listings), and Apartments.com (rental listings), generating revenue from subscriptions, listings, and advertising. The company is publicly traded on NASDAQ (CSGP) and is headquartered in Arlington, Virginia, with inclusion in the S&P 500 index.