Principal Apache NiFi/Data Integration Engineer
Red Arch Solutions1 day ago
Reston, VA, USAStaff+
Base Salary
$220k - $255k/yr
Responsibilities
- Design, develop, test, deploy, maintain, and optimize complex Apache NiFi data flows and scalable ingestion pipelines.
- Develop custom Java-based Apache NiFi processors, controller services, reporting tasks, APIs, and reusable processor libraries.
- Implement metadata extraction, schema validation, data tagging, attribute enrichment, security classification, data quality validation, and workflow automation capabilities.
- Integrate enterprise systems through REST APIs, SOAP services, JDBC/ODBC connections, message queues, Kafka, file transfers, and cloud storage services.
- Support data onboarding into approved repositories and data services while complying with metadata, taxonomy, governance, and stewardship requirements.
- Implement Zero Trust-aligned authentication, authorization, encryption, auditing, ABAC, and policy-driven data access patterns.
- Troubleshoot complex data movement and transformation issues and tune flow performance, throughput, latency, and resource utilization across clustered environments.
- Provide technical guidance and mentorship, participate in architecture reviews and working groups, and create engineering standards, documentation, procedures, and onboarding guides.
Requirements
- At least 20 years of experience is stated, including 12+ years in data integration, ETL, or enterprise middleware and 8+ years of hands-on Apache NiFi development.
- Demonstrated expertise with NiFi flow design, process groups, remote process groups, Site-to-Site communications, controller services, parameter contexts, flow versioning, NiFi Registry, clustered deployments, provenance tracking, back-pressure management, performance tuning, and secure flow design.
- Experience designing and supporting large-scale enterprise data ingestion architectures, onboarding new data sources, and integrating complex systems.
- Advanced experience developing Java-based NiFi processors, controller services, reporting tasks, custom APIs, schema validation, content parsing, metadata extraction, security enforcement, and data quality validation components.
- Strong experience with Java, Spring Framework, JSON, XML, Avro, Parquet, REST APIs, SQL, PostgreSQL, Elasticsearch, Kafka, Git, Docker, Kubernetes, and Linux.
- Understanding of Zero Trust Architecture, data-centric security, PKI, TLS, identity and access management, data governance, metadata management, and enterprise architecture principles.
- Active TS/SCI clearance is required.
- A bachelor's degree in Computer Science, Engineering, Information Systems, or a related technical discipline is desired.
- Preferred qualifications include experience with DIA, NSA, NGA, or other Intelligence Community customers; data catalog and metadata repository integration; ontology, taxonomy, semantic tagging, automated onboarding workflows, Cloudera DataFlow, Hortonworks DataFlow, and regulated or classified environments.
Benefits
- 100% paid employee healthcare premiums and CareFirst Advantage benefits.
- HaloScripts concierge prescription medication service.
- Generous paid time off and 11 paid federal holidays.
- 401(k) contribution of 10%, including a 6% match vested day one and up to 4% profit-sharing contribution.
- Annualized bonus compensation, spot bonuses, and generous referral bonuses.
- Tuition reimbursement, a 529 college savings plan, and a college loan payback program.
Tech Stack
Categories
Data Engineering
About Red Arch Solutions
Red Arch Solutions is an IT and engineering services firm that builds software, cloud, cyber, data, and systems solutions for U.S. intelligence and national security customers. It operates as a government contractor, delivering mission systems integration, zero-trust architectures, and AI/ML-enabled analytics. Founded in 2004 and headquartered in Columbia, Maryland, the company is privately held and focuses on programs within the federal intelligence community.