
Lead Business Intelligence Engineer (Observability Engineer - Splunk / Grafana)
Depository Trust & Clearing Corporation (DTCC)1 day ago
Hyderābād, IndiaStaff+
Responsibilities
- Design and implement observability solutions across distributed systems using Grafana, Splunk ITSI, and Dynatrace.
- Develop and maintain dashboards, visualizations, Splunk reports, SPL correlations, and optimized logging streams.
- Integrate observability tools with Prometheus, CloudWatch, ServiceNow, Snowflake, and other data sources.
- Collaborate with application and infrastructure teams to define SLIs/SLOs and improve system reliability.
- Troubleshoot monitoring gaps, alert noise, and data ingestion issues.
- Provide technical, operational, and rotational 24x7 L3 support for observability tools and their users.
- Perform tool administration, documentation, service-request handling, vulnerability reconciliation, security-baseline activities, and audit support.
- Address and escalate incidents, problems, and change tickets in accordance with DTCC’s ITIL processes.
- Provide user training on Splunk, Grafana, and observability capabilities.
Requirements
- At least 5 years of related experience, including 5+ years of Splunk/Grafana engineering or production support experience across the lifecycle.
- Hands-on production experience with Grafana, Splunk including ITSI, and Dynatrace.
- Strong understanding of telemetry data types, observability architecture, and OpenTelemetry.
- Experience with Python, Bash, PowerShell, and automation tools.
- Familiarity with AWS, Azure, Kubernetes, and networking fundamentals including TCP/IP.
- Proficiency in Grafana queries, dashboard creation, Splunk search language, and Splunk administration.
- Preferred bachelor’s degree or equivalent experience.
- Preferred experience integrating observability tools with CI/CD pipelines.
- Preferred knowledge of ServiceNow, PagerDuty, AIOps, anomaly detection, and predictive analytics.
- Strong problem-solving, communication, and collaboration skills.
Benefits
- Competitive compensation including base pay and annual incentive.
- Comprehensive health and life insurance and well-being benefits, based on location.
- Pension and retirement benefits.
- Paid time off and personal/family care and other leaves of absence.
- Flexible hybrid work model with three days onsite and two days remote, with onsite days typically including Tuesdays and Wednesdays.
- Rotational 24x7 on-call schedule is required.
Tech Stack
Categories
DevOpsSite Reliability
About Depository Trust & Clearing Corporation (DTCC)
Depository Trust & Clearing Corporation (DTCC) runs global post-trade market infrastructure, delivering clearing, settlement, securities depository, trade reporting, and data services for broker-dealers, banks, and asset managers. It provides these services through subsidiaries such as DTC, NSCC, FICC, and the Global Trade Repository, with revenue primarily from transaction and service fees. Founded in 1973 and headquartered in Jersey City, NJ, DTCC is industry-owned and privately held, serving markets across multiple regions.