Observability Engineer

Job Summary

OmniOps is seeking an Observability Engineer to design, deploy, and operate observability platforms for clients across multiple industries. The role involves working with client teams to ensure infrastructure and applications are fully observable using the Grafana ecosystem and related tools across cloud, on-premises, and hybrid environments.

Responsibilities

• Deploy, configure, upgrade, and manage Grafana Alloy agents across Kubernetes, Linux, and Windows environments.
• Design and maintain observability backends using Grafana, Loki, Mimir, Tempo, and Alertmanager, either on-premises or through Grafana Cloud.
• Configure telemetry pipelines for logs, metrics, and traces, ensuring reliable data collection and delivery.
• Manage retention policies, storage capacity, and platform health across client environments.
• Implement platform-level dashboards, alerting rules, notification channels, and monitoring standards.
• Support client teams with application instrumentation and observability best practices.
• Define telemetry collection standards, SDK recommendations, and integration approaches.
• Assist clients in implementing alerting strategies and translating SLO requirements into actionable monitoring solutions.

Must haves

• Proven experience operating Grafana-based observability platforms in production environments.
• Hands-on experience with Grafana Alloy, OpenTelemetry Collector, Promtail, Grafana Agent, or similar telemetry collection tools.
• Strong experience deploying and managing monitoring agents across Kubernetes, Linux, and Windows environments.
• Solid understanding of Kubernetes, including DaemonSets, RBAC, manifests, and Helm.
• Experience with Grafana Cloud, including tenant management, stack configuration, and agent connectivity.
• Strong understanding of observability concepts, including metrics, logs, traces, alerting, retention strategies, and capacity planning.

Nice to haves

• Experience delivering observability services in managed services or multi-tenant environments.
• Knowledge of Infrastructure as Code and GitOps tools such as Terraform, Helm, and Ansible.
• Experience implementing OpenTelemetry instrumentation for applications and services.
• Familiarity with AWS, Azure, and GCP logging integrations.

We refresh listings regularly, but some roles close early on the source platform.