observability on Ritesh Sonawane

LLM Inference KPIs Every SRE Should Know

Mon, 06 Jul 2026 00:00:00 +0000

Monitoring LLM inference is not like monitoring a conventional web service. The metrics are different, the failure modes are different, and the relationship between what you measure and what the user experiences is more indirect.

This guide explains the most useful LLM inference KPIs for SREs, what each metric measures, what causes it to change, and how to use the signals together when production performance degrades.

The mental model: prefill and decode

Every LLM inference request goes through two distinct phases. If you do not understand the difference, the metrics will not make sense.

Scaling Prometheus with Thanos

Tue, 22 Jul 2025 00:00:00 +0000

This blog is based on my work at CloudRaft!

Observability is a crucial pillar of any application, and monitoring is an essential component of it. Having a well-suited, robust monitoring system is crucial. It can help you detect issues in your application and provide insights once it is deployed. It aids in performance, resource management, and observability. Most importantly, it can help you save costs by identifying issues in your infrastructure. One of the most popular tools in monitoring is Prometheus.

100 TB from InfluxDB to Grafana Mimir

Mon, 07 Apr 2025 00:00:00 +0000

This blog is based on my work at CloudRaft!

Introduction

Observability is the backbone of any modern infrastructure, enabling organizations to monitor system health, optimize performance, and ensure seamless operations. However, when legacy observability systems reach their limits—whether due to scalability challenges, high costs, or lack of vendor support—businesses must pivot to more future-proof solutions.

The migration presented a unique set of challenges: handling seven years of monitoring data (roughly 100TB of uncompressed data), complex migration as there is no direct support to migrate from InfluxDB to Grafana Mimir, Grafana dashboards rewrite from InfluxQL to PromQL format for 100s of dashboards. In this blog, we’ll walk through the entire migration process, the challenges faced, and the architectural choices that enabled a seamless transition to Grafana Mimir.

Monitoring with Prometheus

Thu, 04 Jul 2024 00:00:00 +0000

This blog is based on my work at CloudRaft!

Monitoring is the lifeline that ensures the seamless operation of systems, applications, and networks. Whether managing a small startup or a large enterprise, the ability to monitor infrastructure in real time is crucial for maintaining performance, ensuring reliability, and quickly resolving issues before they escalate. The average cost of downtime in IT industry as per Pingdom is between $5600 to $9600 per minute and as more and more digitization happens, it is going to increase further.