https://riteshsonawane.com/tags/llm/Recent content in llm on Ritesh SonawaneHugoen-usSat, 18 Jul 2026 00:00:00 +0000https://riteshsonawane.com/blog/type-of-models/Sat, 18 Jul 2026 00:00:00 +0000https://riteshsonawane.com/blog/type-of-models/<p>Ask someone what model they&rsquo;re running and they&rsquo;ll usually give you a name, Llama, GPT-4, DeepSeek. Ask them what architecture that model actually uses and why it was picked, and the conversation usually stalls.</p> <p>That gap matters. The architecture behind a model decides how it processes tokens, how much GPU memory it eats, how fast it responds, and what kind of hardware it wants. Two models with similar benchmark scores can have completely different cost and latency profiles once you look under the hood. If you work in AI infrastructure, this is the layer where your capacity planning, your hardware choices, and your cost per token all actually come from.</p>https://riteshsonawane.com/blog/llms-inference-kpis/Mon, 06 Jul 2026 00:00:00 +0000https://riteshsonawane.com/blog/llms-inference-kpis/<p>Monitoring LLM inference is not like monitoring a conventional web service. The metrics are different, the failure modes are different, and the relationship between what you measure and what the user experiences is more indirect.</p> <p>This guide explains the most useful LLM inference KPIs for SREs, what each metric measures, what causes it to change, and how to use the signals together when production performance degrades.</p> <h2 id="the-mental-model-prefill-and-decode">The mental model: prefill and decode</h2> <p>Every LLM inference request goes through two distinct phases. If you do not understand the difference, the metrics will not make sense.</p>