Deploying Large Language Models (LLMs) locally—whether for privacy, cost savings, or offline availability—is the new frontier for developers. But unlike deploying a standard web app where you just spin up an AWS EC2 instance and forget about it, deploying LLMs requires precise hardware mathematics. If you guess your VRAM (Video RAM) requirements, you will either overpay for GPUs you don't need, or your inference will crash entirely. Today, we're breaking down the exact math behind LLM VRAM consumption, the impact of quantization, and how to calculate your hardware needs before you hit deploy. The Core Equation: Parameters to Gigabytes The foundational rule of LLMs is simple: Parameters dictate memory. Every parameter in a standard, unquantized model is stored as a 16-bit float (FP16 or BF16). 16 bits = 2 bytes. Therefore, the baseline formula to load a model's weights into memory is: VRAM (in GB) = (Number of Parameters in Billions) × 2 bytes Let's look at Meta's Llama-3-8B as an example: 8 Billion Parameters × 2 bytes = 16 GB of VRAM The Magic of Quantization (4-bit and 8-bit) Most consumer GPUs (like the RTX 3090 or 4090) cap out at 24GB of VRAM. If an 8B model takes 16GB, how on earth are people running 70B models at home? The answer is Quantization. Quantization is the process of compressing the model's weights by reducing their precision. Instead of using 16 bits (2 bytes) per parameter, we compress them down to 8 bits (1 byte) or even 4 bits (0.5 bytes). Here is how the math changes for our Llama-3-8B model: 8-bit Quantization (INT8): 8B × 1 byte = 8 GB VRAM The Hidden Killer: The KV Cache Here is where 90% of developers make their fatal mistake. They calculate the VRAM needed for the weights (e.g., 4GB), they see their GPU has 8GB, and they deploy. Then they send a massive document to the LLM to summarize, and the server crashes. Why? The KV Cache. When an LLM generates text, it needs to remember the previous context (your prompt + what it has generated so far). It stores this memory in the Key-Value (KV) Cache. The KV Cache grows linearly with your context length. The longer your prompt, the more VRAM it consumes. The formula for KV Cache VRAM is complex, but it looks like this: KV Cache VRAM = 2 × Context Length × Layers × Hidden Size × 2 bytes If you are running a server with multiple concurrent users, each user gets their own KV Cache. If you have 10 users sending 4k-token prompts, your KV cache alone could consume 10GB of VRAM! How to Stop Guessing Doing this math manually every time you switch between Llama-3, DeepSeek, or Mistral—while factoring in context windows, batch sizes, and GGUF quantization levels—is exhausting. Because I was tired of spinning up rented cloud GPUs only to find out they didn't have enough VRAM for my context window, I built a pure-math client-side tool to calculate this instantly. It's called the LLM VRAM Calculator. You simply input: The Model Size (e.g., 70B) Why this matters Do the math first. Deploy second. Have you ever hit an unexpected OOM error in production? What model were you trying to run? Let me know in the comments!