The short answer: GPU memory (VRAM) is usually the first filter in a rental decision, because a model that does not fit simply does not run, while a model that fits with room to spare is paying for capacity it does not use. Hourly rates tend to rise with memory and with newer architectures, so the comparison is less about the cheapest card and more about the cheapest card that fits your workload with a sensible margin. This guide walks through the sizing arithmetic using rules of thumb and hypothetical numbers only; it does not describe any tracked provider's prices.
Step 1: estimate memory for the weights
Weight memory is roughly the parameter count multiplied by bytes per parameter. At 16-bit precision (FP16 or BF16) that is 2 bytes, at 8-bit it is about 1 byte, and at 4-bit it is about 0.5 bytes. Applied to common model sizes, the weights alone come to roughly:
- 7 billion parameters: about 14 GB at 16-bit, 7 GB at 8-bit, 3.5 GB at 4-bit.
- 13 billion parameters: about 26 GB, 13 GB and 6.5 GB.
- 70 billion parameters: about 140 GB, 70 GB and 35 GB.
These are lower bounds. Real usage adds a runtime overhead and, for inference, the key-value cache that grows with context length and with the number of concurrent requests. A common planning habit is to leave 20 to 30 percent headroom above the weights, then adjust after measuring.
Step 2: training needs a multiple of inference memory
Full fine-tuning keeps gradients and optimizer state alongside the weights. A frequently cited rule of thumb for mixed-precision training with the Adam optimizer is on the order of 16 bytes per parameter before activations, which would put a 7 billion parameter model near 112 GB. Parameter-efficient methods such as LoRA, and quantized variants of them, train a small set of added weights and can cut this sharply. The exact figure depends on the framework, batch size, sequence length and any memory-saving techniques, so treat the numbers here as a starting point to be tested.
Step 3: compare card sizes, not just card names
Commonly listed data-centre cards come in memory tiers: 24 GB class (for example L4), 40 or 80 GB class (A100), 48 GB class (L40S), 80 GB class (H100) and 141 GB class (H200). Listings on the compare page show which of these are offered and at what INR rate. Two points matter for the arithmetic. First, the same memory size can carry different rates because of memory bandwidth, interconnect and generation. Second, a larger card can sometimes finish the same job sooner, so a higher hourly rate does not automatically mean a higher job cost. The H100 versus A100 comparison and the RAG, fine-tuning and training guide cover that trade-off from different angles.
A worked example with hypothetical numbers
Suppose a team needs to serve a 13 billion parameter model at 8-bit precision. The weights take about 13 GB; with 30 percent headroom the target is roughly 17 GB, which fits a 24 GB card. Now imagine two hypothetical listings: Card X with 24 GB at INR 40 per hour, and Card Y with 80 GB at INR 150 per hour. These are invented figures, not real prices. For a 100-hour month, Card X would cost INR 4,000 and Card Y INR 15,000. The extra memory on Card Y only pays for itself if it enables something the smaller card cannot, such as a longer context, more concurrent requests per GPU, or a larger model, and if the job is throughput-bound, the per-request cost depends on how many requests each card actually serves per hour.
When splitting across GPUs changes the price picture
If a model exceeds one card, it can be sharded across several, but then interconnect speed and per-node pricing begin to matter. The multi-GPU node pricing guide explains what an HGX-style rate does and does not include. Quantizing to a lower precision is the other route; it reduces memory but can affect output quality, so the saving should be validated on your own evaluation set before it is counted.
Compliance and invoicing context
Memory sizing does not change tax treatment: GPU rental from a provider issuing Indian GST invoices carries GST at the prevailing rate regardless of card size, and whether it can be recovered as input tax credit depends on the buyer's own registration; see the GST invoicing guide. If the model processes personal data, the choice of region and the data-handling terms are separate from the memory decision and are covered in the DPDP data residency guide.
Frequently asked questions
Is more VRAM always better value? No. Memory beyond what the workload needs raises the hourly rate without raising useful output, unless it unlocks larger batches or longer contexts that improve throughput per rupee.
How do I check my estimate? Run a short test on a rented instance and read the peak memory figure from the framework or from nvidia-smi, then compare it with your estimate before committing to a longer booking.
Where can I model the cost? The cost calculator takes an hourly rate and usage hours so that different memory tiers can be compared side by side using the live INR listings.
Memory figures here are approximations for planning, and all prices in the worked example are hypothetical. Check current listings and each provider's terms before deciding.