What GPU compute costs

As of 2026-09-18, across 52 providers: an H100 SXM rents for a median of $3.50 per GPU-hour (cheapest $1.68), a B200 for $6.59, an A100 80GB for $1.79, and a consumer RTX 4090 for $0.60. Multiply by 720 for a month. The table above has every GPU we track.

The NVIDIA RTX 3090: 96 GB of memory per dollar-hour at its median of $0.25. Memory per dollar is the right lens when the job is holding a model, not training one — most inference is memory-bound.

It depends on the model and the card. Llama 3.1 70B at 4-bit on one H100 SXM: about $14.28 per million generated tokens at today's median, single stream, from memory bandwidth. The cheapest set that holds it is 4× RTX A4000 at $0.68/hr. Batching many requests lowers the per-token cost by an order of magnitude; these figures are the one-user floor, useful for ranking cards against each other.

Three markets share one name. Hyperscalers price the instance — CPU, RAM, network, ecosystem — and the GPU is a fraction of it. Neoclouds price the GPU. Marketplaces are independent hosts competing on price with consumer-grade guarantees. The median sits with the neoclouds; the cheapest listing is nearly always a marketplace host; the most expensive is a hyperscaler. Each is the right answer for a different buyer.

Every provider's public pricing is read every six hours. Each priced unit becomes a row — instance, SKU, offer — divided by its GPU count. Per GPU, each provider is reduced to its own median, and the headline is the median of those. Spot and reserved are tracked the same way but never enter the headline. There are 54 snapshots so far; every one is kept in a public repository. The full method is on the methodology page.