Runyard is a free hardware-aware AI model browser. You enter your CPU, GPU, and VRAM and it instantly shows every local LLM that will run on your machine, ranked by speed and quality.

How much VRAM do I need to run local LLMs?

8GB of VRAM runs 7B models like Llama 3.1 8B and Mistral 7B at Q4 quantization. 16GB unlocks 13B models. 24GB lets you run Mixtral 8x7B and Llama 3 70B at lower quantization.

What is the best local LLM for my GPU?

Use Runyard at www.runyard.dev — enter your GPU and VRAM and the Model Radar will rank every compatible LLM for your exact hardware, showing estimated tokens per second for each model.

Can I run Llama 3 locally?

Yes. Llama 3.1 8B at Q4 runs on any 8GB VRAM GPU. Llama 3.1 70B needs around 40GB VRAM at Q4, or an Apple Silicon Mac with 64GB+ unified memory.

CPU vs GPU Offload Calculator

P-09

Model size

Quantization

GPU or unified memory

System RAM

Context target

Quality preference

Speed preference

TurboQuant KV compression

Live result

5.59 GB estimated

P-09

CPU vs GPU Offload Calculator puts this setup around 5.59 GB including rough runtime overhead.

Weights

-4.50 GB

Q4_K_M

KV cache

-0.29 GB

8K context

How It Works

3 inputs. Instant results.

01

Set the scenario

Choose realistic hardware, model, and context assumptions.

02

Read the result

The hero shows a working result instead of a decorative promo block.

03

Act on the outcome

Use the result to adjust fit, speed, quantization, or context.

Features

Everything that powers cpu vs gpu offload calculator.

01

Planning-first

Built to make local-AI decisions easier to reason about.

02

Local-AI focused

Built to make local-AI decisions easier to reason about.

03

Interactive hero

Built to make local-AI decisions easier to reason about.

04

Runyard design system

Built to make local-AI decisions easier to reason about.

05

Available system RAM

Grounded in the actual inputs and outputs this page is designed around.

06

Suggested offload strategy

Grounded in the actual inputs and outputs this page is designed around.

07

Useful for borderline hardware

Grounded in the actual inputs and outputs this page is designed around.

08

Standalone tool

Grounded in the actual inputs and outputs this page is designed around.

Spotlight

The differentiator behind cpu vs gpu offload calculator.

20% offloaded layers

Full GPU speed~20% slowerManageable trade

50% offloaded layers

Smooth run~60% slowerCPU bottleneck

Full CPU inference

GPU speed5–15 tok/sViable for 7B only

Visual comparison

Clarity

Fit

Actionability

Reading Results

How to read the output tiers.

Comfortable

<70%

Enough breathing room for normal use.

Tight

70%-95%

Should work, but overhead matters.

Borderline

95%-110%

Likely needs one tradeoff.

Too heavy

>110%

Time to step down.

Quick Reference

Common setups at useful defaults.

Scenario	Baseline	Result	Notes
Starter setup	7B / Q4 / 8K	Light local target	Good first benchmark
Balanced setup	8B / Q4 / 16K	Everyday sweet spot	Works for many users
Heavier setup	14B / Q5 / 16K	Quality-focused target	Needs stronger hardware
Stretch setup	32B / Q4 / 16K	Ambitious local target	Useful upper bound

* These are approximations for planning, not a promise of exact runtime behavior.

Benefits

Why people use cpu vs gpu offload calculator.

01

Faster decisions

It helps eliminate dead-end local AI choices before you download, benchmark, or configure too much.

02

Clearer tradeoffs

The page turns a raw estimate into something you can actually act on.

03

Useful on its own

The hero provides a working tool surface while the rest of the page explains what the output means.

FAQ

Questions people ask before using cpu vs gpu offload calculator.

What is CPU-GPU offload and when is it useful?

When a model is too large for GPU VRAM, you can split it between GPU (fast layers) and CPU/RAM (slow layers). Layers that fit run at GPU speed; the rest run on CPU. Useful when 80–90% of the model fits.

How much speed loss should I expect from partial offload?

If 20% of layers offload to CPU, expect roughly 20–30% speed reduction. If 50% offload, speed can drop 60–70% because CPU becomes the bottleneck for half of every forward pass.

What runtimes support GPU offloading?

llama.cpp and Ollama (which runs llama.cpp internally) support GPU offloading via `--n-gpu-layers`. You can specify exactly how many transformer layers load to the GPU.

Is offloading worth it for a 70B model on a 24 GB GPU?

Often yes. A 70B Q4 model needs ~40 GB. With 24 GB GPU you offload ~40% to RAM. This typically gives 5–15 tok/s instead of 2–3 tok/s CPU-only, which is a meaningful improvement for a quality jump.

How do I control GPU layers in Ollama?

Ollama manages GPU layers automatically but you can override it via a Modelfile with PARAMETER num_gpu_layers or by setting the OLLAMA_NUM_GPU environment variable before starting the service.

When should I just use CPU-only instead of offloading?

For 7B or smaller models, CPU-only at Q4 is often fast enough (8–15 tok/s on modern hardware). Offloading shines most for 13B–32B models where partial GPU acceleration provides a meaningful speed boost.

RUNYARD.DEV / Tools / CPU vs GPU Offload Calculator

Estimates on this page are directional and should be validated against your actual runtime and hardware.