Runyard is a free hardware-aware AI model browser. You enter your CPU, GPU, and VRAM and it instantly shows every local LLM that will run on your machine, ranked by speed and quality.

How much VRAM do I need to run local LLMs?

8GB of VRAM runs 7B models like Llama 3.1 8B and Mistral 7B at Q4 quantization. 16GB unlocks 13B models. 24GB lets you run Mixtral 8x7B and Llama 3 70B at lower quantization.

What is the best local LLM for my GPU?

Use Runyard at www.runyard.dev — enter your GPU and VRAM and the Model Radar will rank every compatible LLM for your exact hardware, showing estimated tokens per second for each model.

Can I run Llama 3 locally?

Yes. Llama 3.1 8B at Q4 runs on any 8GB VRAM GPU. Llama 3.1 70B needs around 40GB VRAM at Q4, or an Apple Silicon Mac with 64GB+ unified memory.

GGUF Variant Chooser | Runyard Tools

P-10

Model size

Quantization

GPU or unified memory

System RAM

Context target

Quality preference

Speed preference

TurboQuant KV compression

Live result

Q5_K_M recommended

P-10

For 12 GB VRAM and a balanced preference, Q5_K_M is the cleanest first choice.

Fit-first

Q4_K_MQ5_K_M

Current preference

Quality-first

Q6_KQ5_K_M

Possible later

How It Works

3 inputs. Instant results.

01

Set the scenario

Choose realistic hardware, model, and context assumptions.

02

Read the result

The hero shows a working result instead of a decorative promo block.

03

Act on the outcome

Use the result to adjust fit, speed, quantization, or context.

Features

Everything that powers gguf variant chooser.

01

Planning-first

Built to make local-AI decisions easier to reason about.

02

Local-AI focused

Built to make local-AI decisions easier to reason about.

03

Interactive hero

Built to make local-AI decisions easier to reason about.

04

Runyard design system

Built to make local-AI decisions easier to reason about.

05

Target model family

Grounded in the actual inputs and outputs this page is designed around.

06

Best-fit GGUF suffixes

Grounded in the actual inputs and outputs this page is designed around.

07

Friendly for Hugging Face browsing

Grounded in the actual inputs and outputs this page is designed around.

08

Standalone tool

Grounded in the actual inputs and outputs this page is designed around.

Spotlight

The differentiator behind gguf variant chooser.

Q4_0 vs Q4_K_M

Q4_0 uniformQ4_K_M calibratedBetter quality same size

Q4 vs Q5 on 12 GB

Q4 at 8.4 GBQ5 at 9.9 GB+1.5 GB for step up

Q8 vs Q6 on 24 GB

Q8 at 15 GBQ6 at 11 GBQ6 fits, Q8 tight

Visual comparison

Clarity

Fit

Actionability

Reading Results

How to read the output tiers.

Comfortable

<70%

Enough breathing room for normal use.

Tight

70%-95%

Should work, but overhead matters.

Borderline

95%-110%

Likely needs one tradeoff.

Too heavy

>110%

Time to step down.

Quick Reference

Common setups at useful defaults.

Scenario	Baseline	Result	Notes
Starter setup	7B / Q4 / 8K	Light local target	Good first benchmark
Balanced setup	8B / Q4 / 16K	Everyday sweet spot	Works for many users
Heavier setup	14B / Q5 / 16K	Quality-focused target	Needs stronger hardware
Stretch setup	32B / Q4 / 16K	Ambitious local target	Useful upper bound

* These are approximations for planning, not a promise of exact runtime behavior.

Benefits

Why people use gguf variant chooser.

01

Faster decisions

It helps eliminate dead-end local AI choices before you download, benchmark, or configure too much.

02

Clearer tradeoffs

The page turns a raw estimate into something you can actually act on.

03

Useful on its own

The hero provides a working tool surface while the rest of the page explains what the output means.

FAQ

Questions people ask before using gguf variant chooser.

What is a GGUF file?

GGUF (GPT-Generated Unified Format) is the standard file format for quantized LLM weights used by llama.cpp and Ollama. A file contains the model weights, quantization metadata, tokenizer, and chat template.

What do suffixes like Q4_K_M and Q5_K_S mean?

The number is quantization bits. K means K-means calibrated. M is medium importance matrix; S is small. Q4_K_M is the community default — good balance of quality and size. Always prefer K variants over plain Q4_0.

Why does the same model have so many GGUF files?

Authors release multiple quant variants so users can choose based on their VRAM budget and quality target. The same 7B model might ship as Q2, Q3, Q4, Q5, Q6, and Q8 variants on Hugging Face.

Which GGUF variant should I download first?

Start with Q4_K_M. It is the community standard for good reason — solid quality, fits most hardware, and is the fastest to experiment with. Only step up to Q5 or Q6 if you have confirmed headroom.

What is the difference between Q4_K_M and the older Q4_0?

Q4_0 uses uniform quantization without calibration. Q4_K_M uses per-block calibration for better quality at the same bit rate. Always prefer K variants when the repo offers them — quality improvement is real.

How do I check if a GGUF file will fit my GPU before downloading?

The file size is a close proxy for VRAM requirement — add ~0.5 GB for runtime overhead. Better yet, use the VRAM Calculator at /tools/vram-calculator to get a precise estimate before the download starts.

RUNYARD.DEV / Tools / GGUF Variant Chooser

Estimates on this page are directional and should be validated against your actual runtime and hardware.