Runyard is a free hardware-aware AI model browser. You enter your CPU, GPU, and VRAM and it instantly shows every local LLM that will run on your machine, ranked by speed and quality.

How much VRAM do I need to run local LLMs?

8GB of VRAM runs 7B models like Llama 3.1 8B and Mistral 7B at Q4 quantization. 16GB unlocks 13B models. 24GB lets you run Mixtral 8x7B and Llama 3 70B at lower quantization.

What is the best local LLM for my GPU?

Use Runyard at www.runyard.dev — enter your GPU and VRAM and the Model Radar will rank every compatible LLM for your exact hardware, showing estimated tokens per second for each model.

Can I run Llama 3 locally?

Yes. Llama 3.1 8B at Q4 runs on any 8GB VRAM GPU. Llama 3.1 70B needs around 40GB VRAM at Q4, or an Apple Silicon Mac with 64GB+ unified memory.

OOM Fix Assistant | Runyard Tools

P-15

Model size

Quantization

GPU or unified memory

System RAM

Context target

Quality preference

Speed preference

TurboQuant KV compression

Live result

Likely headroom issue

P-15

OOM failures usually come from the same few causes: weights too large, context too long, or a setup that is simply too tight.

Current ask

12 GB5.59 GB

Estimated requirement

Fastest fix

Random trial and errorReduce context

Highest-probability move

How It Works

3 inputs. Instant results.

01

Set the scenario

Choose realistic hardware, model, and context assumptions.

02

Read the result

The hero shows a working result instead of a decorative promo block.

03

Act on the outcome

Use the result to adjust fit, speed, quantization, or context.

Features

Everything that powers oom fix assistant.

01

Planning-first

Built to make local-AI decisions easier to reason about.

02

Local-AI focused

Built to make local-AI decisions easier to reason about.

03

Interactive hero

Built to make local-AI decisions easier to reason about.

04

Runyard design system

Built to make local-AI decisions easier to reason about.

05

The error pattern you saw

Grounded in the actual inputs and outputs this page is designed around.

06

Likely root cause

Grounded in the actual inputs and outputs this page is designed around.

07

Good for debugging under pressure

Grounded in the actual inputs and outputs this page is designed around.

08

Standalone tool

Grounded in the actual inputs and outputs this page is designed around.

Spotlight

The differentiator behind oom fix assistant.

Context reduction fix

num_ctx 32K → OOMnum_ctx 8K → stableFastest no-download fix

Quant downgrade fix

Q8 → OOMQ4_K_M → fitsHalves weight VRAM

CPU offload fallback

Hard OOM crashPartial GPU runSpeed trade for stability

Visual comparison

Clarity

Fit

Actionability

Reading Results

How to read the output tiers.

Comfortable

<70%

Enough breathing room for normal use.

Tight

70%-95%

Should work, but overhead matters.

Borderline

95%-110%

Likely needs one tradeoff.

Too heavy

>110%

Time to step down.

Quick Reference

Common setups at useful defaults.

Scenario	Baseline	Result	Notes
Starter setup	7B / Q4 / 8K	Light local target	Good first benchmark
Balanced setup	8B / Q4 / 16K	Everyday sweet spot	Works for many users
Heavier setup	14B / Q5 / 16K	Quality-focused target	Needs stronger hardware
Stretch setup	32B / Q4 / 16K	Ambitious local target	Useful upper bound

* These are approximations for planning, not a promise of exact runtime behavior.

Benefits

Why people use oom fix assistant.

01

Faster decisions

It helps eliminate dead-end local AI choices before you download, benchmark, or configure too much.

02

Clearer tradeoffs

The page turns a raw estimate into something you can actually act on.

03

Useful on its own

The hero provides a working tool surface while the rest of the page explains what the output means.

FAQ

Questions people ask before using oom fix assistant.

What are the most common causes of OOM errors in local LLM inference?

In order of frequency: model weights exceed available VRAM, context window set too high, another application consuming GPU memory, using Q8/F16 when Q4 fits, or a stale Ollama process holding onto memory from a previous session.

How do I fix an OOM error in Ollama?

First, reduce num_ctx in the Modelfile. Second, try Q4_K_M instead of Q8. Third, close other GPU-intensive applications. Fourth, restart Ollama completely — stale model processes often retain memory even after the session ends.

What is the fastest fix that doesn't need a new model download?

Reduce context length. Cutting num_ctx from 32K to 8K can free several GB of VRAM instantly without changing the model. For most chat tasks 8K is more than sufficient and this is always the first thing to try.

Can I use CPU offload as a workaround for OOM?

Yes. If the model slightly exceeds your GPU VRAM, partial CPU offload via `--n-gpu-layers` can load most layers to GPU and overflow to RAM. You lose some speed but avoid the hard OOM failure.

Why does my model load fine but crash mid-conversation?

The KV cache grows as the conversation lengthens. A model that loads at 6 GB can reach 10+ GB after a long session if context is set high. The fix is lowering context length or enabling TurboQuant compression.

How do I check how much VRAM my model is actually using?

On Windows: Task Manager → Performance → GPU → VRAM. On Linux: `nvidia-smi`. On Mac: Activity Monitor → GPU History. Compare actual use against available VRAM to understand how much headroom remains.

RUNYARD.DEV / Tools / OOM Fix Assistant

Estimates on this page are directional and should be validated against your actual runtime and hardware.