RUNYARD.DEV / BLOG

Latest Updates

Guides, deep-dives, and news on local AI models, hardware, and the Runyard platform.

deep-dive

A 27B Model at 30 Tokens a Second, No GPU — We Checked What That Requires

Alibaba says its RISC-V C950 ran Qwen3.8-27B at 30 tokens a second with no graphics card. Nobody inverted the number. It implies about 570 GB/s of memory bandwidth — server territory, and six times what a desktop has.

Read more →9 min read
deep-dive

Local LLM vs Claude, GPT and Gemini: Which API Is Worth Replacing?

Claude Fable 5.1 and GPT-6 Astra cost the same to the cent. Gemini 3.8 Flash is thirteen times cheaper. A $900 card pays back in 101 days against one and 4.2 years against the other — the tier decides, not the brand.

Read more →11 min read
deep-dive

Local LLM vs the OpenAI API: What It Actually Costs to Self-Host

Electricity is $0.18 per million tokens. All-in it is $0.93, and against cheap hosted models a GPU never pays for itself. Here is the comparison done honestly, with the arithmetic shown.

Read more →10 min read
deep-dive

Mac or Nvidia for Local AI in 2026? The Answer Changed This Year

An RTX 5090 costs $4,699 and holds 28.8 GB. A Mac Studio M5 Ultra costs $5,499 and holds 72 GB at 1.2 TB/s. On models both can run the card is 1.5x faster — not 4x. Here is where each one actually wins.

Read more →10 min read
deep-dive

Qwen3.8-27B Does Not Need 17 GB. Here Is What It Actually Needs

The 17 GB figure is a file size, not a memory requirement. Add the KV cache and runtime overhead and Q4_K_M needs about 20.7 GB — which is why 16 GB cards get 5 tokens per second instead of 50.

Read more →9 min read
deep-dive

The GPU You Were Going to Buy Now Costs $163 a Gigabyte

An RTX 5090 launched at $1,999 and sells for $4,699. We recomputed price per usable gigabyte across the catalogue — the flagship is now the worst value in it, and a 128GB Mac costs the same as one card.

Read more →11 min read
deep-dive

What Runs Locally in 2026: The 8, 12, 16, 24 and 32 GB Tier List

Computed from our catalogue, not estimated: 22 models fit 8 GB, 63 fit 24 GB, and the jump from 16 GB to 24 GB more than doubles the quality you can reach.

Read more →12 min read
deep-dive

Claude Code vs Codex vs Cursor: The Harness Is Worth More Than the Model

The same model scored 62.7% and 99.9% on the same benchmark, depending only on the harness around it. Why your agent wrapper matters more than your model choice.

Read more →12 min read
deep-dive

Is GPT-6 Astra Down? How to Tell in 60 Seconds

A 429 is usually you, a 503 is usually them, and a green status page proves neither. How to diagnose an Astra outage properly, and what to do while you wait.

Read more →12 min read
deep-dive

Cost Per Task Beats Cost Per Token. Here's How to Actually Measure It

A model at $50 per million output tokens can be cheaper than one at $20 if it finishes in fewer tokens and fails less often. The three numbers that decide your real bill.

Read more →12 min read
deep-dive

The 272K Token Cliff: How One Extra Token Doubles Your GPT-6 Astra Bill

Cross 272,000 input tokens and Astra reprices the entire request at 2x input and cache rates and 1.5x output. Not the excess — the whole call. The maths, and how to stay under it.

Read more →12 min read
deep-dive

GPT-6 Astra Not Showing Up? Four Reasons, and When You Get It

Astra rolled out in stages, is off by default on Enterprise workspaces, has no free tier, and reports disagree on what Plus actually receives. A checklist for finding out where you stand.

Read more →12 min read
deep-dive

GPT-6 Astra vs Claude: One Model, Two Scores, and What Actually Lost

Astra scored 62.7% and 99.9% on the same benchmark depending on the harness. Here is where OpenAI's model genuinely leads Claude, where it still trails, and how to read either claim.

Read more →13 min read
deep-dive

Claude Fable 5.1 Is Ranked #1. That Means Less Than You Think

Fable 5.1 tops the Artificial Analysis Intelligence Index at 66. That index is an average of nine benchmarks across four categories, and ranking first on the composite is not the same as winning any of them.

Read more →13 min read
deep-dive

GPT-6 Astra vs Hy4 Preview: 12x Cheaper and Apache 2.0

Tencent's Hy4 preview is 770B parameters under Apache 2.0 at $0.83 per million input tokens. Astra is closed at $10. Neither runs on a desktop, and only one lets you own it.

Read more →12 min read
deep-dive

GPT-6 Astra vs GLM-5.3-Flash: 143x Cheaper, and the Weights Are Public

GLM-5.3-Flash costs $0.07 per million input tokens against Astra's $10, ships MIT-licensed weights and carries a larger context window. What $10 actually buys you over $0.07.

Read more →12 min read
deep-dive

GPT-6 Astra vs GLM-5.3: A 7x Price Gap for 1.5 Index Points

GLM-5.3 costs $1.40 per million input tokens against Astra's $10, and scores 59.5 on the Intelligence Index against 61. What that gap is actually buying you.

Read more →12 min read
deep-dive

GPT-6 Astra vs Claude Fable 5.1: Same Price, Different Winner

Both list at $10/$50 per million tokens. Artificial Analysis puts Fable 5.1 ahead on both independent indexes, and its cached input costs a quarter of Astra's. Where Astra still wins.

Read more →12 min read
deep-dive

GPT-6 Astra: 2.5x the Price, the Same Intelligence Score

OpenAI's GPT-6 Astra lists at $10/$50 per million tokens, 2.5x GPT-5.6 Sol, and scores identically on the Intelligence Index. What actually improved, and the 272K repricing cliff nobody mentioned.

Read more →12 min read
deep-dive

Fable 5.1 vs Opus 5: Twice the Price, Behind on Every Published Benchmark

Fable 5.1 costs exactly double Claude Opus 5, scores lower on all three public benchmarks, and Anthropic's own docs say start with Opus 5. When Fable is still the right call.

Read more →12 min read
deep-dive

Fable 5.1 Can Empty a Max Plan in 8 Minutes. Here's What to Run Locally Instead.

Developers reported losing a five-hour window in minutes and burning ~$100 of tokens in one agentic session. Why Fable 5.1 does it, what reduces it, and what runs free on your own GPU.

Read more →12 min read
deep-dive

Fable 5.1 vs Hy4 Preview: Rent It, or Download 770B of It

Tencent's Hy4 preview is 770B parameters under Apache 2.0 with a 1M context, at $0.83 per million input tokens. Fable 5.1 is closed and ~12x dearer. Neither has published benchmarks.

Read more →13 min read
deep-dive

Fable 5.1 vs GLM-5.3-Flash: 143x Cheaper, and You Can Download It

GLM-5.3-Flash costs $0.07 per million input tokens against Fable 5.1's $10, publishes benchmark scores Anthropic has not, and ships downloadable weights. The full comparison.

Read more →12 min read
deep-dive

A 1M-Token Context Costs 147 GB of VRAM. Fable 5.1 Rents It.

Fable 5.1 offers a 1M-token context window. Holding that locally costs about 147 GB of VRAM for an 8B model whose weights are 4.5 GB. The arithmetic, and what your card can really hold.

Read more →10 min read
deep-dive

Fable 5.1 vs Fable 5: What Actually Changed, and What It Costs

Anthropic's Claude Fable 5.1 landed on 1 September 2026 with a 25% price cut and no published benchmarks. Here is what changed, what did not, and what it means if you run models on your own GPU.

Read more →11 min read
deep-dive

vLLM Adds Day-0 Kimi K3 Support: What It Means for Local Serving

vLLM upstreamed Kimi K3's KDA prefix caching for day-0 support as Ollama and llama.cpp shipped fast in July 2026 - what the runtime race means locally.

Read more →10 min read
deep-dive

How to Run DeepSeek-V4-Flash Locally: 284B MoE at ~36 tok/s on 192GB VRAM

DeepSeek-V4-Flash is a 284B MoE with only 13B active params and a 1M context. We map every GGUF quant to real VRAM and clock ~36 tok/s on dual 96GB Blackwell cards.

Read more →12 min read
comparison

GLM-5.2 vs DeepSeek V4 vs Kimi K3: Which Giant MoE Fits Your Hardware

GLM-5.2 needs 239GB even at 2-bit quantization, DeepSeek V4-Flash runs in 128GB, and Kimi K3 has no GGUF yet. Here's what actually fits on a 4x RTX 3090 rig or a maxed-out Mac Studio.

Read more →12 min read
deep-dive

Kimi K3 Is the Largest Open-Weight Model Ever — Can You Run It?

Moonshot's Kimi K3 is a 2.8T open-weight model topping the Frontend Code Arena. Weights land July 27 — here's the honest math on whether you can run it locally.

Read more →10 min read
deep-dive

Inkling: Mira Murati's Lab Ships the Top U.S. Open-Weight Model

Thinking Machines' first model, Inkling, is a 975B open-weight multimodal MoE leading U.S. open releases on the Intelligence Index. What it means for local AI.

Read more →10 min read
comparison

Q4_K_M vs Q5_K_M vs Q6_K vs Q8_0: How Much Quality Do You Lose?

Quantizing Llama-3.1-8B to Q4_K_M cuts the file from 14.96 GB to 4.57 GB — a 69% reduction — while perplexity rises just 0.24 points and MMLU drops about one point. Here is where each GGUF level actually breaks.

Read more →12 min read
hardware

Cloud GPU Pricing in India 2026 — Vast.ai vs RunPod vs Salad vs E2E vs Yotta

Indian devs pay ₹40-180/hr for an A100 depending on provider. We benchmark the 5 major options on price, region latency, billing flexibility, and tax handling.

Read more →8 min read
deep-dive

How Local LLM Inference Actually Works: Loading, Memory, and Quantization Explained

You downloaded a model — now what? A plain-English walkthrough of how inference engines load weights, manage memory with MMAP, and why quantization format matters more than programming language.

Read more →12 min read
local-ai

Run Gemma 4 Locally for Free: Google's Best Open AI Model on Your Own Machine

Google's Gemma 4 runs completely offline — no subscriptions, no data leaving your machine. Here's exactly how to set it up with Ollama in under 10 minutes.

Read more →10 min read
hardware

How Much of a Boost Does TurboQuant Actually Give Your GPU?

TurboQuant compresses the KV cache 4–5×. But the real question is: how does that change the composite score for your specific GPU and model? We ran the numbers. The answer depends on your VRAM.

Read more →6 min read
product

Introducing Runyard Compare: Head-to-Head GPU Benchmarks with TurboQuant

Runyard just shipped a new Compare page — pick any two devices, toggle TurboQuant, and instantly see which GPU wins for every LLM. Here's what's new and why TurboQuant changes the result.

Read more →5 min read
research

How to Run TurboQuant: We Tested Google's Algorithm on a Consumer GPU

We built a ground-up Python implementation of TurboQuant and tested it on Qwen 2.5 3B running on an RTX 3060. 5.3x KV cache compression. 99.5% attention fidelity at 3-bit. Here's exactly what happened.

Read more →12 min read
local-ai

Google TurboQuant: Your Local LLM Just Got a 4x Context Window Upgrade

Google's TurboQuant research (Zandieh et al., ICLR 2026) cuts KV cache memory by 4x — meaning the 7B model on your GPU can now run 32K context where it used to max out at 8K. Here's what changes today.

Read more →8 min read
tutorial

We Built a Hardware MCP Server: Auto-Detect Your GPU and Get the Right LLM

@runyarddev/hw-mcp is a free MCP server that auto-detects your GPU, VRAM, and RAM, then ranks 900+ LLMs by hardware fit — giving any MCP-compatible AI agent instant hardware awareness.

Read more →10 min read
security

LiteLLM Supply Chain Attack: What Happened and How to Fix It

On March 24, 2026, LiteLLM versions 1.82.7 and 1.82.8 were poisoned with a three-stage credential stealer targeting API keys, SSH keys, cloud credentials, and crypto wallets. Here's exactly what happened and what to do now.

Read more →9 min read
security

Have You Been Pawned by LiteLLM?

LiteLLM versions 1.82.7 and 1.82.8 contained a credential-stealing backdoor. Paste your pip freeze output into our checker to find out if your machine was compromised — and what to do next.

Read more →6 min read
guide

Can I Run Image Models Locally on My Computer?

Yes — easier than you think. From LLaVA on an 8GB GPU to MiniCPM-V on a MacBook, every local vision model worth running in 2026 and which fits your hardware.

Read more →9 min read
models

LFM-2.5VL-1.6B: The Vision Model That Runs in Your Browser

LFM-2.5VL-1.6B runs fully in Chrome, Edge, and Firefox via WebGPU — no server, no API key, no data leaving your device. Here's how to use it today.

Read more →10 min read
local-ai

"Claude Code for Free" via OpenRouter — The Real Truth

"Claude Code for free via OpenRouter" is everywhere on YouTube. It's not a lie — but not the full truth. Here's what you're actually getting and the real free alternative.

Read more →8 min read
runyard

How to Use Runyard — Step-by-Step Explainer

Runyard shows which AI models fit your hardware and how fast they run. Complete walkthrough of every feature — from entering GPU specs to clicking Analyze.

Read more →7 min read
guide

Find the Best Local LLM for Your PC (2026 Guide)

Stop downloading models that don't fit. Match an open-source LLM to your GPU, RAM, and use case — then run it in minutes with Ollama or LM Studio.

Read more →10 min read
hardware

How Much VRAM Do You Need to Run Local LLMs?

8GB gets you started, 16GB runs most models comfortably, 24GB+ unlocks the best open-source models. Full breakdown by model size and quantization.

Read more →8 min read
local-ai

Best Local LLMs for Coding in 2026

DeepSeek Coder V2, Qwen2.5 Coder 32B, and CodeLlama 70B lead the pack — but the right choice depends on your VRAM. Full ranked list with benchmarks.

Read more →9 min read
comparison

Ollama vs LM Studio: Which Should You Use in 2026?

Ollama is faster to set up, perfect for developers. LM Studio has a polished GUI and better model discovery. The right choice depends on how you work.

Read more →7 min read
hardware

What LLMs Can You Run with 8GB VRAM?

An RTX 3070, 4060, or any 8GB GPU can run Llama 3.1 8B, Mistral 7B, DeepSeek Coder 6.7B, and more — all fit comfortably at Q4 quantization.

Read more →6 min read
guide

How to Run Llama 3 Locally: Complete Step-by-Step Guide

Meta's Llama 3.1 is one of the best open-source LLMs. This guide walks you through every step — from picking the right size to running it in minutes.

Read more →10 min read
hardware

Best GPU for Running Local LLMs in 2026

RTX 4090 is the gold standard, RTX 4070 Ti the best value, Apple Silicon the wildcard. Every major GPU ranked for local LLM inference with tok/s data.

Read more →9 min read
runyard

Introducing Runyard: Find AI Models for Your Hardware

Runyard tells you exactly which AI models run on your computer and how fast. No more guessing, no more downloading 20GB models that don't fit.

Read more →8 min read