Author

Runyard Team writes for Runyard about local AI, GPU hardware, and the practical side of running language models on your own machine.

Posts by Runyard Team

• Can You Run DeepSeek V4.1-Flash Locally? Not Yet

  614 GB of VRAM to serve it, a 511 GB checkpoint, no Unsloth guide and not one tested GGUF. Here is the actual gap, and what to run tonight instead.

  2026-09-10 · 9 min read

• DeepSeek V4.1-Flash: 552B Parameters, 8 Doing the Work

  An encoder-decoder split, 6 of 384 experts firing per token, and a KV cache of 890 bytes — which is what makes a million-token context affordable. What DeepSeek actually shipped, from the model card.

  2026-09-10 · 10 min read

• AlphaGenome Atlas: 9 Billion Answers, Precomputed

  DeepMind ran a model across every possible single-letter change to human DNA and published the results — a petabyte of them. The reason it worked is the reason it cannot work for a language model.

  2026-09-10 · 10 min read

• Siri AI Not Showing Up on iOS 27? Six Things to Check

  Most cases are not a fault. They are a language setting, a staged rollout queue, a region, or a device that was never going to get it — in that order of likelihood.

  2026-09-09 · 8 min read

• iPhone 18 Pro vs 17 Pro for AI: Skip It

  Both clear Apple's 12GB memory tier, so both get every Siri AI feature. The A20 Pro's 50% more bandwidth makes the same models faster — it does not make bigger ones possible.

  2026-09-09 · 8 min read

• Can You Run a Local LLM on an iPhone? Yes, With Limits

  The apps are free and they work offline. The constraint is memory, not software — about 3B parameters on any iPhone, and roughly 10-15 tokens per second on a recent Pro.

  2026-09-09 · 9 min read

• Siri AI Isn't Launching in the EU. Here's Where That Leaves You

  Apple's own wording: Siri AI will not be initially available in the EU on iOS, iPadOS or watchOS. What that covers, why it happened, and what actually works instead.

  2026-09-09 · 7 min read

• Which iPhones Get Every Siri AI Feature, and Why

  Apple has published a memory requirement for the first time: at least 12GB of unified memory. That one number explains the whole three-tier split across the iPhone line.

  2026-09-09 · 9 min read

• OpenAI Solved Navier-Stokes. Read the Compute Bill

  10,000 agents, 88 hours and somewhere between $2M and $22.5M produced the proof. Verifying it took 17 hours. That gap is the real story — and it is the same one that decides what a local model is good for.

  2026-09-09 · 10 min read

• iPhone Duo vs Galaxy Z Fold8 Ultra, Feature by Feature

  Apple's first foldable against Samsung's eighth. Displays, cameras, build, memory, battery, software and price compared line by line — including the one-character spec difference that matters most.

  2026-09-09 · 10 min read

• iPhone Duo: Price, Specs and Release Date Explained

  Apple's first foldable opens to 7.6 inches, closes to 5.4, starts at $1,999 and ships 23 October. Every figure Apple published - displays, hinge, titanium, cameras, Split View, battery - read plainly.

  2026-09-09 · 7 min read

• Apple Put Three AI Features on a Watch. What Runs There?

  Series 12 brings Audio Intelligence, Siri Recap and Siri AI. A watch holds a model measured in hundreds of megabytes — so the interesting question is which of these actually executes on your wrist.

  2026-09-09 · 8 min read

• The iPhone Duo Folds. Its AI Ceiling Doesn't Move

  Apple's first foldable is $1,999 with up to 2 TB of storage. Storage is not memory — and memory is what decides model size. The biggest iPhone ever made runs the same on-device models as the smallest.

  2026-09-09 · 8 min read

• What 50% More Memory Bandwidth Actually Buys On-Device

  Apple gave a ratio, not a figure in GB/s. Worked through the arithmetic, 1.5x the bandwidth is about 1.5x the tokens per second on the same model — and no change at all to how large a model fits.

  2026-09-09 · 8 min read

• Apple Just Put the Number We Said They'd Hide On Stage

  We predicted Apple would talk neural-engine figures and stay quiet about memory bandwidth. They led with it: 50% more than A19 Pro, and the widest memory interface ever shipped in an iPhone.

  2026-09-09 · 8 min read

• Model Download Keeps Failing? Stop Using the Browser

  A 70B model is a 40 GB download. Browser downloads cannot resume, so every drop starts from zero. The command-line tools can, and that is the whole fix.

  2026-09-08 · 6 min read

• Ollama vs LM Studio vs llama.cpp: Which Should You Use?

  All three run the same engine underneath, so you are not choosing performance — you are choosing an interface. Here is which one fits which way of working.

  2026-09-08 · 7 min read

• Your GPU Is at 5% While Generating. That's Correct

  Low GPU utilisation during text generation is not a fault. Decoding is memory-bound — the cores genuinely are waiting on RAM, and no setting you change will make them busy.

  2026-09-08 · 7 min read

• 770B Parameters Under Apache 2.0. What Does It Take to Run?

  Hy4 is 770B total and 49B active, openly licensed. At Q4 that is 433 GB of weights — more than any consumer machine holds. At Q3 a 512 GB Mac Studio just about does.

  2026-09-08 · 8 min read

• Qwen3.8-27B's KV Cache Is 4x Smaller Than the Formula Says

  Only 16 of its 64 layers keep a KV cache. That is why people run 160K context on a single 24 GB card — and why our own earlier numbers for this model were too conservative.

  2026-09-08 · 8 min read

• The iPhone 18 Pro Spec Apple Won't Mention On Stage

  Two iOS 27 AI features are reported to need 12 GB of RAM, with no cloud fallback. Apple never says RAM in a keynote — but it is the number that decides which iPhones get the features.

  2026-09-06 · 8 min read

• Apple Does Not Need the Best Model. It Needs About 3B Parameters

  Apple's bet is that a good-enough model plus integration beats the best model. A phone gives an on-device model 2-3 GB — about a 3B model. That ceiling is exactly why everything else has to win.

  2026-09-06 · 9 min read

• The Model Sizes Nobody Should Buy Hardware For

  A 49B needs 32.0 GB and misses a 32 GB card. A 70B needs 44.5 and misses two 24 GB cards. Between one big card and unified memory there is a gap nothing fills well.

  2026-09-06 · 8 min read

• We Checked Every 'Runs on 8GB' Claim. It Breaks at 9B

  An 8 GB card gives 7.2 GB usable. An 8B model needs 6.9 and fits. A 9B needs 7.5 and misses. The claim is true right up to the point it silently is not.

  2026-09-06 · 8 min read

• Astra Scored 100% on ExploitBench. That Is a Risk Disclosure

  GPT-6 Astra is the first model OpenAI classes as crossing its Critical threshold for cyber capability. The same document calls its own monitoring fragile.

  2026-09-06 · 7 min read

• GPT-6 Astra: Who Actually Has Access, and What the Limits Are

  Astra launched on 3 September to vetted organisations only. If you have a paid plan and cannot see it, that is the rollout working as designed — not an outage.

  2026-09-06 · 6 min read

• Ollama Ignoring num_ctx? What Is Actually Happening

  Ollama has historically defaulted to a small context and truncates silently. Raising it works — but past 8192 on a 12 GB card you start losing layers to the CPU.

  2026-09-06 · 7 min read

• LM Studio Will Not Load the Model: What Each Error Means

  Almost every load failure is capacity, and the message rarely says so. A 12 GB and a 16 GB card both top out at 14B at Q4_K_M — the tier does not move.

  2026-09-06 · 7 min read

• Why Your Local Model Gets Slower the Longer You Chat

  The weights never move. The KV cache does — from 0.22 GB at a thousand tokens to 3.59 GB at sixteen thousand on a 14B model. That growth is the slowdown.

  2026-09-06 · 7 min read

• CUDA Out of Memory: The Five Causes, in Order

  A 14B model at Q4_K_M fits a 12 GB card by 0.09 GB. That margin is why the same setup loads on Monday and crashes on Tuesday — and which fix frees the most.

  2026-09-06 · 8 min read

• Ollama Is Using CPU Instead of GPU: The Real Cause

  It is almost never a driver problem. A 14B model holds all 40 layers on a 12 GB card at num_ctx 8192 and drops to 31 at 16384 — that is where the CPU offload starts.

  2026-09-06 · 7 min read

• Have We Reached AGI With GPT-6 Astra? The 37-Point Gap Says No

  OpenAI reports 99.9% on ARC-AGI. ARC Prize's provider-neutral harness scored the same model at 62.7%. Both numbers are real — and the 37 points between them were contributed by the scaffolding, not the intelligence.

  2026-09-06 · 8 min read

• A 27B Model at 30 Tokens a Second, No GPU — We Checked What That Requires

  Alibaba says its RISC-V C950 ran Qwen3.8-27B at 30 tokens a second with no graphics card. Nobody inverted the number. It implies about 570 GB/s of memory bandwidth — server territory, and six times what a desktop has.

  2026-09-05 · 9 min read

• Local LLM vs Claude, GPT and Gemini: Which API Is Worth Replacing?

  Claude Fable 5.1 and GPT-6 Astra cost the same to the cent. Gemini 3.8 Flash is thirteen times cheaper. A $900 card pays back in 101 days against one and 4.2 years against the other — the tier decides, not the brand.

  2026-09-05 · 11 min read

• Local LLM vs the OpenAI API: What It Actually Costs to Self-Host

  Electricity is $0.18 per million tokens. All-in it is $0.93, and against cheap hosted models a GPU never pays for itself. Here is the comparison done honestly, with the arithmetic shown.

  2026-09-05 · 10 min read

• Mac or Nvidia for Local AI in 2026? The Answer Changed This Year

  An RTX 5090 costs $4,699 and holds 28.8 GB. A Mac Studio M5 Ultra costs $5,499 and holds 72 GB at 1.2 TB/s. On models both can run the card is 1.5x faster — not 4x. Here is where each one actually wins.

  2026-09-05 · 10 min read

• Qwen3.8-27B Does Not Need 17 GB. Here Is What It Actually Needs

  The 17 GB figure is a file size, not a memory requirement. Add the KV cache and runtime overhead and Q4_K_M needs about 20.7 GB — which is why 16 GB cards get 5 tokens per second instead of 50.

  2026-09-05 · 9 min read

• The GPU You Were Going to Buy Now Costs $163 a Gigabyte

  An RTX 5090 launched at $1,999 and sells for $4,699. We recomputed price per usable gigabyte across the catalogue — the flagship is now the worst value in it, and a 128GB Mac costs the same as one card.

  2026-09-05 · 11 min read

• What Runs Locally in 2026: The 8, 12, 16, 24 and 32 GB Tier List

  Computed from our catalogue, not estimated: 22 models fit 8 GB, 63 fit 24 GB, and the jump from 16 GB to 24 GB more than doubles the quality you can reach.

  2026-09-04 · 12 min read

• Claude Code vs Codex vs Cursor: The Harness Is Worth More Than the Model

  The same model scored 62.7% and 99.9% on the same benchmark, depending only on the harness around it. Why your agent wrapper matters more than your model choice.

  2026-09-04 · 12 min read

• Is GPT-6 Astra Down? How to Tell in 60 Seconds

  A 429 is usually you, a 503 is usually them, and a green status page proves neither. How to diagnose an Astra outage properly, and what to do while you wait.

  2026-09-04 · 12 min read

• Cost Per Task Beats Cost Per Token. Here's How to Actually Measure It

  A model at $50 per million output tokens can be cheaper than one at $20 if it finishes in fewer tokens and fails less often. The three numbers that decide your real bill.

  2026-09-04 · 12 min read

• The 272K Token Cliff: How One Extra Token Doubles Your GPT-6 Astra Bill

  Cross 272,000 input tokens and Astra reprices the entire request at 2x input and cache rates and 1.5x output. Not the excess — the whole call. The maths, and how to stay under it.

  2026-09-04 · 12 min read

• GPT-6 Astra Not Showing Up? Four Reasons, and When You Get It

  Astra rolled out in stages, is off by default on Enterprise workspaces, has no free tier, and reports disagree on what Plus actually receives. A checklist for finding out where you stand.

  2026-09-04 · 12 min read

• GPT-6 Astra vs Claude: One Model, Two Scores, and What Actually Lost

  Astra scored 62.7% and 99.9% on the same benchmark depending on the harness. Here is where OpenAI's model genuinely leads Claude, where it still trails, and how to read either claim.

  2026-09-04 · 13 min read

• Claude Fable 5.1 Is Ranked #1. That Means Less Than You Think

  Fable 5.1 tops the Artificial Analysis Intelligence Index at 66. That index is an average of nine benchmarks across four categories, and ranking first on the composite is not the same as winning any of them.

  2026-09-04 · 13 min read

• GPT-6 Astra vs Hy4 Preview: 12x Cheaper and Apache 2.0

  Tencent's Hy4 preview is 770B parameters under Apache 2.0 at $0.83 per million input tokens. Astra is closed at $10. Neither runs on a desktop, and only one lets you own it.

  2026-09-04 · 12 min read

• GPT-6 Astra vs GLM-5.3-Flash: 143x Cheaper, and the Weights Are Public

  GLM-5.3-Flash costs $0.07 per million input tokens against Astra's $10, ships MIT-licensed weights and carries a larger context window. What $10 actually buys you over $0.07.

  2026-09-04 · 12 min read

• GPT-6 Astra vs GLM-5.3: A 7x Price Gap for 1.5 Index Points

  GLM-5.3 costs $1.40 per million input tokens against Astra's $10, and scores 59.5 on the Intelligence Index against 61. What that gap is actually buying you.

  2026-09-04 · 12 min read

• GPT-6 Astra vs Fable 5.1: Same Price, Different Winner

  Both list at $10/$50 per million. Fable 5.1 leads on both independent indexes and its cached input costs a quarter as much. Where Astra still wins.

  2026-09-04 · 12 min read

• GPT-6 Astra: What Changed, and Why It Costs 2.5x

  Astra lists at $10/$50 per million, 2.5x Sol, and scores the same on the Intelligence Index. What improved, and what the 272K repricing does.

  2026-09-04 · 12 min read

• Fable 5.1 vs Opus 5: Twice the Price, Behind on Every Published Benchmark

  Fable 5.1 costs exactly double Claude Opus 5, scores lower on all three public benchmarks, and Anthropic's own docs say start with Opus 5. When Fable is still the right call.

  2026-09-01 · 12 min read

• Fable 5.1 Can Empty a Max Plan in 8 Minutes. Here's What to Run Locally Instead.

  Developers reported losing a five-hour window in minutes and burning ~$100 of tokens in one agentic session. Why Fable 5.1 does it, what reduces it, and what runs free on your own GPU.

  2026-09-01 · 12 min read

• Fable 5.1 vs Hy4 Preview: Rent It, or Download 770B of It

  Tencent's Hy4 preview is 770B parameters under Apache 2.0 with a 1M context, at $0.83 per million input tokens. Fable 5.1 is closed and ~12x dearer. Neither has published benchmarks.

  2026-09-01 · 13 min read

• Fable 5.1 vs GLM-5.3-Flash: 143x Cheaper

  GLM-5.3-Flash is $0.07 per million input against Fable 5.1's $10, publishes benchmarks Anthropic has not, and ships downloadable weights.

  2026-09-01 · 12 min read

• A 1M-Token Context Costs 147 GB of VRAM. Fable 5.1 Rents It.

  Fable 5.1 offers a 1M-token context window. Holding that locally costs about 147 GB of VRAM for an 8B model whose weights are 4.5 GB. The arithmetic, and what your card can really hold.

  2026-09-01 · 10 min read

• Fable 5.1 vs Fable 5: What Changed and What It Costs

  Fable 5.1 landed on 1 September with a 25% price cut and no published benchmarks. What changed, what did not, and what it means if you build on it.

  2026-09-01 · 11 min read

• vLLM Adds Day-0 Kimi K3 Support: What It Means for Local Serving

  vLLM upstreamed Kimi K3's KDA prefix caching for day-0 support as Ollama and llama.cpp shipped fast in July 2026 - what the runtime race means locally.

  2026-07-21 · 10 min read

• How to Run DeepSeek-V4-Flash Locally: 284B MoE at ~36 tok/s on 192GB VRAM

  DeepSeek-V4-Flash is a 284B MoE with only 13B active params and a 1M context. We map every GGUF quant to real VRAM and clock ~36 tok/s on dual 96GB Blackwell cards.

  2026-07-21 · 12 min read

• GLM-5.2 vs DeepSeek V4 vs Kimi K3: Which Giant MoE Fits Your Hardware

  GLM-5.2 needs 239GB even at 2-bit quantization, DeepSeek V4-Flash runs in 128GB, and Kimi K3 has no GGUF yet. Here's what actually fits on a 4x RTX 3090 rig or a maxed-out Mac Studio.

  2026-07-21 · 12 min read

• Kimi K3 Is the Largest Open-Weight Model Ever — Can You Run It?

  Moonshot's Kimi K3 is a 2.8T open-weight model topping the Frontend Code Arena. Weights land July 27 — here's the honest math on whether you can run it locally.

  2026-07-21 · 10 min read

• Inkling: Mira Murati's Lab Ships the Top U.S. Open-Weight Model

  Thinking Machines' first model, Inkling, is a 975B open-weight multimodal MoE leading U.S. open releases on the Intelligence Index. What it means for local AI.

  2026-07-20 · 10 min read

• Q4_K_M vs Q5_K_M vs Q6_K vs Q8_0: How Much Quality Do You Lose?

  Quantizing Llama-3.1-8B to Q4_K_M cuts the file from 14.96 GB to 4.57 GB — a 69% reduction — while perplexity rises just 0.24 points and MMLU drops about one point. Here is where each GGUF level actually breaks.

  2026-06-18 · 12 min read

• Cloud GPU Pricing in India 2026 — Vast.ai vs RunPod vs Salad vs E2E vs Yotta

  Indian devs pay ₹40-180/hr for an A100 depending on provider. We benchmark the 5 major options on price, region latency, billing flexibility, and tax handling.

  2026-05-17 · 8 min read

• How Local LLM Inference Actually Works: Loading, Memory, and Quantization Explained

  You downloaded a model — now what? A plain-English walkthrough of how inference engines load weights, manage memory with MMAP, and why quantization format matters more than programming language.

  2026-04-29 · 12 min read

• Run Gemma 4 Locally for Free: Google's Best Open AI Model on Your Own Machine

  Google's Gemma 4 runs completely offline — no subscriptions, no data leaving your machine. Here's exactly how to set it up with Ollama in under 10 minutes.

  2026-04-07 · 10 min read

• How Much of a Boost Does TurboQuant Actually Give Your GPU?

  TurboQuant compresses the KV cache 4–5×. But the real question is: how does that change the composite score for your specific GPU and model? We ran the numbers. The answer depends on your VRAM.

  2026-03-31 · 6 min read

• Introducing Runyard Compare: Head-to-Head GPU Benchmarks with TurboQuant

  Runyard just shipped a new Compare page — pick any two devices, toggle TurboQuant, and instantly see which GPU wins for every LLM. Here's what's new and why TurboQuant changes the result.

  2026-03-31 · 5 min read

• How to Run TurboQuant: We Tested Google's Algorithm on a Consumer GPU

  We built a ground-up Python implementation of TurboQuant and tested it on Qwen 2.5 3B running on an RTX 3060. 5.3x KV cache compression. 99.5% attention fidelity at 3-bit. Here's exactly what happened.

  2026-03-31 · 12 min read

• Google TurboQuant: Your Local LLM Just Got a 4x Context Window Upgrade

  Google's TurboQuant research (Zandieh et al., ICLR 2026) cuts KV cache memory by 4x — meaning the 7B model on your GPU can now run 32K context where it used to max out at 8K. Here's what changes today.

  2026-03-27 · 8 min read

• We Built a Hardware MCP Server: Auto-Detect Your GPU and Get the Right LLM

  @runyarddev/hw-mcp is a free MCP server that auto-detects your GPU, VRAM, and RAM, then ranks 900+ LLMs by hardware fit — giving any MCP-compatible AI agent instant hardware awareness.

  2026-03-27 · 10 min read

• LiteLLM Supply Chain Attack: What Happened and How to Fix It

  On March 24, 2026, LiteLLM versions 1.82.7 and 1.82.8 were poisoned with a three-stage credential stealer targeting API keys, SSH keys, cloud credentials, and crypto wallets. Here's exactly what happened and what to do now.

  2026-03-25 · 9 min read

• Have You Been Pawned by LiteLLM?

  LiteLLM versions 1.82.7 and 1.82.8 contained a credential-stealing backdoor. Paste your pip freeze output into our checker to find out if your machine was compromised — and what to do next.

  2026-03-25 · 6 min read

• Can I Run Image Models Locally on My Computer?

  Yes — easier than you think. From LLaVA on an 8GB GPU to MiniCPM-V on a MacBook, every local vision model worth running in 2026 and which fits your hardware.

  2026-03-25 · 9 min read

• LFM-2.5VL-1.6B: The Vision Model That Runs in Your Browser

  LFM-2.5VL-1.6B runs fully in Chrome, Edge, and Firefox via WebGPU — no server, no API key, no data leaving your device. Here's how to use it today.

  2026-03-25 · 10 min read

• "Claude Code for Free" via OpenRouter — The Real Truth

  "Claude Code for free via OpenRouter" is everywhere on YouTube. It's not a lie — but not the full truth. Here's what you're actually getting and the real free alternative.

  2026-03-24 · 8 min read

• How to Use Runyard — Step-by-Step Explainer

  Runyard shows which AI models fit your hardware and how fast they run. Complete walkthrough of every feature — from entering GPU specs to clicking Analyze.

  2026-03-24 · 7 min read

• Find the Best Local LLM for Your PC (2026 Guide)

  Stop downloading models that don't fit. Match an open-source LLM to your GPU, RAM, and use case — then run it in minutes with Ollama or LM Studio.

  2026-03-24 · 10 min read

• How Much VRAM Do You Need for Local LLMs? (2026)

  8GB runs 7B models, 16GB runs 14B, 24GB unlocks 32B. Exact figures by model size and quantisation, computed rather than guessed.

  2026-03-18 · 8 min read

• Best Local LLMs for Coding in 2026

  DeepSeek Coder V2, Qwen2.5 Coder 32B, and CodeLlama 70B lead the pack — but the right choice depends on your VRAM. Full ranked list with benchmarks.

  2026-03-15 · 9 min read

• Ollama vs LM Studio: Which Should You Use in 2026?

  Ollama is faster to set up, perfect for developers. LM Studio has a polished GUI and better model discovery. The right choice depends on how you work.

  2026-03-12 · 7 min read

• What LLMs Can You Run with 8GB VRAM?

  An RTX 3070, 4060, or any 8GB GPU can run Llama 3.1 8B, Mistral 7B, DeepSeek Coder 6.7B, and more — all fit comfortably at Q4 quantization.

  2026-03-10 · 6 min read

• How to Run Llama 3 Locally: Complete Step-by-Step Guide

  Meta's Llama 3.1 is one of the best open-source LLMs. This guide walks you through every step — from picking the right size to running it in minutes.

  2026-03-08 · 10 min read

• Best GPU for Running Local LLMs in 2026

  RTX 4090 is the gold standard, RTX 4070 Ti the best value, Apple Silicon the wildcard. Every major GPU ranked for local LLM inference with tok/s data.

  2026-03-01 · 9 min read

• Introducing Runyard: Find AI Models for Your Hardware

  Runyard tells you exactly which AI models run on your computer and how fast. No more guessing, no more downloading 20GB models that don't fit.

  2026-02-28 · 8 min read