Star on GitHub

Best Personal AI Hardware

Power efficiency and speed of open-weight models on personal AI machines, measured at the wall.

For one person Buy a DGX Spark and run Qwen3.6 35B A3B. The most efficient of the smarter models we tested, on about — watts.

For a team of coding agents with long prompts Buy a QuietBox 2 and run Qwen3.8 27B. Each agent gets — tokens a second and the quickest first token of any build.

Budget US list prices (MSRP), November 2026. Card prices don't include the computer around them.

Energy efficiency

Output tokens per joule, by model

Power test (1,024 tokens in, 512 out) with 16 users. Wall power from a Shelly plug on each machine, averaged over the run. Further right is more efficient; log scale.

Throughput against wall power

Each mark is one model on one system, 16 users. Diagonals are equal tokens per joule; up and to the left is better.

Tokens per joule by architecture

Every model as a dot, grouped by architecture and system, 16 users. The bar marks each group's median.

Tokens per joule against model size

Each mark is one model on one system, 16 users, placed by the parameters it reads for each token (all of them for a dense model, only the active experts for MoE). Both scales are log.

Running cost

Tokens per month and electricity cost, running flat out

Each bar is the output tokens one system could generate in a 30-day month at the measured throughput with 16 users, running 24 hours a day. The label adds the month's electricity at the measured wall power and the cost per million tokens; the price starts at $0.39/kWh, a California residential time-of-use rate averaged over the whole day. Hardware isn't included.

Speed

Single-user decode speed

Tokens per second one user sees while the reply streams (chat test: 128 tokens in, 256 out). Greedy sampling.

Throughput as users are added

Total output tokens per second across all users (1,024 in, 256 out each).

Long prompts

Coding agents on a large codebase, research over documents and agents with a lot of saved context send long prompts and get short answers back, so processing the prompt (prefill) can take longer than writing the reply. Real coding-agent traffic is longer still: in TraceLab's traces of Claude Code and Codex sessions, a typical step resends about 120K tokens of history, adds under 1K new tokens and writes about 250, and 96% of prompt tokens were already in the server's cache. These charts use one user unless they say otherwise.

Prefill speed

Prompt tokens processed per second for one user: prompt length divided by time to first token. Longer bars are better.

Time for one long-prompt request

How long one user waits for a whole answer with a 32K-token prompt: the solid part is waiting for the first token (prefill), the faded part is writing the 256-token reply. Shorter is better.

Energy per long-prompt request

Wall-power joules per request with a 16K-token prompt and a 256-token reply. Output tokens per joule gives no credit for prompt work, so this counts the whole request. Further left is more efficient; log scale.

Repeated context: fresh prompt vs. cached

An agent resends its whole context every turn. Here each request repeats the same 16K-token history and adds 512 new tokens. A server that caches prompt prefixes only processes the new part, so time to first token drops; one that doesn't processes everything again. Hollow marks are a fresh prompt and filled marks a repeated one, with 1 user. Further left is better; log scale.

Model quality

Tokens per joule counts tokens, not how good they are. Larger and newer models usually score higher on benchmarks, so these charts add each model's Artificial Analysis Intelligence Index (v4.3.2, published scores; higher is better).

Artificial Analysis Intelligence Index

Composite of nine evaluations, for the variant Artificial Analysis scored (mostly reasoning mode). Marked "est." where Artificial Analysis lists an estimate. Dense models in grey, MoE in color. Ornith 1.5 and Qwen3-Coder-Next aren't listed there.

How efficient is each model, best scoring first?

Models ranked by Intelligence Index, highest at the top (grey bar; * is an Artificial Analysis estimate). The dots show tokens per joule on each system with 16 users; further right is cheaper to run. The best trade-offs sit near the top with dots far to the right.

Add your hardware

Star on GitHub Starring the repo helps more people find it.

Have a machine that isn't here? Run the same tests on it and your results show up on this page next to everything else, with the same charts, filters and budget.

What you need

  • The machine, on wall power, with nothing else heavy running. Linux with Docker for NVIDIA and AMD GPUs; macOS for Apple silicon.
  • A Shelly Gen2 or newer smart plug between the machine and the wall, such as the Plug US Gen4 (US, 15 A) or the Plug S MTR Gen3 (EU). The harness reads it once a second over the local network. Optional, but without it there's no tokens per joule or running cost.
  • uv and a clone of the repo.

How

  1. Describe the machine in bench.yaml: its name, price and whether it's a complete system or a card.
  2. Run ./bench.py run <machine> -w power for a few models, then the full set if you have time.
  3. Send the results/ folder in a pull request.

The step-by-step guide covers Macs, NVIDIA and AMD GPUs, and Tenstorrent cards.

Wanted

Reading these results

  • Wall power covers the whole machine: CPU, memory, fans and power-supply losses as well as the accelerators. QuietBox 2 draws several hundred watts even when idle, so its efficiency mostly follows its speed.
  • On the Sparks, the dense models (Qwen3.6 27B, Devstral, Gemma 4, Muse Glimmer) ran in BF16. An FP8 or NVFP4 build would read fewer bytes per token and likely be faster and more efficient there.
  • QuietBox 2 runs tt serve bundles; the Sparks run vLLM in NVIDIA's container; Macs run 4-bit MLX builds with mlx_lm.server. Same models, different builds, precisions and kernels.
  • The Intelligence Index scores come from Artificial Analysis’s published runs, not from these machines. Most are for reasoning mode, which writes more tokens per answer, so energy per answer is higher than energy per token suggests. These benchmarks used a fixed output length.
  • DeepSeek V4.1 Flash on QuietBox 2 and Gemma 4 with many users look slower than they should be and are under investigation.
  • Two DGX Sparks run one copy of gpt-oss-120b split across both machines over the direct QSFP link, and both plugs are summed.
  • The default QuietBox 2 builds of gpt-oss-20b and Qwen3.6 35B A3B run on one Blackhole chip; their 4-chip builds spread the same model over all four.
  • Tenstorrent p150 is one p150a card (one Blackhole chip, 32 GB) in a desktop with a 6-core Ryzen 5 5500 on PCIe 3.0, running the same gpt-oss-20b and Qwen3.6 35B A3B bundles as QuietBox 2's single-chip builds on Talos Linux and Kubernetes (Tenstorrent's tt-operator) instead of Ubuntu. On QuietBox 2 that stack matched Ubuntu on decode speed and power under load. The p150's wall power is from the power test only, measured with the operator's telemetry collector off; the box idles at 98 W. Decode speed matches a QuietBox 2 chip, at about half the wall power. Qwen3.6 was only run with one user.
  • Ornith 1.5 35B A3B, Qwen3-Coder 30B A3B and Qwen3.5 122B A10B on QuietBox 2 are the community bundles' 2026-10-27 rebuilds, which fuse each decode step into a few large kernels; Qwen3-Coder-Next 80B A3B is new with them; in BF16 it doesn't fit one DGX Spark, so the Spark runs Qwen's official FP8 build (vLLM 26.04 on Talos Linux and Kubernetes). They ran on Talos Linux and Kubernetes with wall power from the power test only (telemetry collector off). Against the first builds on the same machine, one user went from 80 to 200 tok/s on Ornith, 49 to 93 on Qwen3-Coder and 51 to 130 on Qwen3.5 122B, and 32 users from 419 to 858, 648 to 882 and 233 to 462 tok/s total. With these builds a cached 16K-token prompt starts no sooner than an uncached one. The first builds' results stay in the repo as results/qb2/<model>-v1.
  • Qwen3 8B is the small-model comparison: the same 8B dense model on four single-accelerator machines, all on Talos Linux and Kubernetes. Tenstorrent n300 (two Wormhole chips, 24 GB) and Quadro RTX 6000 (Turing, 24 GB) sit in the same Ryzen 5 5500 desktop as the p150. The Tenstorrent cards run Tenstorrent's tt-transformers build through vLLM; there is no official p150 entry for this model, so the p150 ran the n300 entry adapted for one Blackhole chip. The RTX 6000 runs vLLM 26.04 in fp16 (Turing has no bf16), which leaves room in 24 GB for only one 40K-token request at a time; the Spark runs vLLM 26.04 in bf16. Wall power for the Tenstorrent cards is from the power test only, with the telemetry collector off; the RTX 6000 and Spark have wall power for every test. The n300 is missing the 32K-token test, and 32 of its 96 requests failed with 32 users in the users test.