Per joule of wall power, one DGX Spark produced a median —× the tokens QuietBox 2 did with 16 users, and —× with one user. QuietBox 2 was faster: a median —× the tokens per second with 16 users and —× with one, over the models that ran on both.
Power test (1,024 tokens in, 512 out) with 16 users. Wall power from a Shelly plug on each machine, averaged over the run. Further right is more efficient; log scale.
Each mark is one model on one system, 16 users. Diagonals are equal tokens per joule; up and to the left is better.
Every model as a dot, grouped by architecture and system, 16 users. The bar marks each group's median.
Each mark is one model on one system, 16 users, placed by the parameters it reads for each token (all of them for a dense model, only the active experts for MoE). Both scales are log.
Each bar is the output tokens one system could generate in a 30-day month at the measured throughput with 16 users, running 24 hours a day. The label adds the month's electricity at the measured wall power and the cost per million tokens; the price starts at $0.39/kWh, a California residential time-of-use rate averaged over the whole day. Hardware isn't included.
Tokens per joule counts tokens, not how good they are. Larger and newer models usually score higher on benchmarks, so these charts add each model's Artificial Analysis Intelligence Index (v4.3.2, published scores; higher is better).
Composite of nine evaluations, for the variant Artificial Analysis scored (mostly reasoning mode). Marked "est." where Artificial Analysis lists an estimate. Dense models in grey, MoE in color. Ornith 1.5 isn't listed there.
Models ranked by Intelligence Index, highest at the top (grey bar; * is an Artificial Analysis estimate). The dots show tokens per joule on each system with 16 users; further right is cheaper to run. The best trade-offs sit near the top with dots far to the right.
Tokens per second one user sees while the reply streams (chat test: 128 tokens in, 256 out). Greedy sampling.
Total output tokens per second across all users (1,024 in, 256 out each).
Coding agents on a large codebase, research over documents and agents with a lot of saved context send long prompts and get short answers back, so processing the prompt (prefill) can take longer than writing the reply. Real coding-agent traffic is longer still: in TraceLab's traces of Claude Code and Codex sessions, a typical step resends about 120K tokens of history, adds under 1K new tokens and writes about 250, and 96% of prompt tokens were already in the server's cache. These charts use one user unless they say otherwise.
Prompt tokens processed per second for one user: prompt length divided by time to first token. Longer bars are better.
How long one user waits for a whole answer with a 32K-token prompt: the solid part is waiting for the first token (prefill), the faded part is writing the 256-token reply. Shorter is better.
Wall-power joules per request with a 16K-token prompt and a 256-token reply. Output tokens per joule gives no credit for prompt work, so this counts the whole request. Further left is more efficient; log scale.
This test is queued; results appear here when it finishes.
An agent resends its whole context every turn. Here each request repeats the same 16K-token history and adds 512 new tokens. A server that caches prompt prefixes only processes the new part, so time to first token drops; one that doesn't processes everything again. Hollow marks are a fresh prompt and filled marks a repeated one, with 1 user. Further left is better; log scale.
This test is queued; results appear here when it finishes.