Subscribe
Inference Speed Lab
Projects Inference Speed Lab

Inference speed demo

Inference Speed Lab

A small interactive demo for feeling how near-1,000-token-per-second inference changes agent loops and human attention.

Inference Speed Lab is a small simulator for feeling the difference between a model that can generate close to 1,000 output tokens per second and the slower frontier reasoning loops most AI products were designed around.

It is built for the Cerebras essay research. Speed does not replace judgement, and one model release is not the whole story. Speed changes the shape of the interaction: what can stay synchronous, what needs a queue, and how many drafts, checks, and repairs can fit inside one attention window.

What it demonstrates

The app starts with a Cerebras Kimi K2.6 lane set to 981 output tokens per second, based on Cerebras' enterprise-trial measurement. Kimi is the measurement anchor here, not the whole argument. OpenAI separately announced GPT-5.6 Sol on Cerebras at up to 750 tokens per second for selected customers. The comparison lane is intentionally adjustable because GPT-5.6, Claude, and other frontier systems vary by effort level, serving route, provider, and workload.

Why the wait changes the work

Slow inference pushed AI products toward progress spinners, background jobs, parallel agent runs, and streaming interfaces that exist mostly to hide latency.

Fast inference changes that default. A short answer may not need streaming. A multi-step agent loop may fit inside the request path. A critic pass can become part of the same turn instead of a separate task.

How to read the numbers

The demo is a timing model, not a benchmark. It uses output-token speed to make the waiting time visible, then adds a separate agent-loop calculator for repeated model calls plus non-model overhead.

The sourced hard number in the simulator is Cerebras' Kimi K2.6 result. The GPT-5.6 figure is an announced upper rate for a separate selected-customer route, not the simulator's measured anchor. The comparison lane is a practical baseline you can move around.