Why Chutes’ Input Caching Could Matter More Than It Sounds

Most teams running production AI workloads are paying full price for the exact same tokens, request after request, without realizing how much of their bill is repetition. 

A 4,000-token system prompt sent across 10,000 daily requests means 40 million identical tokens hitting the API every day, all billed at the standard input rate.

Chutes (SN64) just rolled out input caching that cuts the price on every repeated token in half, with no flags to set and no code changes required.

The Math, in Plain Numbers

The savings stack up faster than most teams expect. Using Kimi K2.6 TEE at $0.74 per million input tokens as the benchmark:

a. Without caching: 40M tokens per day costs $29.60.

b. With caching: the same workload drops to $14.80 per day.

c. That is $444 per month saved on a single system prompt alone.

What qualifies for the cached rate goes well beyond system prompts:

a. System prompts sent on every request.

b. Conversation history carried forward between turns.

c. Few-shot examples repeated across calls.

d. RAG (Retrieval-Augmented Generation) preambles appended to retrieved context.

e. Anything sent twice gets the cached rate automatically.

The cache hits whenever content repeats across requests, which means the savings compound across every layer of the stack a team is already running.

What This Actually Proves

The interesting thing about Chutes’ caching rollout is that it removes one of the standard reasons developers default to centralized inference providers in the first place: the assumption that decentralized infrastructure cannot match the operational efficiencies of established players on pricing.

Cutting repeated input tokens in half is exactly the kind of margin work that distinguishes serious inference infrastructure from a research demo, and it is shipping on a Bittensor subnet rather than on Together AI or Fireworks.

The question every team running production workloads should be asking is whether they are still paying full price for tokens they have sent before, because the answer is no longer a fixed cost of doing business.

Explore Chutes’ Documentation for More on This.