SparsePR method

Executable sparsity from response geometry.

SparsePR builds hardware-ready sparse routes and repairs their output error using information from the same attention call.

One response geometry.
Two coupled stages.

01Sample responses

Evaluate a compact set of query rows exactly against K/V.

→
02Partition support

Form paired K/V groups and response-aligned query groups.

→
03Execute sparse routes

Select and evaluate hardware-ready query-to-K/V cells.

→
04Reconstruct residual

Fit a call-specific correction from the exact probe rows.

RCP

Response-Coupled Partitioning

Sampled query responses define value-aware paired K/V groups. Their centroids provide response coordinates for aligning queries that can efficiently share an executable route.

PFRR

Probe-Fitted Residual Reconstruction

A stratified set of exact query rows exposes the post-softmax residual. SparsePR fits an affine correction in a low-rank probe-residual subspace without updating model parameters.

Why per-query sparsity is not executable sparsity.

Queries that share one route must use the union of their K/V supports. SparsePR groups queries by response similarity to increase support overlap before routing.

Per-query attention support density compared with pooled support density and retained attention mass compared with output error
Per-query concentration does not determine pooled executable support, and retained attention mass does not determine output error.