Coarse-Graining

Coarse-graining reduces the size of a trained reservoir by merging neurons that perform redundant computation. For avalanche-critical reservoirs, the compressed network preserves the original dynamics and can be used with the same readout weights without retraining.

When to Use Coarse-Graining

Coarse-graining is most useful when:

You want to reduce inference cost after training a large reservoir
Your reservoir is avalanche-critical (branching ratio $\sigma \approx 1$ ), in which case the transferred readout weights may work directly
You want to compare a coarse-grained small reservoir against a fresh small reservoir of the same size to verify that the weight structure carries useful structure

For non-critical reservoirs, the merged dynamics differ from the original and retraining is required. The compressed reservoir still trains to a useful solution, but offers no structural advantage over a fresh reservoir of the same size.

Workflow

1. Train a large reservoir


spires_reservoir *r = NULL;
spires_reservoir_create(&cfg, &r);
spires_train_ridge(r, input, target, T, 1e-6);

2. Compute a threshold from the weight distribution

Use a high percentile of the positive weights — the top 1% is a good starting point. This avoids cascade collapse, where merging a few pairs triggers a chain reaction that reduces the reservoir to almost nothing.


static int cmp_asc(const void *a, const void *b) {
    double da = *(const double *)a, db = *(const double *)b;
    return (da > db) - (da < db);
}
 
double percentile_threshold(const double *W, size_t n2, double top_frac) {
    double *pos = malloc(n2 * sizeof(double));
    size_t count = 0;
    for (size_t i = 0; i < n2; i++)
        if (W[i] > 0.0) pos[count++] = W[i];
    qsort(pos, count, sizeof(double), cmp_asc);
    size_t idx = (size_t)((1.0 - top_frac) * count);
    if (idx >= count) idx = count - 1;
    double thresh = pos[idx];
    free(pos);
    return thresh;
}
 
size_t N = spires_num_neurons(r);
double *W = spires_copy_weights(r);
double threshold = percentile_threshold(W, N * N, 0.01); /* top 1% */
free(W);

3. Coarse-grain


spires_reservoir *r_cg = NULL;
spires_coarse_grain(r, threshold, &r_cg);
 
printf("%zu -> %zu neurons\n", N, spires_num_neurons(r_cg));

4. Evaluate or retrain

The coarse-grained reservoir inherits adjusted W_out weights. For a critical reservoir, evaluate directly:


/* Evaluate with transferred W_out */
spires_reservoir_reset(r_cg);
for (size_t t = 0; t < test_len; t++) {
    spires_step(r_cg, &test_input[t]);
    double y;
    spires_compute_output(r_cg, &y);
}

Or retrain to adapt W_out to the new network size:


spires_train_ridge(r_cg, input, target, T, 1e-6);

Threshold Selection

The weight_threshold parameter controls how aggressively the network is compressed. The merge condition is:

$W_{ij} > w_{th} \quad \text{or} \quad W_{ji} > w_{th}$

A percentile of the positive weight distribution is a robust way to set the threshold — for example, the 99th percentile (top 1%) merges only the strongest connections while leaving the rest of the network intact.

Criticality and the RG Fixed Point

The theory behind coarse-graining is strongest when the reservoir is avalanche-critical — when neuronal avalanches (bursts of activity initiated by a single spike) follow a power-law size distribution. This occurs when the branching ratio $\sigma \approx 1$ :

$\sigma = \frac{\sum_{t:\, s(t) > 0} s(t+1)}{\sum_{t:\, s(t) > 0} s(t)}$

where $s(t)$ is the number of neurons firing at timestep $t$ .

At criticality, the coarse-grained reservoir is a renormalization group fixed point: its avalanche statistics are scale-invariant, and the macro-scale dynamics are preserved regardless of the compression level. This means the same W_out trained on the large reservoir remains valid on the small one.

For LIF reservoirs, criticality is reached by tuning the input strength (gain) to the transition between sparse, irregular firing and synchronous, high-rate firing. The NRMSE on a memory task is minimized near this transition.

See Coarse-Graining Theory for the full theoretical background.