Coarse-Graining

Coarse-graining provides a novel framework for reducing reservoir size without disrupting critical properties. The technique merges neurons and synaptic connections based on a graph-theoretic distance derived from synaptic weight strength, effectively pruning the reservoir while preserving the scale-free spiking dynamics indicative of criticality [1].

This page covers the theoretical basis of coarse-graining, the merging rules, and the fixed-point argument that justifies its use on critical reservoirs. For practical usage see Coarse-Graining in the User Guide.

Criticality

Criticality is a defining feature of systems that exhibit second-order phase transitions. At the critical point, these systems demonstrate emergent phenomena, including scale-invariance, power-law distributions, and long-range correlations that propagate across the entire system. The study of criticality in spiking neural networks provides insights into their emergent dynamics and computational potential.

A hallmark of critical phenomena is the presence of scale-free events which closely follow a power-law distribution. In vitro and in vivo testing of cortical activity has found that the size and duration of neuronal events — so-called “neuronal avalanches” — follow a power-law distribution. Such neuronal avalanches are thought to be a result of biological neural networks self-organizing into a near-critical state.

Criticality may also be generally beneficial to information processing, with critical systems thought to optimize certain computational capabilities by maximizing information transfer and sensitivity to input stimuli, while maintaining a high capacity for memory storage. This is due in part to the long-range correlations and scale-invariance inherent in critical states, which enable efficient propagation of information across the system. For neural networks, both biological and artificial, these properties may enhance learning and generalization by enabling the network to encode and process features more effectively.

Input strength $\gamma$ is used as a control parameter to tune a spiking reservoir to criticality. Overstimulation of the reservoir results in saturation and synchronous firing, while insufficient input strength causes activity to die out over time. When the input strength is tuned appropriately, the reservoir exhibits scale-free spiking dynamics characterized by a power-law distribution of neuronal avalanches.

Renormalization

The renormalization group (RG) is a set of mathematical techniques for coarse-graining complex systems by identifying scale-invariant features and systematically redefining divergent bare parameters to dressed parameters which are finite under the defined scale transformations. Seminal work by Wilson demonstrated that renormalization can be applied to critical phenomena such as ferromagnetism. Wilson found that, through a careful construction of scale transformations, certain quantities will flow to fixed points under repeated transformations, thereby providing insight into universal, scale-invariant features which are independent of fine-grain details.

RG has been applied to complex networks by grouping nodes within a graph-theoretic “distance” $\ell_B$ from one another into renormalized “supernodes.” In this transformation, any given supernode is connected to another supernode if they both contain nodes that were connected in the original network. This analysis reveals that RG flow categorizes complex networks into universality classes, distinguishing between self-similar (i.e., fractal) networks and non-self-similar networks based on fixed points under RG flow.

Coarse-Graining Procedure

The proposed coarse-graining procedure comprises pruning the reservoir by grouping nearest-neighbor neurons (nodes) and their synaptic connections (edges) based on geometric or real-space renormalization.

As the neurons of a spiking reservoir are not embedded within Euclidean space, a graph-theoretic analog for distance must be chosen in order to identify “nearest-neighbors” for the length-scale transformation. The graph-theoretic distance between two neurons is defined as the inverse of the synaptic weight between them:

\tag{1} d_{ij} = \frac{1}{W_{ij}}

Accordingly, two neurons are “closer” together if their synaptic weight is higher. Nearest neighbors are identified by finding any set of neurons whose graph-theoretic distance is closer than a threshold distance ( $d_{ij} < d_\text{th}$ , or equivalently $W_{ij} > W_\text{th}$ ). Any such set of neurons identified as nearest neighbors are merged into a single neuron, with the membrane potentials of the neighboring neurons averaged to form the new membrane potential of the merged neuron.

The synaptic connections in the renormalized network are updated as follows:

If another neuron shared a synaptic connection with either of the pre-merged neurons, the synaptic connection is preserved with the newly merged neuron.
If another neuron was originally connected to both merged neurons, the synaptic connection is updated to the average of the original weights.

This coarse-graining procedure preserves the scale-free neuronal avalanches of a critical network while significantly altering non-critical networks. Accordingly, this procedure, like other renormalization techniques, is a suitable length-scale transformation for a critical network, with the overall network dynamics of a critical network being substantially unchanged. This enables a critical network to be significantly reduced in size with minimal impact to its critical dynamics [1].

References

Mastin, T. & Teuscher, C. Coarse-Graining Spiking Reservoirs: Reducing Reservoir Size While Preserving Critical Dynamics. Northeast Journal of Complex Systems (NEJCS) 7(2), Article 2 (2025). https://doi.org/10.63562/2577-8439.1106

← Training Theory | Memory & Information →