Barabasi-Albert Scale-Free Networks

The Barabasi-Albert model generates networks with a scale-free degree distribution, where a small number of highly connected hub neurons coexist with many sparsely connected neurons. This topology arises from the principle of preferential attachment and produces networks with distinctive dynamical properties relevant to reservoir computing.

Preferential Attachment

The Barabasi-Albert model grows a network incrementally. Starting from a small seed graph of $m_0$ nodes, new nodes are added one at a time, each connecting to $m \leq m_0$ existing nodes. The probability that a new node connects to existing node $i$ is proportional to the current degree of $i$ :

P(i) = \frac{k_i}{\sum_j k_j}

where $k_i$ is the degree of node $i$ . Nodes that already have many connections are more likely to receive new connections — a “rich get richer” mechanism.

Power-Law Degree Distribution

Preferential attachment produces a degree distribution that follows a power law:

P(k) \sim k^{-\gamma}

with exponent $\gamma = 3$ for the basic Barabasi-Albert model. This means:

Most neurons have relatively few connections (degree close to $m$ ).
A small number of hub neurons have extremely high degree.
The distribution has no characteristic scale — hence “scale-free.”

The presence of hubs is the defining structural feature. In a network of $n = 500$ neurons with $m = 5$ , the highest-degree hub might have 50—100 connections, while the median neuron has only 5—10.

Properties Relevant to Reservoir Computing

Ultra-Short Path Lengths

Scale-free networks have even shorter average path lengths than Erdos-Renyi graphs of the same size and density. The hubs serve as efficient relay points:

\langle d \rangle \sim \frac{\ln n}{\ln \ln n}

This ultra-short diameter means that input signals can reach any neuron in very few synaptic steps, even in large reservoirs.

Heterogeneous Dynamics

The extreme degree heterogeneity creates a natural hierarchy in the network:

Hub neurons integrate information from many sources and broadcast their activity widely. They act as global integrators.
Peripheral neurons receive input from few sources and transmit to few targets. They act as local feature detectors.
Intermediate neurons bridge local and global scales.

This hierarchy produces richer dynamical patterns than homogeneous networks, potentially improving the separation property of the reservoir.

Robustness and Vulnerability

Scale-free networks are highly robust to random failure (removing a random neuron is unlikely to hit a hub) but vulnerable to targeted attack (removing a hub disconnects large portions of the network). In the context of reservoir computing, this means:

Noise robustness: Random perturbations to neuron states affect primarily the numerous low-degree neurons, with limited impact on global dynamics.
Critical hubs: The performance of the reservoir may depend disproportionately on a few high-degree neurons.

Clustering

The Barabasi-Albert model produces moderate clustering, higher than Erdos-Renyi but lower than Watts-Strogatz. The clustering coefficient scales as:

C \sim \frac{(\ln n)^2}{n}

which vanishes for large $n$ but remains non-negligible for the reservoir sizes typical in SPIRES (hundreds to low thousands of neurons).

SPIRES API

The Barabasi-Albert topology is selected with:


cfg.connectivity_type = SPIRES_CONN_SCALE_FREE;
cfg.connectivity      = 0.1;  /* controls edge density */

In SPIRES, the connectivity parameter controls the overall edge density of the generated scale-free graph. The internal parameter $m$ (number of edges per new node) is derived from the connectivity value and the number of neurons to produce a graph with approximately the specified fraction of all possible edges.

Example Configuration


spires_reservoir_config cfg = {
    .num_neurons       = 500,
    .num_inputs        = 1,
    .num_outputs       = 1,
    .spectral_radius   = 0.95,
    .ei_ratio          = 0.8,
    .input_strength    = 0.1,
    .connectivity      = 0.1,
    .dt                = 1.0,
    .connectivity_type = SPIRES_CONN_SCALE_FREE,
    .neuron_type       = SPIRES_NEURON_FLIF_GL,
    .neuron_params     = NULL,
};

Hub Neurons and the Readout

Hub neurons have an outsized influence on reservoir dynamics and, consequently, on the readout. In the trained output weights $W_{\text{out}}$ , hub neurons often receive the largest (in absolute value) weights because they aggregate information from many parts of the network.

This concentration of information in hub neurons can be both an advantage and a risk:

Advantage: Hubs provide a compressed, high-quality summary of the reservoir’s state, making the readout’s job easier.
Risk: Over-reliance on a few neurons can reduce the effective dimensionality of the readout, making the system sensitive to noise in those specific neurons.

Regularization during training (e.g., the $\lambda$ parameter in ridge regression) helps mitigate this risk by distributing the readout weights more evenly across the reservoir.

Spectral Properties

The eigenvalue spectrum of scale-free adjacency matrices differs qualitatively from Erdos-Renyi matrices. The largest eigenvalue scales with the square root of the maximum degree:

\lambda_{\max} \sim \sqrt{k_{\max}}

Because $k_{\max}$ grows faster than the mean degree, the raw spectral radius of a scale-free graph is larger than that of a random graph with the same average degree. This is why SPIRES rescales the weight matrix to the user-specified spectral_radius after generation — without rescaling, scale-free networks would tend toward chaotic dynamics.

Comparing Topologies

Property	Erdos-Renyi	Watts-Strogatz	Barabasi-Albert
Degree distribution	Poisson (narrow)	Narrow (similar to lattice)	Power-law (broad)
Clustering	Low ( $= p$ )	High	Moderate
Path length	Short ( $\sim \ln n$ )	Short (with rewiring)	Ultra-short ( $\sim \ln n / \ln\ln n$ )
Hubs	No	No	Yes
Construction	Random edges	Rewired lattice	Preferential attachment
SPIRES enum	`SPIRES_CONN_RANDOM`	`SPIRES_CONN_SMALL_WORLD`	`SPIRES_CONN_SCALE_FREE`

When to Use Barabasi-Albert

Choose the Barabasi-Albert topology when:

Tasks benefit from hierarchical processing: Classification problems where global integration of features is important.
Information aggregation is needed: Tasks where the reservoir must combine many input streams into a coherent output.
You want to explore topology effects: Comparing scale-free against random and small-world can reveal whether degree heterogeneity helps or hurts for your specific problem.
Biological realism at the network level: Some neural circuits (e.g., hub-and-spoke architectures in cortical networks) exhibit approximate scale-free properties.

For tasks where local structure is more important than global hubs, consider Watts-Strogatz. For the simplest possible baseline, use Erdos-Renyi.

← Watts-Strogatz | Training: Ridge Regression →