Online Delta Rule

The online delta rule provides a streaming alternative to batch ridge regression. Instead of collecting all reservoir states and solving a global linear system, the delta rule updates the readout weights incrementally after each time step. This makes it suitable for real-time applications, non-stationary data, and memory-constrained environments.

The Delta Rule

The delta rule is a gradient descent algorithm applied to the instantaneous squared error at each time step. Given:

• Reservoir state vector $\mathbf{x}_t \in \mathbb{R}^N$ at time $t$
• Predicted output $\hat{\mathbf{y}}_t = W_{\text{out}}\, \mathbf{x}_t$
• Target output $\mathbf{y}_t$

The error is:

The weight update is:

where $\eta > 0$ is the learning rate.

For multi-output systems where $W_{\text{out}} \in \mathbb{R}^{N_{\text{out}} \times N}$, the update is:

This is a rank-1 update to the weight matrix at each time step, costing $O(N_{\text{out}} \times N)$ operations.

Derivation from Gradient Descent

The delta rule can be derived as stochastic gradient descent on the instantaneous loss:

The gradient with respect to $W_{\text{out}}$ is:

Moving in the negative gradient direction with step size $\eta$ gives the delta rule update. Because the readout is linear, there are no local minima — the loss landscape is a convex quadratic, and gradient descent converges to the global optimum given a sufficiently small learning rate.

Convergence Properties

Learning Rate Selection

The learning rate $\eta$ controls the speed and stability of convergence:

• Too large: The weights oscillate and may diverge. The critical stability threshold depends on the spectral norm of the state correlation matrix.
• Too small: Convergence is slow. The weights take many iterations to approach the optimal solution.
• Optimal range: For reservoir computing, $\eta \in [10^{-4}, 10^{-2}]$ is a common starting range, but the exact value depends on the signal amplitude, reservoir size, and desired convergence speed.

A useful rule of thumb: the learning rate should satisfy $\eta \lt 2 / \|\mathbf{x}\|^2_{\max}$ where $\|\mathbf{x}\|^2_{\max}$ is the maximum squared norm of the state vector encountered during training.

Relationship to Ridge Regression

In the limit of infinitely many passes over a stationary training set, the delta rule converges to the ordinary least-squares solution (unregularized). To obtain the effect of ridge regularization, you can add a weight decay term:

where $\lambda$ plays the same role as the ridge parameter. This is equivalent to L2-regularized stochastic gradient descent.

SPIRES API

Online training is performed with spires_train_online(), which performs a single-step weight update using the reservoir’s current state. It must be called inside a loop, interleaved with spires_step():

spires_status spires_train_online(
    spires_reservoir *r,
    const double *target_vec,
    double lr
);

Parameters:

Returns: SPIRES_OK on success, or an error status code.

Example Usage

#include <spires.h>
#include <stdio.h>
 
double eta = 1e-3;
 
for (size_t t = 0; t < series_length; t++) {
    /* Step the reservoir with the current input */
    spires_step(r, &input_train[t * num_inputs]);
 
    /* Update weights toward the target for this timestep */
    spires_status s = spires_train_online(r, &target_train[t * num_outputs], eta);
    if (s != SPIRES_OK) {
        fprintf(stderr, "Online training failed at t=%zu\n", t);
        break;
    }
}

Incremental Training

Because spires_train_online() is a single-step call, incremental training across multiple batches is straightforward — just continue the loop with new data. The reservoir state and weights carry over naturally:

/* Train on first batch */
for (size_t t = 0; t < batch1_len; t++) {
    spires_step(r, &input_batch1[t * num_inputs]);
    spires_train_online(r, &target_batch1[t * num_outputs], eta);
}
 
/* Continue training on second batch -- weights and state carry over */
for (size_t t = 0; t < batch2_len; t++) {
    spires_step(r, &input_batch2[t * num_inputs]);
    spires_train_online(r, &target_batch2[t * num_outputs], eta);
}

Comparison with Other Methods

When to Use the Delta Rule

Choose the online delta rule when:

• Memory is severely constrained: $O(N^2)$ is not affordable (e.g., very large reservoirs on embedded hardware). The delta rule uses only $O(N_{\text{out}} \times N)$ memory for the weights and $O(N)$ for the current state — no auxiliary matrices.
• Per-step compute budget is tight: Each delta rule update costs $O(N \cdot N_{\text{out}})$, versus $O(N^2)$ for RLS. For large reservoirs, this difference is significant.
• Convergence speed is secondary: The delta rule requires many more steps than RLS to reach an equivalent solution, but if training data is plentiful this is not a concern.
• Multiple passes are acceptable: For offline data, you can run the delta rule over the same data multiple times (epochs) to improve convergence.

Practical Considerations

Learning Rate Scheduling

For better convergence, you can decrease the learning rate over time. A common schedule is:

where $\eta_0$ is the initial learning rate and $\tau_\eta$ is a decay time constant. SPIRES uses a fixed learning rate per call; to implement scheduling, reduce $\eta$ between successive calls to spires_train_online().

Initialization

The readout weights $W_{\text{out}}$ are initialized to zero when the reservoir is created. This is appropriate for the delta rule, which will learn the correct weights from the training data. If you call spires_train_online() after spires_train_ridge(), the delta rule will refine the ridge solution rather than starting from zero.

Washout

As with ridge regression, the first few time steps of the reservoir response are transient. With the delta rule, the weights update during these transient steps, which can introduce small errors. In practice, these are overwritten by subsequent updates. For critical applications, you can drive the reservoir with a washout input before beginning online training.

← Recursive Least Squares | AGILE Optimizer →