NNUE

Stage 21 · in progress · code in crates/nnue, crates/tools/src/selfplay.rs, crates/trainer, crates/engine/src/search.rs

This chapter is being written as stage 21 is built. So far it covers the network’s definition (its inputs, its layers and its whole-number weights), its training data, the trainer and the network in the search. What it’s worth comes once a network has been trained on the full data and measured.

Chapters 15 and 16 built an evaluation from terms a person designed and tuned their weights on cb2026’s own games. An NNUE (“efficiently updatable neural network”) replaces those terms with a small neural network that learns what matters from millions of positions. Since 2020 it has been the evaluation of almost every top engine. This chapter builds one from scratch, with a trainer of cb2026’s own, so every step can be followed.

The idea

A neural network is layers of numbers. Each value in a layer is a weighted sum of the values in the layer before, passed through a simple function (the activation). Training adjusts the weights so the last layer predicts something: here, how good the position is.

Two things make this work for chess at engine speed:

1. Inputs that barely change. The network’s inputs say which piece stands on which square. A move changes only a few of them, so the first layer, which does almost all of the work, can be updated by adding and subtracting a few rows of weights instead of being recomputed. That running first layer is the accumulator.
2. Whole numbers. The weights are rounded to integers, so the arithmetic is exact and the CPU can do many small integer operations at once.

The inputs

An input is one fact, such as “a white knight stands on f3”. There are 2 colours × 6 piece types × 64 squares = 768, one on for each piece. The network reads them from both sides’ points of view: from a side’s view its own pieces are “ours”, and Black sees the board flipped top to bottom. So the same weights describe White’s knight on f3 for White and Black’s knight on f6 for Black.

/// The input for `piece` standing on `square`, seen from `view`'s side of the board.
pub fn index(view: Color, piece: Piece, square: Square) -> usize {
    let side = if piece.color == view { 0 } else { 1 };
    let square = match view {
        Color::White => square,
        Color::Black => square.flip_rank(),
    };
    side * PER_SIDE + piece.piece_type.index() * SQUARES + square.index()
}

None of these inputs depends on where a king stands, which keeps the network small and its updates cheap: no move ever forces a full recomputation. Stronger networks add the king’s square (“king buckets”); they need much more training data, and cb2026 will try them later, measured like everything else.

A move switches off the inputs of the pieces it lifts and switches on those it puts down. A capture also lifts the captured piece, en passant lifts a pawn beside the destination, and castling lifts and puts down both the king and the rook. That is at most two off and two on:

/// The pieces a move lifts and puts down, which are the inputs it switches off and on. They are
/// pieces and squares, not input numbers, so they serve both views.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub struct Changes {
    removed: [Option<(Piece, Square)>; 2],
    added: [Option<(Piece, Square)>; 2],
}

impl Changes {
    /// What playing `mv` changes in `pos`, the position before the move. `mv` must be legal in
    /// `pos`.
    pub fn of(pos: &Position, mv: Move) -> Changes {
        let us = pos.side_to_move();
        let (from, to) = (mv.from(), mv.to());
        let piece = pos.piece_at(from).expect("a piece on the from-square");

        if mv.kind() == MoveKind::Castle {
            // "King takes own rook": `to` is the rook's square, as in `Position::make_move`.
            let side = if to.file() > from.file() {
                CastlingSide::KingSide
            } else {
                CastlingSide::QueenSide
            };
            let (king_to, rook_to) = castling_destinations(us, side);
            let rook = pos.piece_at(to).expect("the castling rook");
            return Changes {
                removed: [Some((piece, from)), Some((rook, to))],
                added: [Some((piece, king_to)), Some((rook, rook_to))],
            };
        }

        let captured = mv.is_capture().then(|| {
            // En passant captures the pawn beside the destination, not on it.
            let square = match mv.kind() {
                MoveKind::EnPassant => Square::new(to.file(), from.rank()),
                _ => to,
            };
            (pos.piece_at(square).expect("a piece to capture"), square)
        });
        let landing = match mv.promotion() {
            Some(piece_type) => Piece::new(us, piece_type),
            None => piece,
        };
        Changes {
            removed: [Some((piece, from)), captured],
            added: [Some((landing, to)), None],
        }
    }

    /// The pieces lifted: their inputs go off.
    pub fn removed(&self) -> impl Iterator<Item = (Piece, Square)> + '_ {
        self.removed.iter().flatten().copied()
    }

    /// The pieces put down: their inputs come on.
    pub fn added(&self) -> impl Iterator<Item = (Piece, Square)> + '_ {
        self.added.iter().flatten().copied()
    }
}

The layers

  768 inputs ──► 256 hidden ─┐                 (from the side to move's view)
                             ├──► 1 output ──► centipawns
  768 inputs ──► 256 hidden ─┘                 (from the other side's view)

Each view has a hidden layer of 256 values. Both views use the same first-layer weights, which the flipped board makes possible. The output layer takes the side to move’s 256 values first, then the other side’s. A position and its mirror image (colours swapped, board flipped) give exactly the same inputs in the same order, so they get exactly the same score, which a test checks.

The activation clips each hidden value to the range 0 to 1 and squares it (SCReLU, the squared clipped ReLU). Squaring lets the network weigh a large hidden value more than a middling one. Plain clipping (CReLU) is also supported, so the two can be compared.

    /// The network's output for `pos`, from the side to move's view, before the eval scale.
    /// The trainer compares `σ(output)` with its targets.
    pub fn output(&self, pos: &Position) -> f32 {
        let us = pos.side_to_move();
        let (our_weights, their_weights) = self.output_weights.split_at(self.hidden);
        let half = |view: Color, weights: &[f32]| -> f32 {
            self.hidden_layer(pos, view)
                .iter()
                .zip(weights)
                .map(|(&value, weight)| self.activate(value) * weight)
                .sum()
        };
        half(us, our_weights) + half(!us, their_weights) + self.output_bias
    }

That is the network as the trainer sees it, in floating point. A hidden layer of 256 means about 200,000 weights, a good size for the 100 million positions a first network trains on. The size is stored in the network’s file, so networks of 128 or 512 can be tried too.

Whole numbers instead of floats

The engine runs the same network with whole-number weights: the first layer’s weights times 255, the output weights times 64.

/// The scale of first-layer weights and biases: a hidden value of 1.0 is 255.
pub const QA: i32 = 255;
/// The scale of output weights.
pub const QB: i32 = 64;
/// The largest first-layer weight or bias: [`WEIGHT_LIMIT`](crate::float::WEIGHT_LIMIT) × [`QA`], rounded.
pub const MAX_INPUT_WEIGHT: i16 = 505;
/// The largest output weight: [`WEIGHT_LIMIT`](crate::float::WEIGHT_LIMIT) × [`QB`], rounded.
pub const MAX_OUTPUT_WEIGHT: i16 = 127;

A hidden value of 1.0 becomes 255, so the clipping is a clamp to 0..255. The accumulator is the first layer kept up to date move by move:

    /// The accumulator for `pos`, computed from scratch: the biases plus the row of every input
    /// that is on.
    pub fn accumulator(&self, pos: &Position) -> Accumulator {
        let views = Color::ALL.map(|view| {
            let mut values = self.input_biases.clone();
            for input in features::active(pos, view) {
                add(&mut values, self.row(input));
            }
            values
        });
        Accumulator { views }
    }

    /// Brings `acc` up to date after a move with these `changes`: subtract the rows of the
    /// inputs switched off, add the rows of those switched on. Same result as
    /// [`Network::accumulator`] on the new position, for a fraction of the work.
    pub fn update(&self, acc: &mut Accumulator, changes: &Changes) {
        for view in Color::ALL {
            let values = &mut acc.views[view.index()];
            for (piece, square) in changes.removed() {
                subtract(values, self.row(features::index(view, piece, square)));
            }
            for (piece, square) in changes.added() {
                add(values, self.row(features::index(view, piece, square)));
            }
        }
    }

Integers can overflow, so the sizes are chosen to make that impossible. The trainer keeps every weight within ±1.98, so a first-layer weight is at most 505 and an output weight at most 127. A legal position has at most 32 pieces, so a hidden value is at most 33 × 505 = 16,665 (the 32 pieces plus the bias), inside an i16’s 32,767. An activation times an output weight is at most 255 × 127 = 32,385, which fits too.

/// A hidden value clipped to 0..1, at the scale QA: 0..255.
fn clipped(value: i16) -> i32 {
    i32::from(value).clamp(0, QA)
}

/// The output layer's sum with CReLU: each clipped hidden value times its weight.
fn crelu_dot(values: &[i16], weights: &[i16]) -> i64 {
    values
        .iter()
        .zip(weights)
        .map(|(&v, &w)| i64::from(clipped(v) * i32::from(w)))
        .sum()
}

/// The output layer's sum with SCReLU: each clipped hidden value squared, times its weight.
/// Computed as `(a × w) × a`: `a × w` is at most 255 × 127 = 32,385 and fits in an `i16`.
fn screlu_dot(values: &[i16], weights: &[i16]) -> i64 {
    values
        .iter()
        .zip(weights)
        .map(|(&v, &w)| {
            let a = clipped(v);
            i64::from(a * i32::from(w)) * i64::from(a)
        })
        .sum()
}

Rounding costs a little accuracy. With the random weights the tests use, the integer network is on average 2–4 centipawns from the floating-point one. Given the rounded weights, it computes the same thing to within a centipawn.

The output’s scale

The output, times an eval scale, is in centipawns. The scale isn’t arbitrary: it has to match the hand-crafted evaluation’s centipawns, because the search’s pruning margins were all tuned in those centipawns.

The trainer will fit the network to two things at once, the search score of each position and the result of its game, turning scores into expected results with a sigmoid, σ(cp / scale). If the scale doesn’t match how cb2026’s centipawns actually relate to results, the two targets pull against each other and the network’s centipawns stretch. Chapter 16’s Texel tuner fits exactly this relation as its K: on 40,304 self-play positions it came out at 1.19, which is a scale of 400 / (1.19 × ln 10) ≈ 146. Many engines use 400; cb2026’s centipawns are steeper.

    /// The position's value in centipawns, from `side_to_move`'s point of view, given its
    /// accumulator. Nothing here keeps it below the search's mate scores: the engine clamps it.
    pub fn evaluate(&self, acc: &Accumulator, side_to_move: Color) -> i32 {
        let ours = acc.view(side_to_move);
        let theirs = acc.view(!side_to_move);
        let (our_weights, their_weights) = self.output_weights.split_at(self.hidden);
        // The sum is at the scale QA × QB. SCReLU's squares are at QA × QA × QB: divide one QA out.
        let sum = match self.activation {
            Activation::ClippedRelu => {
                crelu_dot(ours, our_weights) + crelu_dot(theirs, their_weights)
            }
            Activation::SquaredClippedRelu => {
                (screlu_dot(ours, our_weights) + screlu_dot(theirs, their_weights)) / i64::from(QA)
            }
        };
        let output = sum + i64::from(self.output_bias);
        // The sum is at most 2 × 4096 × 255 × 127 ≈ 2.7 × 10^8 and the bias an i32 (≈ 2.1 × 10^9);
        // times an eval scale of at most 10^4 that fits easily in an i64, and divided by
        // QA × QB it is at most about 1.5 × 10^9, inside an i32.
        (output * i64::from(self.eval_scale) / i64::from(QA * QB)) as i32
    }

The network file

A network is stored as a 48-byte header followed by the numbers. The header says everything about the network’s shape (inputs, hidden size, activation, scales) and carries a checksum, so a damaged or foreign file is refused before it is used. Each input’s weights are one row, so switching an input on or off adds one row to the accumulator. The default network is part of the source, in nets/, and each release carries it as a download too; other networks aren’t kept in the repository.

The training data

The network learns from positions of cb2026’s own games, the same self-play games as chapter 16’s tuner, played at a fixed number of nodes per move. The tuner only needed each position and its game’s result. The trainer also wants the search’s score, and since producing data takes the test Mac about 12 hours per 100 million positions, the format should keep anything a later network might want. Each position becomes a record of 32 bytes:

A position has at most 32 pieces, so 32 four-bit codes, 16 bytes, hold them all. The codes are the piece types plus 8 for Black, and one more: code 6 is a rook that can still castle. That is how a record keeps castling rights, and it works for Chess960, where a right belongs to one particular rook. The layout follows marlinformat, the format of the trainer of the engine Marlin.

    /// The record of `pos`, where the search chose `best_move` with a score of `score`
    /// centipawns from White's side, in a game that ended in `result`. A score beyond an `i16`
    /// is clamped.
    pub fn new(
        pos: &Position,
        score: i32,
        result: GameResult,
        best_move: Move,
    ) -> Result<Record, RecordError> {
        let occupied = pos.occupied();
        if occupied.count() > MAX_PIECES {
            return Err(RecordError::TooManyPieces(occupied.count()));
        }
        let castling_rooks = castling_rooks(pos);
        let mut codes = 0u128;
        for (i, square) in occupied.squares().enumerate() {
            let piece = pos
                .piece_at(square)
                .expect("an occupied square has a piece");
            let kind = if castling_rooks.contains(square) {
                CASTLING_ROOK
            } else {
                piece.piece_type.index() as u8
            };
            let code = kind
                | if piece.color == Color::Black {
                    BLACK
                } else {
                    0
                };
            codes |= u128::from(code) << (4 * i);
        }
        let side = if pos.side_to_move() == Color::Black {
            0x80
        } else {
            0
        };
        let en_passant = pos.en_passant().map_or(NO_EN_PASSANT, |s| s.index() as u8);
        Ok(Record {
            occupied: occupied.0,
            codes,
            turn: side | en_passant,
            halfmove_clock: pos.halfmove_clock().min(255) as u8,
            score: score.clamp(i32::from(i16::MIN), i32::from(i16::MAX)) as i16,
            result,
            best_move: best_move.to_bits(),
        })
    }

The record keeps the position itself, not the network’s inputs. The trainer turns the pieces into inputs as it reads them, so a later network with different inputs, such as king buckets, can learn from the same records without generating them again:

    /// Every piece and its square, in square order from a1. This is all the trainer needs to
    /// compute the network's inputs.
    pub fn pieces(&self) -> impl Iterator<Item = (Piece, Square)> + '_ {
        Bitboard(self.occupied)
            .squares()
            .enumerate()
            .map(|(i, square)| (piece_of(self.code(i)), square))
    }

Balanced starts

Each game starts from one of 50 balanced openings plus 8 random moves, so that no two games are alike. Random moves are often bad ones, though, and many games began with one side already far ahead. In 600 games, only 18.5% of the recorded positions were within a pawn of level and 37.5% were more than five pawns from it; White won 54% of the games.

So when writing records, datagen searches each start like a move of the game and draws the random moves again while the start scores beyond 200 centipawns:

/// The start for game `index`, roughly balanced if `settings.max_start_score` is set: each try
/// is searched like a move of the game, and the first that scores within the limit is used. If
/// none of the `START_ATTEMPTS` (20) tries does, the most balanced is.
pub fn balanced_start(engine: &mut Engine, settings: &DatagenSettings, index: usize) -> Position {
    let Some(limit) = settings.max_start_score else {
        return starting_position(settings, index);
    };
    let mut best: Option<(i32, Position)> = None;
    for attempt in 0..START_ATTEMPTS {
        let start = random_start(settings, index, attempt);
        let imbalance = start_score(engine, &start, settings.nodes);
        if imbalance <= limit {
            return start;
        }
        if best.as_ref().is_none_or(|(b, _)| imbalance < *b) {
            best = Some((imbalance, start));
        }
    }
    best.expect("at least one try").1
}

A quarter of the positions are still far from level, because games at a few thousand nodes a move do get decided, and a network has to judge won positions too. What the limit removes is games that were lopsided from their first move. It costs nothing: balanced games last longer and give more positions each, which pays for the extra search. The tuner’s text output keeps every start, so chapter 16’s runs can still be repeated.

datagen --format records writes records, and records checks a file and shows what is in it. Sixty games make about 4,100 records, 132 KB; 100 million records take 3.2 GB. Records have no file header, so files from separate runs can simply be joined.

That makes a 12-hour run safe to interrupt. scripts/mac-nnue-data.sh makes the data in chunks of 56,000 games, each its own file, and the trainer reads them all (--data once per file). A run that stops loses only the chunk it was on: started again, it keeps the finished files and makes the rest. Each game’s random opening moves come from its number, and chunk k’s numbers start where chunk k − 1’s ended, so the chunks never repeat a game, and a chunk made twice holds the same games.

The trainer

The trainer (crates/trainer) learns the weights. It uses no machine-learning library: every gradient is worked out by hand, so each step can be read.

What it learns to predict

The network’s output y means y × eval scale centipawns, and σ(y) the expected result for the side to move. Each record gives two estimates of what σ(y) should be: the search’s score, σ(score / eval scale), which is precise but sometimes wrong, and the game’s result, which is right about the game but noisy about any one position in it. The target blends them, a quarter result by default:

    /// The example for `record`.
    pub fn from_record(record: &Record, targets: Targets) -> Example {
        let us = record.side_to_move();
        let mut inputs = [[0u16; MAX_ACTIVE]; 2];
        let mut count = 0;
        for (piece, square) in record.pieces() {
            inputs[0][count] = features::index(us, piece, square) as u16;
            inputs[1][count] = features::index(!us, piece, square) as u16;
            count += 1;
        }
        let score = f32::from(record.score_for_side_to_move());
        let result = match us {
            Color::White => record.result().for_white(),
            Color::Black => 1.0 - record.result().for_white(),
        };
        let w = targets.result_weight;
        let target = (1.0 - w) * sigmoid(score / targets.eval_scale) + w * result;
        Example {
            inputs,
            count,
            target,
        }
    }

The eval scale comes from the data, fitted as chapter 16 fitted K: on 134,000 records with balanced starts it came out at 159, close to the 146 found earlier.

Forward, loss, backward

The forward pass is the network itself, run on the example’s inputs:

/// The network's output for `example`, before the sigmoid. Leaves the hidden layer in
/// `scratch` for [`backward`].
pub fn forward(net: &FloatNetwork, example: &Example, scratch: &mut Scratch) -> f32 {
    let h = net.hidden;
    let mut output = net.output_bias;
    for view in 0..2 {
        let values = &mut scratch.hidden[view * h..(view + 1) * h];
        values.copy_from_slice(&net.input_biases);
        for &input in example.inputs(view) {
            let row = &net.input_weights[usize::from(input) * h..(usize::from(input) + 1) * h];
            values.iter_mut().zip(row).for_each(|(v, w)| *v += w);
        }
        let weights = &net.output_weights[view * h..(view + 1) * h];
        output += values
            .iter()
            .zip(weights)
            .map(|(&v, w)| net.activate(v) * w)
            .sum::<f32>();
    }
    output
}

The loss is the squared difference between σ(y) and the target. Training needs its gradient: for every weight, how much the loss changes as that weight changes. The chain rule gives it, working back from the output. The loss’s slope with respect to y is 2 (σ(y) − target) σ(y) (1 − σ(y)):

/// The loss for an output and a target, `(σ(output) − target)²`, and its slope with respect to
/// the output.
pub fn loss(output: f32, target: f32) -> (f32, f32) {
    let p = sigmoid(output);
    let error = p - target;
    (error * error, 2.0 * error * p * (1.0 - p))
}

From there, an output weight’s gradient is that slope times its hidden value’s activation, and a hidden value’s is the slope times its output weight times the activation’s own slope, which is zero wherever the clipping holds the value at 0 or 1. A first-layer weight collects its hidden value’s gradient once for each view in which its input is on: only about 32 of the 768 rows change for any one position.

/// Adds `example`'s gradients to `grads`, given the hidden layer [`forward`] left in `scratch`
/// and `d_output`, the loss's slope with respect to the output.
pub fn backward(
    net: &FloatNetwork,
    example: &Example,
    scratch: &mut Scratch,
    d_output: f32,
    grads: &mut Gradients,
) {
    let h = net.hidden;
    grads.output_bias += d_output;
    for view in 0..2 {
        let values = &scratch.hidden[view * h..(view + 1) * h];
        let weights = &net.output_weights[view * h..(view + 1) * h];
        let output_grads = &mut grads.output_weights[view * h..(view + 1) * h];
        for i in 0..h {
            output_grads[i] += d_output * net.activate(values[i]);
            scratch.d_hidden[i] = d_output * weights[i] * slope(net.activation, values[i]);
        }
        let d_hidden = &scratch.d_hidden;
        grads
            .input_biases
            .iter_mut()
            .zip(d_hidden)
            .for_each(|(g, d)| *g += d);
        for &input in example.inputs(view) {
            let row =
                &mut grads.input_weights[usize::from(input) * h..(usize::from(input) + 1) * h];
            row.iter_mut().zip(d_hidden).for_each(|(g, d)| *g += d);
        }
    }
}

Hand-written gradients need checking. The test nudges weights a little each way, measures how the loss moves, and compares that with the formulas: they agree to 0.1%. The check itself taught two lessons. Near the clipping’s corners a nudge measures the wrong slope, so those hidden values are left out. And in f32, σ(y) near ½ is only exact to about 6 × 10⁻⁸, a few percent of the change a small nudge makes, so the test measures the loss in f64.

Stepping: AdamW

The gradients of a batch of records are averaged, and AdamW moves each weight against its gradient, as chapter 16’s tuner did. It keeps a running average of each weight’s gradient and of its square, and steps by about their ratio, so rarely used inputs learn as fast as common ones. It also pulls every weight a little towards zero (weight decay), and afterwards clips it to ±1.98, the limit the whole-number network relies on:

    /// Moves every weight of `net` one step against `grads`, then clips it to the weight limit.
    pub fn step(&mut self, net: &mut FloatNetwork, grads: &Gradients) {
        self.steps += 1;
        let correction1 = (1.0 - f64::from(Self::BETA1).powf(self.steps as f64)) as f32;
        let correction2 = (1.0 - f64::from(Self::BETA2).powf(self.steps as f64)) as f32;
        let rate = self.learning_rate;
        let update = |weight: &mut f32, g: f32, m: &mut f32, v: &mut f32, decay: f32| {
            *m = Self::BETA1 * *m + (1.0 - Self::BETA1) * g;
            *v = Self::BETA2 * *v + (1.0 - Self::BETA2) * g * g;
            let (m_hat, v_hat) = (*m / correction1, *v / correction2);
            *weight -= rate * (m_hat / (v_hat.sqrt() + Self::EPSILON) + decay * *weight);
        };
        let (first, second) = (&mut self.first, &mut self.second);
        let groups = [
            (
                &mut net.input_weights,
                &grads.input_weights,
                &mut first.input_weights,
                &mut second.input_weights,
            ),
            (
                &mut net.input_biases,
                &grads.input_biases,
                &mut first.input_biases,
                &mut second.input_biases,
            ),
            (
                &mut net.output_weights,
                &grads.output_weights,
                &mut first.output_weights,
                &mut second.output_weights,
            ),
        ];
        for (weights, g, m, v) in groups {
            for i in 0..weights.len() {
                update(
                    &mut weights[i],
                    g[i],
                    &mut m[i],
                    &mut v[i],
                    self.weight_decay,
                );
                weights[i] = weights[i].clamp(-WEIGHT_LIMIT, WEIGHT_LIMIT);
            }
        }
        // The output bias: no decay and no clipping.
        let (m, v) = (&mut first.output_bias, &mut second.output_bias);
        update(&mut net.output_bias, grads.output_bias, m, v, 0.0);
    }

The loop

Each epoch visits every training record once in a new random order, in batches of 16,384. A batch is shared out among the CPU’s threads, and their gradients are added up in a fixed order, so the same settings always give the same network. The learning rate drops to a tenth for the last quarter of the epochs. After each epoch the trainer writes the network, with whole-number weights, as a file the engine can read, and a checkpoint to resume from.

The last 1% of the records, whole games the training never sees, are the validation set. The training loss shows how well the network fits the positions it learns from; the validation loss shows how well it has learnt chess. A test makes the difference plain. A student network learns the scores of a random “teacher” network of the same shape: on 1,920 positions its training loss falls 200-fold, but on unseen positions it is only 38% better than guessing the average, and with 576 positions no better at all. It has learnt the positions by heart. Only many positions teach a network chess, which is why a first one needs about 100 million.

On the M1 Max, with its 8 performance cores, the trainer handles about 2 to 2.4 million records a second, so an epoch of 100 million records takes under a minute and 20 epochs under 20 minutes. Generating those records takes about 12 hours: data, not training, is what takes the time. Every thread gets an equal share of each batch and the batch waits for the slowest, so the trainer leaves the efficiency cores out: with them, 10 threads were 17% slower than 8.

On the GPU

The same training runs on the Mac’s GPU with --backend gpu (in a trainer built with --features gpu). It goes through wgpu, a Rust library for WebGPU, which runs on Metal on the Mac and on Vulkan or DirectX 12 elsewhere, with an API that is safe Rust. The code that runs on the GPU, a kernel, is written in WGSL, WebGPU’s shading language. Thousands of copies of a kernel, its invocations, run at once, each computing one number, such as one hidden value of one example, in workgroups of 64 that the GPU schedules together. A training step is a row of kernels, each one step of the formulas above: the hidden layer, the output and loss, the hidden values’ slopes, the gradients, AdamW, the loss’s running total. The network and AdamW’s averages stay on the GPU for the whole run; only each batch’s inputs and targets go up, and the network comes back once an epoch, for the file and the checkpoint. The CPU and GPU versions sit behind one interface, Backend, with one call per batch, so the loop above drives both.

A GPU is fast at work that splits into many equal, independent pieces, and slow at anything else. Two parts of the backward pass weren’t like that at first.

Sums over the batch. An output weight’s gradient is a sum over all 16,384 examples. With one invocation per weight looping over the whole batch, a network of 256 had only 513 invocations for a GPU that runs tens of thousands, and those sums took 25 of a batch’s 35 milliseconds. So the batch is cut into chunks of 128 examples: one kernel sums each weight over each chunk, and a second adds up the chunks.

The sparse gradient. A first-layer weight collects a term from every example in which its input is on. On the CPU, each thread adds its examples’ terms into its own copy of the gradients. On the GPU, thousands of invocations would be adding into the same weights at once, which needs atomic addition, and WGSL has none for floats. So the CPU sorts each batch’s rows by input first, and each weight adds up its own list with nobody else writing to it. The sort is a counting sort: count each input’s rows, turn the counts into starting places, then put each row in its place:

    // A counting sort: input i's rows will be entries[offsets[i]..offsets[i + 1]], in row
    // order.
    for i in 1..=INPUTS {
        offsets[i] += offsets[i - 1];
    }
    let mut next = offsets.clone();
    let mut entries = vec![0u32; offsets[INPUTS] as usize];
    for (row, inputs) in inputs.chunks_exact(MAX_ACTIVE).enumerate() {
        for &input in inputs.iter().take_while(|&&input| input != NONE) {
            entries[next[input as usize] as usize] = (first_row + row) as u32;
            next[input as usize] += 1;
        }
    }

The lists turned out to be very uneven: in a batch the average input is on in about 700 of the 32,768 rows, but a king on its first square is on in over 11,000. A kernel is as slow as its slowest invocation, so one invocation per weight, looping over its list, made this kernel as slow as its longest list, whatever the network’s size: a network of 512 trained no faster than one of 1,024. So the lists are also cut into segments of up to 256 rows. One kernel sums each segment for every hidden value, and a second adds up each input’s segments:

// The first layer's weight gradients, first stage. A weight's gradient is its hidden value's
// slope, summed over every example and view in which its input is on. WGSL has no atomic
// addition for floats, so instead of each row adding into the weights it touches, the rows are
// grouped by input beforehand (`entries`, built on the CPU) and each weight sums its own group.
// The groups are very uneven: in a batch of 16,384 the average input is on in about 700 rows,
// but a king on its first square in over 11,000, and the invocation adding those up would keep
// the rest of the GPU waiting. So each group is cut into segments of up to SEGMENT rows, one
// invocation sums one hidden value over one segment, and input_gradients.wgsl adds up each
// input's segments. The order is fixed, so every run gives the same result.

@group(0) @binding(1) var<storage, read> d_hidden: array<f32>;
// The rows, grouped by input.
@group(0) @binding(2) var<storage, read> entries: array<u32>;
// Segment s is entries[segments[2s]..segments[2s + 1]].
@group(0) @binding(3) var<storage, read> segments: array<u32>;
// Per segment: its sum for each hidden value.
@group(0) @binding(4) var<storage, read_write> segment_sums: array<f32>;

@compute @workgroup_size(64)
fn main(@builtin(global_invocation_id) id: vec3<u32>) {
    let unit = id.x;
    let segment = id.y;
    if (unit >= sizes.hidden || segment >= sizes.segments) {
        return;
    }
    var sum = 0.0;
    for (var n = segments[2u * segment]; n < segments[2u * segment + 1u]; n++) {
        sum += d_hidden[entries[n] * sizes.hidden + unit];
    }
    segment_sums[segment * sizes.hidden + unit] = sum;
}

That doubled the speed of networks of 512. Every sum runs in a fixed order, so the same GPU gives the same network every time, but not quite the CPU’s: floats added in another order round differently. The tests run both on the same batch and compare the gradients, an AdamW step and a short training run; breaking the segments on purpose makes the gradients’ test fail.

Measured on the M1 Max (32 GPU cores) on 2.8 million records, in records a second:

The training losses were the same to six decimals. The CPU’s time grows with the network, since it does every multiplication itself. The GPU’s grows more slowly. At first it didn’t even set the pace at 256: the CPU prepared each batch, decoding the records and sorting the rows, on one thread in about 4 milliseconds, while the GPU’s work took 3.5, so the GPU waited. Now each of the trainer’s threads prepares a share of the batch, and the shares are joined input by input, in order, so the lists come out exactly as one thread would make them:

    // Input i's rows are each share's rows for input i, the shares in order: the same lists, in
    // the same row order, whatever the number of threads.
    let mut offsets = Vec::with_capacity(INPUTS + 1);
    let mut entries = Vec::with_capacity(shares.iter().map(|share| share.entries.len()).sum());
    for i in 0..INPUTS {
        offsets.push(entries.len() as u32);
        for share in &shares {
            let rows = share.offsets[i] as usize..share.offsets[i + 1] as usize;
            entries.extend_from_slice(&share.entries[rows]);
        }
    }
    offsets.push(entries.len() as u32);
    // Each input's list cut into segments of up to SEGMENT rows.
    let mut segments = Vec::new();
    let mut first_segment = Vec::with_capacity(INPUTS + 1);
    for i in 0..INPUTS {
        first_segment.push((segments.len() / 2) as u32);
        for start in (offsets[i]..offsets[i + 1]).step_by(SEGMENT) {
            segments.extend([start, offsets[i + 1].min(start + SEGMENT as u32)]);
        }
    }
    first_segment.push((segments.len() / 2) as u32);

On 8 threads the preparation takes 0.8 milliseconds, and training at 256 went from 2.7 to 5.3 million records a second, with the same losses. The GPU’s own work is now the limit at every size.

Measuring had a trap of its own. On a file of 141,000 records the GPU seemed no faster than the CPU at 256. An epoch was only nine batches there, and the trainer reads the loss back 20 times an epoch for its progress line, so after every batch the CPU waited for the GPU to finish, and the two never worked at the same time. A real run has thousands of batches an epoch. A benchmark must look like the run it stands for.

Other Macs

The numbers belong to one Mac. The M3 Max, where cb2026’s long runs go, differs from the M1 Max in each of the things that set them. Its performance cores are faster, and it has 10 or 12 of them rather than 8, so the CPU trainer speeds up. Its GPU is a newer design, with 30 or 40 cores rather than 24 or 32, that gives each running kernel only the on-chip memory it uses rather than the most it might need, so more invocations fit at once (Apple calls this Dynamic Caching). Its memory bandwidth is 300 or 400 GB/s, depending on the model, against the M1 Max’s 400, and the sparse gradient’s kernels do little arithmetic for each number they read, so memory is likely to bound them. So the GPU’s lead, 2.3 to 2.8 times on the M1 Max, could grow or shrink there. scripts/mac-trainer-speed.sh measures both backends on the Mac it runs on. On the M3 Max with 10 performance cores and 30 GPU cores, on 2.9 million records (cjus/chessengine#187):

Its CPU trainer is about 45% faster than the M1 Max’s, and its GPU about as fast, so the GPU’s lead shrank to 1.5 to 1.9 times. It still leads at every size, so the long runs train on the GPU. With all 14 cores, the efficiency cores too, the CPU trained more slowly at 256, 3.1 million records a second against 3.3: each thread gets an equal share of a batch, and the batch waits for the slowest, which is why the trainer uses only the performance cores unless told otherwise. A GPU of another design may also round a little differently, so a network is only repeated exactly on the same machine, backend and thread count.

The network in the search

The search evaluates millions of positions a second, each a move away from one it has just seen. So each search thread keeps a stack of accumulators, one per ply: the one at ply p belongs to the position p moves from the root. Wherever the search is about to look at a child, it builds the child’s accumulator from its own with the move’s changes; for a null move, which moves no piece, it copies it. The root’s is computed from scratch at the start of each search.

    /// Writes into `child` the accumulator after a move with these `changes`, given `parent`,
    /// the accumulator before it: what the search does at every move, without allocating.
    pub fn update_into(&self, parent: &Accumulator, child: &mut Accumulator, changes: &Changes) {
        child.copy_from(parent);
        self.update(child, changes);
    }

    /// Computes the root's accumulator from scratch: every search starts here.
    fn refresh_accumulator(&mut self, root: &Position) {
        if let Some(nnue) = &mut self.nnue {
            nnue.net.refresh(root, &mut nnue.accumulators[0]);
        }
    }

    /// Makes the accumulator for the child at `ply + 1` from the one at `ply`, for `mv` played
    /// in `pos` (`None`: a null move, which changes no piece). Called wherever a child is
    /// about to be searched.
    fn update_accumulator(&mut self, pos: &Position, mv: Option<Move>, ply: u32) {
        if let Some(nnue) = &mut self.nnue {
            let (parents, children) = nnue.accumulators.split_at_mut(ply as usize + 1);
            let (parent, child) = (&parents[ply as usize], &mut children[0]);
            match mv {
                Some(mv) => nnue.net.update_into(parent, child, &Changes::of(pos, mv)),
                None => child.copy_from(parent),
            }
        }
    }

    /// Whether `pos` is an ending the hand-crafted evaluation plays better than the network:
    /// a lone king ([`Techniques::lone_king`]) or a small pawnless ending
    /// ([`Techniques::pawnless_endings`]).
    fn hand_crafted_ending(&self, pos: &Position) -> bool {
        hand_crafted_ending(&self.techniques, pos)
    }

    /// The quiescence search from `pos` alone, as at a leaf of the main search: captures (and
    /// queen promotions, and every evasion in check) until the side to move does better to stop.
    /// Returns its score, from the side to move's point of view, and its line. CNP's
    /// `engine/evaluate` with `quiet` evaluates the position at the end of that line, which is
    /// where a search's score comes from when its principal variation ends mid-exchange.
    pub(crate) fn quiet_line(&mut self, pos: &Position) -> (i32, Vec<Move>) {
        self.refresh_accumulator(pos);
        let mut line = Vec::new();
        let score = self.quiescence(pos, 0, -INFINITY, INFINITY, &mut line);
        (score, line)
    }

    /// The static evaluation of `pos`, at `ply`: the network's, from the accumulator the
    /// search has kept up to date, or the hand-crafted evaluation's (also with a network in
    /// the endings it plays better, see [`Searcher::hand_crafted_ending`]).
    fn evaluate(&self, pos: &Position, ply: u32) -> i32 {
        match &self.nnue {
            Some(nnue) if !self.hand_crafted_ending(pos) => {
                let acc = &nnue.accumulators[ply as usize];
                // Debug builds check every incremental update against a fresh computation.
                debug_assert!(
                    *acc == nnue.net.accumulator(pos),
                    "accumulator out of step at ply {ply}"
                );
                let value = nnue.net.evaluate(acc, pos.side_to_move());
                value.clamp(-NNUE_EVAL_LIMIT, NNUE_EVAL_LIMIT)
            }
            _ => evaluate(pos, self.params),
        }
    }

Getting this right everywhere is the hard part. The search makes moves in five places: the main move loop, null-move pruning, ProbCut, the quiescence search and the root. Miss one and the network quietly evaluates the wrong position. So in a debug build every evaluation compares the accumulator it was handed with one computed from scratch, and a test searches positions with castling, en passant, promotions, Chess960 castling and checks. Leaving out the null move’s copy, or the quiescence search’s update, fails it at once.

The network’s score is clamped to ±20,000 centipawns, so that nothing it computes could be taken for a tablebase win or a mate. Correction history (chapter 14.2) corrects it just as it corrects the hand-crafted evaluation.

Since the first network passed its test (below), it is built in and used by default. The UCI option EvalFile, the shell’s set evalfile and cb2026 bench NET load another; none, or the nnue switch, goes back to the hand-crafted evaluation, and then the bench is what it was before networks, to the node.

A lesson from a random network

The first test searched with a network of random weights, and on one position, Kiwipete, it took 11 million nodes to reach depth 4, where the hand-crafted evaluation needs 5,000. The accumulators were right. The evaluation was the problem: random scores tell the quiescence search nothing, so almost no capture sequence is cut off early, and Kiwipete is full of captures. A network built by hand to count material, a sensible evaluation if a simple one, searched 26,000. A network trained on only 134,000 positions searches the bench in 4.6 times as many nodes as the hand-crafted evaluation. Pruning is only as good as the evaluation it trusts.

Each node costs about 5% more with a 256-wide network than with the hand-crafted evaluation: 1.50 million nodes a second against 1.57 million on the M1 Max, in plain Rust that the compiler vectorises.

The first network

The first network’s data took the M3 Max 12.4 hours: 1,344,000 games of the hand-crafted evaluation against itself, on 9 threads, gave 104 million positions, 3.3 GB. White won 40.5% of the games, 23.8% were drawn and Black won 35.6%, and 30% of the scores were within a pawn of level, thanks to the balanced starts. Training took 7 minutes on the M3 Max’s GPU for a network of 256 and 13 minutes for one of 512.

The losses tell how it went. The 256 network’s validation loss fell from 0.0103 after the first epoch to 0.0094 after the twentieth, with a visible step at the sixteenth, where the learning rate drops. Its training loss ended at 0.0090, close to the validation loss: it learnt chess, not the positions. (The 134,000-position sample above ended at 0.047.) The 512 network did a little better on unseen games, 0.0093, but its training loss pulled further away, 0.0086: with 100 million positions, a wider network starts to learn the positions by heart.

Then the checks that a network is sane before it plays a game:

The material is roughly where the hand-crafted evaluation puts it, the pawn about to promote is worth far more to the networks than to the hand-crafted evaluation, and every position mirrored (the colours swapped and the board turned round) scores exactly the same. The bench, the other side of the lesson above: with the 256 network the search needs 716,942 nodes, fewer than the hand-crafted evaluation’s 749,181, where the weak networks needed several times as many. A good evaluation prunes well.

The first games: 200 at one second each plus 0.01 a move, the 256 network against the hand-crafted evaluation, ended 172 wins, 4 losses and 24 draws, about +424 Elo ± 71. Games that fast exaggerate any difference in evaluation, so that is a first sign rather than a measurement; the SPRT at the usual 8 seconds plus 0.08 is what counts. It passed after 14 games, all of them won: the log-likelihood ratio reached 3.21, beyond the 2.94 that accepts the gain. A result that lopsided proves the network stronger but not by how much: with no losses or draws there is nothing to measure a difference from, and that takes a fixed number of games. Stage 9’s piece-square tables passed the same way, 14 to 0.

The network by default

Passing made the network the default. Its file is part of the source (395 KB, in nets/, with a note of how it was made), and the build puts its bytes into the binary, where the engine reads them the first time it needs them:

/// The default NNUE network, built into the binary: gen3, 256 wide, trained on 201 million
/// positions of gen2's self-play; gen2 learnt from gen1's, and gen1 from the hand-crafted
/// evaluation's. It beat gen2 by SPRT, about +53 Elo (cjus/chessengine#222), as gen2 had beaten
/// gen1 (cjus/chessengine#214) and gen1 the hand-crafted evaluation (cjus/chessengine#195).
/// `nets/README.md` says how each was made.
const BUILTIN_NETWORK: &[u8] = include_bytes!("../../../nets/cb2026-gen3-256.nnue");

/// What `EvalFile` shows when the built-in network is in use, and accepts to choose it.
pub const BUILTIN_NETWORK_NAME: &str = "<built-in>";

/// The built-in network, read the first time it is asked for and shared from then on.
pub fn builtin_network() -> Arc<Network> {
    static NETWORK: OnceLock<Arc<Network>> = OnceLock::new();
    NETWORK
        .get_or_init(|| {
            Arc::new(
                Network::from_bytes(BUILTIN_NETWORK)
                    .expect("the built-in network is a valid network file (tested)"),
            )
        })
        .clone()
}

EvalFile now shows <built-in>. Empty, or none, means the hand-crafted evaluation, and a path another network; the shell’s set evalfile, cb2026 bench and datagen --evalfile read the same words, decided in one place (EvalChoice). Two kinds of position keep the hand-crafted evaluation even so.

The strength levels. Chapter 19 calibrated them with the hand-crafted evaluation. With the network the same node limits would play stronger, and the ratings would be wrong, so below full strength the search keeps the evaluation they were measured with, until they are calibrated again.

A lone king. Making the network the default broke one test: king and queen against king, played at 3,000 nodes a move, with which the self-play tools check that they record a won game. The network drew it. Five such endings, each played out by both evaluations:

To mate a lone king, the side ahead drives it to the edge, and the hand-crafted evaluation rewards that, through its king tables and mobility. The network was trained on balanced games, where a bare king is rare, and it scores those positions much alike: a big lead, whatever the kings do. So the search shuffles until a repetition or the fifty-move rule ends the game. A deep search finds the mate anyway, which is why the games at 8 seconds never showed it, but fast games and shallow analysis don’t. Stockfish kept special evaluations for such endings beside its networks for years. Here, when exactly one side has only its king, the search uses the hand-crafted evaluation (the loneking switch), and the network wins all five at 3,000 nodes, as quickly as the hand-crafted evaluation.

Small pawnless endings. The lone king wasn’t the only gap. Before v1.0.0 was tagged, a test of endgames played out won positions, each confirmed by the Lichess tablebases, with cb2026 on both sides and no tablebases of its own, once with each evaluation. Where the defender keeps a pawn, or a rook against a rook and pawn, the network converted as often as the hand-crafted evaluation or more. With a queen against a single piece it failed, at 200,000 nodes a move, about what a game at 8+0.08 gives a move:

The cause is the lone king’s again. A queen ahead is a big lead to the network wherever the kings stand, and nothing in its few such training positions showed it how to make progress, so nearly every failure was a draw by the fifty-move rule. A queen against a rook is hard for the hand-crafted evaluation too, since the rook can harass the king for many moves, but its king terms push the defending king towards the edge, and that is enough most of the time. So the pawnless switch widens the rule: in a pawnless ending of at most five men (three pieces besides the kings, PAWNLESS_ENDING_PIECES), the search uses the hand-crafted evaluation. Those are the endings the common Syzygy tables solve, so an engine with tablebases plays them perfectly with either evaluation. With the switch, the network converts exactly what the hand-crafted evaluation does, 32, 15 and 15, since the search is the same. Its SPRT, from 100 such endings, wins and draws (tests/data/pawnless-endings.epd), passed at +39 ± 16 Elo in 1,195 games, and 1,000 games from the usual openings showed nothing else changed, +1.7 ± 15.3 (cjus/chessengine#208).

The bench now searches with the network: 715,827 nodes, against the hand-crafted evaluation’s 749,181. A few of its positions reach a bare king or a small pawnless ending deep in the search, where the hand-crafted evaluation takes over. Every commit message’s Bench: line changes with it.

A new evaluation is a new way for the engine to fail: an accumulator updated at every move, and nodes about 6% slower, so less time to spare. Before v1.0.0 was tagged, the release’s robustness run was played again with the network: 10,000 games against itself, 6,000 of them at 1+0.01, with no time losses, illegal moves, failures or aborted games. So were 2,400 more with four search threads each, which start and stop on every move.

Against other engines the network counts for even more than the SPRT could say. In the gauntlet of chapter 19, seven rated engines at 8+0.08, the engine with the network rated 3236 ± 16, where the hand-crafted evaluation had rated 2782 ± 11: about 450 Elo. The reference engines are now too weak for a sharp figure. The engine won at least 96.6% of its games against five of the seven, and such scores say little, so the rating rests on the two strongest, against which it performed at 3223 and 3245.

The second network

The first network learnt from the hand-crafted evaluation’s games: that evaluation’s searches gave the scores, and its play chose the positions and decided how the games ended. The usual next step is for a network to teach its successor. Its searches judge positions better, and its games go where a stronger player goes. So the second network, gen2, learnt from the first network’s own games, with everything else as before: the same balanced starts, 6,000 nodes a move, about 100 million positions and 20 epochs. Whatever changed came from who played.

The data took the M3 Max 15.1 hours, against 12.4 for the first: 1,288,000 games gave 102.9 million positions, each taking about 23% longer to make than with the hand-crafted evaluation. White won 40.9% of the games, 25.9% were drawn (23.8% before) and Black won 33.2%. Training took 7 minutes for a network of 256 and 13 for one of 512, as before.

The validation losses ended at 0.0097 for 256 and 0.0096 for 512, above the first networks’ 0.0094 and 0.0093. That says nothing about which is better: each loss is measured on its own generation’s games, whose targets came from a different player, so losses compare only networks trained on the same data. The gap between the training and validation losses tells the same story as before: 0.0003 for 256 and 0.0007 for 512, which again learnt more of its positions by heart.

The first network’s checks, again:

The material barely moved, and the dead draw came closer to zero, most of all in the wider network. All 285 positions of the test openings and the endgame tests score the same mirrored, in all three networks. The 256 network’s bench is 701,720 nodes, against gen1’s 715,827, at the same speed. The 512 network’s is 732,931, and it searches 25% fewer nodes a second.

In the endgame test above, with the rules for a bare king and small pawnless endings as shipped, gen2 converts exactly what gen1 does, 98 and 108 of 130 positions at 3,000 and 20,000 nodes a move: those endings get the hand-crafted evaluation either way. Where the network does play, it was quicker: a queen against a pawn took 19 plies on average at 20,000 nodes, against gen1’s 46. Without the rules, at 200,000 nodes, gen2 had learnt a little of what the queen endings need:

Better, but short of the hand-crafted evaluation, so the pawnless rule stays. The network’s own games hold few such endings to learn from; data with more of them is a job for the next generation.

None of these checks says whether gen2 plays better. That takes games. In the SPRT against gen1 (cjus/chessengine#214), gen2 given by EvalFile against the built-in gen1 at 8+0.08, gen2 won 98 games, lost 16 and drew 95, about +144 ± 35 Elo, and passed after 209 games. That is a big step for a network of the same size trained the same way on as many positions, and it came from the data alone: the first network’s searches gave better scores, and its games went where a stronger player goes. The figure is likely too high. An SPRT that stops this early stops because the games went well, and self-play overstates gains against other engines. gen2 is clearly stronger; how much stronger against other engines is a gauntlet’s question.

v1.0.0 wasn’t tagged yet, so gen2 replaced gen1 as the built-in network before it was (cjus/chessengine#216). Only the weights changed, so the code the release’s tests had run stayed the same, and the bench went from 715,827 nodes to 701,720. The gauntlet and the robustness run were played again with gen2 before the tag (cjus/chessengine#215). In the gauntlet it rated 3286 ± 17, 50 above gen1’s 3236 ± 16: about a third of what the SPRT measured, since self-play overstates gains against other engines. Against the two engines that carry the rating it scored 91.6% and 78.4%, where gen1 had scored 86.8% and 75.6%. The robustness run’s 10,000 games were clean again.

The third network

gen3 changed two things at once: it learnt from gen2’s games, and from twice as many of them, 200 million positions. More positions are the cheapest gain a network can get, since they need no new code, only time on the test Mac: 29.1 hours for these, 45 chunks of 56,000 games (cjus/chessengine#219). The games came out much as gen2’s had: White won 40.8%, 26.2% were drawn and Black won 33.0%. Training took 13 minutes for a network of 256 and 24 for one of 512, twice as long as before, since each epoch held twice the records.

The extra data showed first in the losses. The gap between training and validation loss halved, to 0.00018 for 256 and 0.00038 for 512: with twice the positions, a network learns fewer of them by heart. The wider network gained more. Its validation loss pulled ahead of the 256 network’s by 0.00029, against 0.00011 with gen2’s data, since a network with more weights needs more positions before its size pays.

The same checks again, now against gen2:

The dead draw came closer to zero again, and all 285 test positions score the same mirrored. The networks are settling: across those positions gen3’s 256 network differs from gen2’s by 37 centipawns on average, where gen2 had differed from gen1 by 78. Its bench is 767,812 nodes, more than gen2’s 701,720, at the same speed; the 512 network’s is 868,798.

In the endgame test, with the rules as shipped, gen3 converts what gen2 does at 3,000 nodes a move, 98 of 130, and one fewer at 20,000, 107, a rook and pawn against a rook that it drew. Without the rules it won a few more queen endings at 200,000 nodes: 41 of 60, against gen2’s 38 and the hand-crafted evaluation’s 50. Each generation learns a little more of them, but slowly, so the pawnless rule stays.

Whether gen3 plays better is again the SPRT’s question. Against gen2 (cjus/chessengine#222), at 8+0.08, gen3 won 176 games, lost 86 and drew 333, about +53 ± 19 Elo, and passed after 595 games. Each generation gains less than the one before, +144 for gen2 and +53 for gen3, as the networks’ evaluations close in on what their games can teach them. The bench had pointed the other way, 767,812 nodes against gen2’s 701,720: fewer nodes often means a sharper evaluation, but not always, and only games decide. v1.0.0 ships gen2 even so: it had been through the release’s tests, and a gain this size waits for the next release.

Twice the data had helped the 512 network most, so it had its own SPRT, against gen3’s 256 (cjus/chessengine#223). After 8,180 games at 8+0.08 it stood at 1,653 wins, 1,627 losses and 4,900 draws, about +1 ± 5 Elo, with no verdict, and was stopped. The wider network knows more but searches about a quarter fewer nodes a second, and at this time control the two cancel. So the 256 stays the candidate: as strong, and faster. Wider networks tend to do relatively better at longer time controls, where each extra ply is worth less and so a quarter fewer nodes costs less; that test is for a later network.

v1.0.0’s tag went on the commit its tests had run on, so main could move on: gen3 became the built-in network for v1.1 (cjus/chessengine#228), and the bench went from 701,720 nodes to 767,812. Its gauntlet (cjus/chessengine#227) rated it 3379 ± 21, 93 above gen2: more than the SPRT’s +53, where gen2’s gauntlet had shown only a third of its SPRT. Against Stash v27.0, the strongest reference engine, it scored 88.4%, where gen2 had scored 78.4%. One explanation, untested: self-play measures a network against a near-copy of itself, which shares its blind spots, while twice the data mostly taught gen3 to generalise (the smaller gap between its training and validation losses), which may count for more against engines unlike it. The figure is still blunt: five of the seven engines lost nearly every game, so it rests on two.

Pitfalls

• Trainer and engine must agree on the numbering, the layers and the scales. They share crates/nnue, and tests pin it down: every input has its own number, a move’s changes turn one position’s inputs into the next one’s, updates match recomputing, and mirrored positions score the same.
• An eval scale that doesn’t match quietly changes every margin in the search.
• Overflow in the integer arithmetic gives silently wrong scores. The weight limit and the bounds above rule it out, and a test runs the largest possible weights.
• Endings the data never showed. A network learns only what its positions show. Balanced self-play has few bare kings, so the first network couldn’t drive one to the edge. Test the endings every beginner must win, king and queen or rook against king, at low node counts, where the search can’t cover the gap.
• A network that quietly doesn’t load. An engine that can’t read its EvalFile says so in an info string and plays on with the hand-crafted evaluation, and a match tool ignores info strings, so a test of the network would measure the old evaluation against itself. mac-sprt.sh loads each network through UCI before the match and stops if it doesn’t load.
• A move the accumulators miss. Every place the search makes a move must update the child’s accumulator; a debug build checks each evaluation.
• A GPU kernel is as slow as its slowest invocation. One invocation per weight looks natural, but if some weights have far more work than others (the kings’ inputs in a batch), the GPU sits idle while they finish. Cut the work into equal pieces and add them up after.
• A benchmark that doesn’t look like the real run. A small file made the GPU wait at every progress report and look slower than it is.
• Scores from the wrong side. The search scores for the side to move; records keep White’s side, like the result, and the trainer turns both to the side to move’s view. Mixing the two up teaches the network nonsense for every position with Black to move.

Further reading

• Chess Programming Wiki: NNUE.
• WGSL, WebGPU’s shading language, and wgpu, the Rust library the trainer’s GPU backend runs on.
• Yu Nasu, Efficiently Updatable Neural-Network-based Evaluation Functions for Computer Shogi (2018), which introduced NNUE.