ARCHON / The record

01 / 07Equivariance under rotations unseen in training

Two routes, one answer.

The network

Layers···
Parameters···
Final training loss···

Act, then map. Or map, then act.

ARCHONEGNNLieConvLearned operator
Gap, at this rotation···
Median along the path···
Median over 50 seeds···

Equivariance means two routes round one square give one answer: act on the input and then map it, or map it and then act.

How to read this figure

Equivariance means two routes round one square give one answer: act on the input and then map it, or map it and then act. The figure walks a closed path of rotations that none of the networks met in training, and at each one draws both routes for the same input. Solid is act, then map. Dashed is map, then act. Rose is the gap between them, and the strip is that gap along the path, on a log scale.

The left pane is the network under test: its real layers at their real widths, lit by what each layer returned on this input, over its own training loss. ARCHON's operator here is built from the group itself, so its pane shows the group element moving through the rotation group and the channel it keeps. ARCHON and EGNN hold the gap at the rounding precision of the arithmetic, 7.9e-8 and 1.2e-7 in single precision over 50 seeds, and are level. LieConv lifts its input to the group by sampling, so it answers afresh on every call and carries a 1.9% error each time. The learned operator fits the map and holds 2.7e-4.

Satorras, Hoogeboom & Welling, arXiv:2102.09844 (2021) · Finzi, Stanton, Izmailov & Wilson, arXiv:2002.12880 (2020)

One-page brief