MegaDepth-1500
- Unsteered1
- Exact156
- PaSteer-free272
+129 recovered that Exact misses
WACV 2027 Algorithms Track Round 1 Accept
PaSteer Pair-Relative Partial Equivariance for Rotation-Robust Keypoint Descriptor Matching
TL;DR Keypoint descriptors trained on upright photos fail when the image is rotated. Steerers fix this for a frozen descriptor with a small linear map that is forced to chain exactly like a rotation at every angle; PaSteer relaxes that map, fits it to the rotations image pairs actually show, and, on frozen descriptors, improves matching on all four benchmarks at the same inference cost.
Hero pair: MegaDepth-1500, 112° in-plane rotation, correct matches
Exact (Steerers)
148
PaSteer-free (ours)
242
113 recovered that Exact misses, 19 of Exact's lost, net +94. Same frozen DeDoDe-B, keypoints and matcher; only the steering changes.
The result in four numbers, each with its setting
- +1.4to+9.5 Better than Exact on all four benchmarks Points gained on every rotated metric: Roto-360, MegaDepth-1500, AIMS, IMC-PT. frozen DeDoDe-B, full-scale All rows
- 49.0→49.3 Upright: within a fraction of a point MegaDepth-1500 without rotation, AUC@5°. Matched budget: 49.8 → 49.0. frozen DeDoDe-B, Exact → PaSteer-free Control row
- 51.1→80.6 Larger DeDoDe-G: the gap widens Roto-360 % correct @5px, Exact → PaSteer-free. Paper's reading: the cost of the exact constraint grows with descriptor capacity. frozen DeDoDe-G DeDoDe-G
- 0.85M Small add-on, same cost 6.3% of the DeDoDe-B descriptor; maps precomputed for 8 angles. inference cost identical to Exact How
Terms. AUC@5°: the area under the cumulative pose-accuracy curve for error thresholds from 0° to 5°, in percent; higher is better. C4 / SO2: the second image is rotated by a multiple of 90° / by an arbitrary angle. SO(2): the group of all in-plane rotations.
ScopeA retrofit for frozen descriptors. With joint descriptor + steerer training, Exact and PaSteer-free reach parity on DeDoDe-B (Exact keeps an edge on DeDoDe-G).01Insight
For a frozen descriptor, the best steering maps do not chain like rotations
A steerer is a small linear map ρ(θ) that “rotates” a descriptor: it turns the descriptor of an upright keypoint into the one the same keypoint has after the image turns by θ. The descriptor network itself stays frozen.
Exact steering (Steerers, Bökman et al., CVPR 2024) makes this map a group representation: one generator D for every angle, ρ(θ) = exp(θD), so the 30° map applied twice is exactly the 60° map.
Fit the best map separately for each angle on a frozen descriptor, though, and those maps do not chain:
0.60–0.70
Composition violation of the best per-angle maps. 0 means they chain perfectly, as an exact steerer always does. Frozen DeDoDe-B, HPatches. Formula in Method details.
So PaSteer stops forcing exactness and measures equivariance where image pairs actually live: against q(Δθ), the distribution of relative rotations between the two images of a pair, not uniformly over all angles. That is pair-relative partial equivariance.
PaSteer-free nearly reaches the optimal floor
Steering error 1 − cos per rotation angle θ: how far the steered descriptor points from the true rotated one (0 = it lands exactly on it; lower is better). The floor ρ* is an oracle: the best linear map fitted separately at each angle. Frozen DeDoDe-B, HPatches evaluation set (paper Fig. 4). Exact here is our retrained steerer; the sweep below uses the released Steerers model.
Drag the rotation-angle slider, use the arrow keys, or point at the chart to read the three errors at one angle.
Data table
| θ | Exact | PaSteer-free | oracle ρ* |
|---|---|---|---|
| 0° | 0.000 | 0.006 | 0.000 |
| 10° | 0.089 | 0.078 | 0.072 |
| 20° | 0.213 | 0.169 | 0.161 |
| 30° | 0.305 | 0.239 | 0.229 |
| 40° | 0.343 | 0.286 | 0.267 |
| 50° | 0.360 | 0.305 | 0.282 |
| 60° | 0.381 | 0.308 | 0.290 |
| 70° | 0.370 | 0.296 | 0.278 |
| 80° | 0.357 | 0.285 | 0.267 |
| 90° | 0.360 | 0.280 | 0.264 |
| 100° | 0.381 | 0.289 | 0.276 |
| 110° | 0.421 | 0.318 | 0.300 |
| 120° | 0.441 | 0.327 | 0.308 |
| 130° | 0.442 | 0.325 | 0.308 |
| 140° | 0.460 | 0.324 | 0.306 |
| 150° | 0.479 | 0.315 | 0.297 |
| 160° | 0.464 | 0.288 | 0.272 |
| 170° | 0.446 | 0.260 | 0.248 |
| 180° | 0.452 | 0.248 | 0.236 |
Evidence: steering matters when q(Δθ) has large rotations
RotoReal: MegaDepth-1500 pairs resampled under a measured handheld rotation distribution (T1) or uniformly over all angles (T3). Pose AUC@5°, frozen DeDoDe-B, steerers at matched training, only the steering changes (paper Tab. 8). Bars on a 0–60 scale.
T1 handheldT3 uniform
Unsteered: T1 50.8, T3 10.7 AUC@5°.
Exact: T1 50.7, T3 19.7 AUC@5°.
PaSteer-soft: T1 50.7, T3 24.7 AUC@5°.
PaSteer-free: T1 49.9, T3 35.0 AUC@5°.
Released Steerers model (ref.): T1 50.7, T3 30.6 AUC@5°.
Handheld rotations: all steerers within about a point. Uniform rotations: steering decides the result, and PaSteer-free leads. The released row was trained differently and is shown for reference.
02Method
Three steps. Only the steering map changes.
-
Step 1
Freeze the descriptor
Take a released descriptor as it is. Nothing in the backbone is retrained.
- Descriptor DeDoDe-B or DeDoDe-G, frozen
- Detector, keypoints unchanged (SO(2) detector)
- Matcher unchanged
-
Step 2
Relax the steerer, train it on the rotations real pairs have
Replace the exact map by a relaxed one, ρ tilde(θ), trained only on angles drawn from q(Δθ). An orbit is the path a descriptor traces as the image turns.
- Exact exp(θD), the baseline
- PaSteer-soft exp(θD + Δ(θ)), Δ(0) = 0
- PaSteer-free free-form Fourier map, K = 6
-
Step 3
Try 8 rotations, keep the best
Steer each query descriptor to the C8 probe angles {0°, 45°, …, 315°} and keep the best-scoring angle per keypoint.
- Maps precomputed once per steerer
- Matching same max-similarity matcher
- Cost same inference cost as Exact
Details: equations, why exact maps fall short, training, pipeline
Exactρ(θ) = exp(θD)Steerers representation, retrained at our budget. Rigid, with exact upright identity ρ(0) = I.
PaSteer-softρ tilde(θ) = exp(θD + Δ(θ))Low-rank Fourier residual Δ(θ) = Σk Sk sin kθ + Ck(cos kθ − 1), so Δ(0) = 0: zero upright steering error.
PaSteer-freeρ tilde(θ) = A0 + Σk=1..K Ak cos kθ + Bk sin kθDrops the homomorphism and orthogonality constraints. K = 6: 0.85M parameters, 6.3% of DeDoDe-B; a one-time training and storage cost, not a per-match one.
Why exact maps fall short. For a fixed descriptor f and angle θ, the best linear steerer is a ridge least-squares solution, ρ*(θ) = C(θ)(G + λI)−1 with C(θ) = E[f(RθI) f(I)⊤] and G = E[f(I) f(I)⊤]. As λ → 0 it obeys the composition law ρ*(θ+φ) = ρ*(θ)ρ*(φ) only if C(θ+φ) = C(θ)G−1C(φ), which upright-trained descriptors violate (composition violation 0.60–0.70, orthogonality deviation 0.85–1.04; both are 0 for an orthogonal representation).
Training. With the descriptor frozen, only ρ tilde is trained, minimizing EΔθ∼q EI[1 − cos(f(RΔθI), ρ tilde(Δθ) f(I))]. The distribution q both draws and weights the training angles.
Matching. Sab = maxθ∈Θ ⟨ρ tilde(θ)fAa, fBb⟩ over the C8 grid Θ (normalized descriptors), followed by mutual-nearest-neighbour dual-softmax.
03Results
Gains on every benchmark, largest under large rotation
Exact is the Steerers representation exp(θD) retrained at our budget, with the same detector, keypoints, matcher and training data. Only the steering map differs.
Exact → PaSteer-free on frozen DeDoDe-B, full-scale training
ExactPaSteer-freeaxis 0–100, gain in points on the right
Roto-360, % correct matches @5px: Exact 92.0, PaSteer-free 93.4, gain +1.4.
MegaDepth-1500 · C4, pose AUC@5°: Exact 32.5, PaSteer-free 36.4, gain +3.9.
MegaDepth-1500 · SO2, pose AUC@5°: Exact 20.0, PaSteer-free 22.9, gain +2.8.
AIMS, average precision: Exact 48.6, PaSteer-free 50.1, gain +1.5.
IMC-PT · C4, pose AUC@20°: Exact 67.2, PaSteer-free 75.2, gain +8.0.
IMC-PT · SO2, pose AUC@20°: Exact 41.1, PaSteer-free 50.6, gain +9.5.
MegaDepth-1500 · upright, control, no rotation · pose AUC@5°: Exact 49.0, PaSteer-free 49.3, gain +0.3.
Roto-360: % correct matches @5px. MegaDepth-1500 (MD-1500): pose AUC@5° under C4 or SO2 rotations of the target. AIMS: average precision over the full FMAP set (323 frames). IMC-PT: pose AUC@20°. Paper Tab. 1 and Tab. 2.
A larger descriptor, a larger gap: DeDoDe-G
ExactPaSteer-freeDeDoDe-B gain shown for reference
Roto-360, % correct matches @5px: Exact 51.1, PaSteer-free 80.6, gain +29.5; on DeDoDe-B the gain is +1.4.
MegaDepth-1500 · C4, pose AUC@5°: Exact 12.9, PaSteer-free 21.6, gain +8.7; on DeDoDe-B the gain is +3.9.
MegaDepth-1500 · SO2, pose AUC@5°: Exact 11.8, PaSteer-free 20.1, gain +8.3; on DeDoDe-B the gain is +2.8.
Frozen DeDoDe-G (DINOv2 backbone), means over 3 training seeds (2 for Exact). The paper reads the wider margin as the cost of enforcing an exact representation growing with descriptor capacity. Paper Tab. 3.
Ahead at most angles, including the 90° probe
MegaDepth-1500 pose AUC@5° as the relative in-plane rotation Δθ is swept. Frozen DeDoDe-B. Here Exact is the released Steerers model (their Setting-A; paper Fig. 5). Shaded as in the paper: the C8 dip, the discretization-worst angles where exact steering is weakest; triangles mark the probe angles 0°, 45°, 90°.
Drag the slider in the side card, use the arrow keys, or point at the chart; the card says where each angle sits.
PaSteer-free also leads at the 90° probe itself (43.3 vs 35.1), so the gain is not just filling gaps between probes. Not everywhere: Exact is slightly ahead at 0° (50.9 vs 50.4) and at 70° (28.0 vs 27.2).
Data table
| Δθ | Unsteered | Exact (released Steerers model) | PaSteer-free |
|---|---|---|---|
| 0° | 50.9 | 50.9 | 50.4 |
| 10° | 49.2 | 49.2 | 50.1 |
| 20° | 46.3 | 46.3 | 47.3 |
| 30° | 40.7 | 40.8 | 41.0 |
| 40° | 29.7 | 30.3 | 33.5 |
| 50° | 9.1 | 24.3 | 33.1 |
| 60° | 0.2 | 23.8 | 29.1 |
| 70° | 0.0 | 28.0 | 27.2 |
| 80° | 0.0 | 31.2 | 35.9 |
| 90° | 0.0 | 35.1 | 43.3 |
Controls: the gain comes from relaxing, not from budget or parameters
Same budget, three steerers
Roto-360 % correct matches @5px, frozen DeDoDe-B, all steerers trained on the same HPatches data and budget (paper Tab. 4). Exact < PaSteer-soft < PaSteer-free. Bars on a 0–100 scale.
Unsteered: 20.5 % correct @5px.
Exact: 78.3 % correct @5px.
PaSteer-soft: 90.1 % correct @5px.
PaSteer-free: 95.1 % correct @5px.
Three “Exact” numbers on this page, three regimes: full-scale MegaDepth training (Roto-360 92.0, chart above), this matched HPatches budget (78.3), and the released Steerers model, Setting-A (93.0, reference only, different protocol).
Relaxation, not parameter count
PaSteer-free with Fourier order K, minus Exact, on MegaDepth-1500 C4 pose AUC@5° (paper Tab. 7). Parameters as % of the DeDoDe-B descriptor; Exact uses 0.48%.
K=1, 1.5% parameters: −11.6 over Exact.
−11.6
K=2, 2.4% parameters: −2.4 over Exact.
−2.4
K=3, 3.4% parameters: +2.3 over Exact.
+2.3
K=4, 4.4% parameters: +2.7 over Exact.
+2.7
K=6, 6.3% parameters: +5.6 over Exact.
+5.6
C4 shown; ΔSO2 follows the same pattern (−7.9 at K=1 to +3.7 at K=6, Full tables). K=1 already has more parameters than Exact yet is markedly worse; the margin turns positive from K=3. PaSteer-soft's out-of-band error barely moves with more rank (0.374 at r=12, 0.379 at r=24).
Full tables
Matched-budget comparison with released models as reference
Roto-360 correct matches (%), frozen DeDoDe-B, same HPatches training data and budget for all three steerers. Released models used a different protocol and are listed for reference only; SO2-Spread-B is a jointly trained released descriptor.
| steering (matched budget) | @3px | @5px | @10px |
|---|---|---|---|
| none (unsteered) | 19.7 | 20.5 | 21.6 |
| Exact | 77.3 | 78.3 | 79.2 |
| PaSteer-soft (r=12) | 89.1 | 90.1 | 91.0 |
| PaSteer-free | 93.9 | 95.1 | 96.0 |
| Released models, reference only (different protocol) | |||
| Exact, released Steerers model (Setting-A) | 91.4 | 93.0 | 94.1 |
| Steerers SO2-Spread-B (joint) | 95.0 | 97.0 | 98.5 |
The ordering also holds on AIMS (AP 37.1 / 44.4 / 51.2) and MegaDepth-1500 (supplement).
RotoReal table: handheld vs uniform rotations
MegaDepth-1500 pose AUC@5° (paper Tab. 8).
| steering | T1 handheld | T3 uniform |
|---|---|---|
| none (dual softmax) | 50.8 | 10.7 |
| Exact | 50.7 | 19.7 |
| PaSteer-soft (r=12) | 50.7 | 24.7 |
| PaSteer-free | 49.9 | 35.0 |
| Released Steerers model (Setting-A) | 50.7 | 30.6 |
Harmonic order K and mechanism tables
Δ is PaSteer-free(K) minus Exact on MD-1500 AUC@5° (paper Tab. 7).
| steerer | params % | Roto@5px | C4 AUC@5° | SO2 AUC@5° | ΔC4 | ΔSO2 |
|---|---|---|---|---|---|---|
| Exact | 0.48 | 92.1 | 32.3 | 20.3 | – | – |
| K=1 | 1.5 | 89.0 | 20.8 | 12.4 | −11.6 | −7.9 |
| K=2 | 2.4 | 92.2 | 29.9 | 19.2 | −2.4 | −1.0 |
| K=3 | 3.4 | 93.1 | 34.6 | 22.6 | +2.3 | +2.3 |
| K=4 | 4.4 | 93.4 | 35.1 | 23.5 | +2.7 | +3.2 |
| K=6 (default) | 6.3 | 93.6 | 37.9 | 23.9 | +5.6 | +3.7 |
Steering error 1 − cos inside and outside the partial-equivariance band, and at 0°; parameters relative to the exact generator (paper Tab. 6).
| steerer | +params vs Exact generator | in-band | out-of-band | err(0) |
|---|---|---|---|---|
| Exact | – | 0.152 | 0.410 | 0.000 |
| PaSteer-soft (r=12) | +75% | 0.156 | 0.374 | 0.000 |
| PaSteer-soft (r=24) | +150% | 0.155 | 0.379 | 0.000 |
| PaSteer-free | +1200% | 0.123 | 0.297 | 0.006 |
04Gallery
Four datasets, three steerers, same keypoints
PaSteer-free: the blue lines are correct matches that exact steering misses on the same keypoints.
correct match recovered by PaSteer-free rejected / outlier
IMC-PT
- Unsteered2
- Exact164
- PaSteer-free323
+164 recovered that Exact misses
Roto-360
- Unsteered0
- Exact109
- PaSteer-free188
+92 recovered that Exact misses
AIMS
- Unsteered9
- Exact108
- PaSteer-free233
+133 recovered that Exact misses
1 / 4
Illustrative, not a benchmark: one pair per dataset from the supplement (Figs. 6–9), with the same frozen DeDoDe-B descriptor, SO(2) detector and keypoints for all three steerers.
How these pairs were chosen and counted
The synthetic rotations (112°, 70°) lie off the C8 probe grid, where the exact representation is weakest. Drawn lines are capped for legibility; counts are the true numbers. “Recovered” counts correct PaSteer-free matches that Exact misses. The MegaDepth pair here is a different pair from the hero pair.
- MegaDepth-1500: scene 0022_0.5_0.7 (pair 2); ground-truth-verified matches.
- IMC-PT: reichstag; ground-truth-verified matches.
- Roto-360: sequence v_charing; GT-homography reprojection ≤ 5 px.
- AIMS: astronaut frame ISS052-E-27335 vs. a satellite tile; MAGSAC homography inliers.
05Scope and limitations
A retrofit for frozen descriptors, not a replacement for joint training
Where PaSteer helps
- An already-trained, released descriptor that you cannot or do not want to retrain.
- Pairs with large relative rotations. On RotoReal, uniform rotations (T3) lift Exact 19.7 → PaSteer-free 35.0 AUC@5°, while handheld rotations (T1) leave all steerers within about a point.
- Frozen DeDoDe descriptors matched with max-similarity steering on a C8 grid (the setting evaluated): train the steerer once, same inference cost.
Where it does not
When descriptor and steerer are trained jointly, the two reach parity on DeDoDe-B, suggesting that joint training lets the descriptor absorb the representation constraint (paper Sec. 4):
| joint, DeDoDe-B | upright | C4 | SO2 |
|---|---|---|---|
| Exact | 50.3 | 47.0 | 43.3 |
| PaSteer-free | 49.7 | 47.1 | 43.6 |
The larger DeDoDe-G joint result shows the same scope boundary, with exact steering ahead (MD-1500 C4 AUC@5° 46.1 vs 41.5, SO2 42.3 vs 38.2; supplement).
Limitations
- Evaluated on DeDoDe descriptors only.
- Compared against exact steering, not against rotation-equivariant methods that retrain the backbone. Released Steerers models (different protocol) remain stronger on some benchmarks and are shown as reference only.
- Every pair uses the same C8 probe grid; the steerer does not adapt its degree of equivariance per pair.
- PaSteer-free is not exactly the identity at 0° (steering error 0.006); PaSteer-soft is, by construction.
06Acknowledgements
Thanks to the open-source work we build on
PaSteer builds heavily on these repositories. We thank their authors for releasing their code and models.
- Steerers Bökman et al., CVPR 2024 The exact steering framework and released models that PaSteer extends and compares against. github.com/georg-bn/rotation-steerers
- DeDoDe Edstedt et al., 3DV 2024 The keypoint detector and descriptors (DeDoDe-B, DeDoDe-G) that PaSteer steers, kept frozen. github.com/Parskatt/DeDoDe
- Partial G-CNNs Romero and Lohit, NeurIPS 2022 Learning partial equivariances from data: the idea of relaxing equivariance that PaSteer moves from network layers to the steering map. github.com/merlresearch/partial_gcnn
- REKD Lee et al., CVPR 2022 The Roto-360 benchmark used for evaluation. github.com/bluedream1121/REKD
07Paper
Abstract
Learned keypoint descriptors trained on upright imagery often fail under large in-plane rotation. Steerers address this by applying a single exact SO(2) representation to the descriptor space, but this global group constraint can be too rigid for keypoint descriptors, whose angular responses need not follow a single group representation. We propose pair-relative partial equivariance, which measures equivariance under the empirical distribution of relative rotations in image pairs, and PaSteer, a relaxed descriptor-steering framework that allows the steering map to leave the exact representation manifold while retaining the lightweight, backbone-free design of steering. PaSteer instantiates this relaxation with two variants: an identity-preserving residual steerer (PaSteer-soft) and a free-form Fourier steerer (PaSteer-free). We show that the per-angle optimal linear steerer for a fixed non-equivariant descriptor generally does not obey the group composition law, explaining why exact representations can be suboptimal in the frozen-descriptor regime. On Roto-360, MegaDepth-1500, AIMS, and IMC-PT, PaSteer consistently improves rotation-robust matching over an exact steerer on frozen DeDoDe descriptors, especially under large relative rotations, with minimal upright degradation.
PaperWACV 2027, PDF
Supplementaryproofs, extra tables, all qualitative pairs
BibTeX
@inproceedings{lee2027pasteer,
title = {Pair-Relative Partial Equivariance for Rotation-Robust Keypoint Descriptor Matching},
author = {Lee, Jongmin},
booktitle = {Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)},
year = {2027}
}
Acknowledgements
This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (RS-2026-25598824).