Symmetry Regularization for Quadruped Locomotion
Symmetry rewards, curriculum training, and sim-to-real evaluation for quadruped locomotion.
UC Berkeley · Locomotion · Reinforcement Learning · Symmetry in Motion
Can symmetry priors improve high-speed tracking without damaging a quadruped's gait at everyday speeds? I tested reward terms and a mirror-policy loss across A1 and Go1 simulation, then evaluated selected policies on a real Go1.
The Question
A policy can reach a high simulated speed yet move awkwardly at 2 m/s, or fail after transfer to hardware. The experiments therefore track three separate outcomes: maximum tested forward speed, the contact pattern at low and medium speeds, and whether the policy survives a real-world speed ramp.
Motion Priors And Training
PPO policies were trained with randomized dynamics and evaluated through command sweeps and foot-contact plots. Reward ablations isolate four relations between limbs or motion phases; a separate mirror loss asks the policy to produce mirrored actions for mirrored observations.
Diagonal Trot
Penalizes differences in joint positions between each diagonal pair, with speed-dependent weighting.
Gallop
Tests rear-leg air-time and front/rear joint-configuration terms, alone and with trot.
Time Reversal
Penalizes knee and calf velocities that fail to reverse between lift-off and touchdown.
Left-Right Timing
Penalizes air-time differences between left and right front legs and between rear legs.
Speed Is Only One Metric
In the initial 1,000-iteration A1 reward sweep, the baseline reached 5.3 m/s in simulation. Combining trot and gallop terms reached 6.3 m/s and produced a gallop; left-right timing reached 6.0 m/s, while some other variants had visibly poor or failing gaits. These are individual experiment records, not multi-seed averages.
A1, 1,000 training iterations. Bar lengths are relative to the fastest run in this sweep; gait quality was judged separately.
Checkpoint Rollouts Across Speed
The curriculum first trained for 200 iterations at a low-speed range, then continued for 1,300 iterations over the full command range. On A1, the baseline reached 6.0 m/s, time reversal 6.6 m/s, and left-right timing 6.7 m/s. The latter two-speed advantage came with a tradeoff: the notes flag poor low-speed gaits for left-right timing and its combination with time reversal.
The 2 and 6.6 m/s videos below are rollouts of the same time-reversal + left-right curriculum checkpoint. The 6.6 m/s time-reversal-only clip is a different checkpoint. The export labels a second copy of the 2 m/s file as 4 m/s, so it is omitted here.
Baseline Curriculum Rollout · 6.0 m/s
Low-Speed Gait Matters
A mirror-policy loss compares the action from an observation with the mirrored action from its reflected observation. In one A1 run, loss scale 1.5 yielded a more regular trot at 2 and 4 m/s without a low-speed curriculum; the unmodified run sometimes barely placed a foot. The records also warn that baseline outcomes varied, so this is a qualitative comparison, not a robustness claim.
Higher mirror-loss weight reached 6.6 m/s in another run, but the notes describe a poorer tapping gait at low speed. More regular motion and peak speed did not always move together.
Testing The Same Idea On Go1
In the full-observation Go1 comparison, each policy trained for 1,500 iterations with a 5.0 m/s training command. The recorded maximum tested speeds were 5.2 m/s for the baseline, 5.8 m/s with time reversal, and 6.0 m/s with left-right reward plus mirror loss. All three were described as trotting at low and maximum speeds. These simulated speeds should not be read as hardware results.
Compare Baseline And Time-Reversal Plots
What Transferred To The Real Robot
Real-Go1 trials were much slower than full-observation simulation and exposed failures the simulated peak-speed numbers did not capture. A July 19 policy reached about 10.7 km/h at a 3.5 m/s command but fell at a 4.0 m/s command. With a 0.4 m/s² ramp on July 26, a diagonal-reward policy reached about 11 km/h; gallop and time-reversal variants fell before 10 km/h. An August trot-policy trial exceeded 12 km/h, while a later body-height adjustment improved its stance but did not exceed 12 km/h.
Embedded clips retain only the active test segments; the two full recordings remain linked above. Hardware speeds are recorded in km/h; simulated evaluation speeds above are in m/s.
Where The Experiments Point
Symmetry terms can improve a particular simulated speed sweep, but they are not a free, universal gain. Gait quality, observation design, command sampling, and hardware stability must be evaluated separately.
- Curriculum helped, but did not fix every gait. Left-right terms retained high-speed A1 performance while producing poor low-speed behavior in some checkpoints.
- Policy symmetry is worth testing alongside reward symmetry. Moderate mirror loss produced a cleaner 2 m/s trot in one no-curriculum A1 run; stronger weighting traded that behavior away.
- Simulation was not the final answer. Partial-observation runs were less stable, wider command sampling did not reliably raise top speed, and several fast-looking policies fell on hardware.
Source: 2024 experiment log and its 14 subpages. Reported maxima are observations from individual runs, not statistical confidence estimates.
This project was conducted at University of California, Berkeley with Lingfeng Sun and Boyuan Liang, advised by Prof. Masayoshi Tomizuka.