PPO is a valid Ant control
The current joint-order, bounded-action Ant PPO actor moves reliably under the same evaluation path used below.
Newton differentiability report
SHAC with MuJoCo Warp
The report now separates PPO controls from SHAC updates. PPO validates the current Ant setup; guarded SHAC improves that Ant baseline, while native MuJoCo-contact gradients remain the open blocker.
MJWarp PR #1423
DiffRL reference setup
PPO = positive control
SHAC = differentiable update
Open: native MuJoCo-contact VJP
Ant PPO control
...
Loading Ant PPO audit...
Ant SHAC guarded
...
Loading Ant SHAC audit...
Hopper SHAC
...
Loading Hopper audit...
Cheetah SHAC
...
Loading Cheetah audit...
Guarded SHAC improves Ant
The promoted Ant SHAC update uses Newton-contact gradients and MuJoCo-contact validation before accepting the checkpoint.
Direct MuJoCo-contact SHAC is not trusted
Finite differences still break around grazing foot contacts, so native MuJoCo-contact updates are not the headline path.
Scope
PPO rows
- Environment and action-order sanity checks.
- Not SHAC training evidence.
SHAC rows
- Differentiable policy updates.
- Ant headline uses guarded acceptance.
Removed from the main path
- Long debug archive tables.
- Non-diagnostic performance charts.
- Duplicate and historical clips.
Promoted Results
One chart, one table: current no-reset evaluations only.
No-Reset Forward Displacement
The Ant PPO row is a control. The other rows are promoted SHAC checkpoints.
Current Matrix
| Run | Role | Displacement | Failures | Read |
|---|
Where Things Still Fail
The main unresolved problem is not PPO, Hopper, Cheetah, or video capture. It is Ant contact-gradient reliability.
Joint-order PPO gradients
4.06e-4
Corrected Ant d6 PPO H=8 policy gradients match finite differences across contact settings.
Newton-contact path
1.37e-1
Newton contacts are noisy but usable for guarded SHAC updates with MuJoCo validation.
MuJoCo-contact state path
9.96e-1
Native MuJoCo-contact initial-state sensitivities remain the red path.
Gradient Failure Table
| Check | Mean relative error | Sign agreement | Read |
|---|---|---|---|
| Ant d6 PPO H=8 policy, MuJoCo / Newton / no contact | 4.06e-4 / 4.05e-4 / 4.06e-4 | 1.00 / 1.00 / 1.00 | PPO baseline is clean after fixing joint order. |
| Ant current H=2 policy, no-contact / Newton / MuJoCo | 2.22e-3 / 1.37e-1 / 1.26e-1 | 1.00 / 1.00 / 1.00 | Contact gradients are noisier than no-contact; Newton remains guardable. |
| Ant current H=2 fixed actions, no-contact / Newton / MuJoCo | 7.79e-2 / 1.20e-1 / 3.74e-1 | 1.00 / 1.00 / 0.75 | The native MuJoCo-contact row is localized to the first contacted step. |
| Ant current H=2 initial state, no-contact / Newton / MuJoCo | 1.67e-4 / 2.90e-4 / 9.96e-1 | 1.00 / 1.00 / 0.50 | Native MuJoCo-contact state-position sensitivities are still wrong. |
Only The Relevant Clips
The page keeps the Ant control and the accepted Ant SHAC update. Historical clips are out of the main report.
Ant PPO Control
Ant SHAC Guarded Update
Raw artifacts kept for audit
The full artifact set remains in the repository; only the main report view has been shortened.