Driving policy in MetaDrive
A camera alone copies the easy parts of driving. The car’s own state covers the bends the image attends to poorly, and a second phase of learning is what keeps the car up.
- EfficientNet
- behavioural cloning
- PPO
- Grad-CAM
- ONNX
- 81.5%lower MSE than camera-only
- 7.5×longer episodes after PPO
- 90kexpert steps copied first
Copy an expert, with both sensors
Imitation learns the action the expert took. Most of those frames are straight or a sharp turn. A camera-only net can fit that average and still leave the road on a gentle bend.
- About 90,000 steps from MetaDrive’s IDM driver. Each step is a frame plus 19 state numbers and the action.
- EfficientNet-B0 for the image, a small MLP for the state, concatenated into steering and acceleration.
- Fusion cut mean squared error by 81.5% versus the camera-only baseline. Best validation MSE on the fusion run was 0.00043.
What the picture was looking at
A low imitation loss does not mean the car stays in lane. Grad-CAM was the check on where the vision branch was actually looking when the imitator left the road.
- On those turns the vision branch attended to sky and roadside terrain. That drift accounts for 95% of the out-of-road episodes.
- The state vector is there so speed, heading, and the nearby road can cover what the image attends to poorly.
- The network is 4.89 million parameters. About 882 thousand train while the backbone is frozen, then the backbone is unfrozen.
Practice, not only copying
The second phase has to drive, not only replay the expert. PPO was warm-started from the fusion checkpoint so it did not begin from a random policy.
- Average episode length went from 299 steps to 2,256, about 7.5 times longer.
- Policy exported to ONNX, 19.6 MB.
- A Streamlit dashboard shows the Grad-CAM maps and the cloning curve next to the PPO curve.