ApolloSimFuzz is a simulation-based fuzzing framework for testing Baidu Apollo autonomous driving systems. It integrates Apollo with TrafficSandbox, a lightweight traffic simulator, to enable systematic, closed-loop testing of Apollo's decision-making under diverse traffic scenarios -- without relying on high-fidelity simulators.
The framework uses perfect perception and control commands to focus testing on the decision-making pipeline (planning, routing, prediction), making it efficient for large-scale fuzzing campaigns.
+---------------------------+ +---------------------------+
| ApolloSimFuzz | | TrafficSandbox |
| | ZMQ | (Docker Container) |
| Fuzzer Engine | <-----> | Physics Simulation |
| ├── Scenario Sampler | RPC | HD Map Queries |
| ├── Oracle (Violations) | | Actor Management |
| └── Feedback (Metrics) | | Visualization (:18888) |
| | +---------------------------+
| Scenario Runner |
| ├── Agent Processes | +---------------------------+
| ├── Criteria (Runtime) | | Apollo 7.0 |
| └── Observation Recorder | WS | (Docker Container) |
| | <-----> | Planning / Routing |
| Apollo Bridge | ROS | Prediction / Control |
| ├── Publishers (Percep.) | | Dreamview (:8888) |
| └── Subscribers (Ctrl) | +---------------------------+
+---------------------------+
- Random fuzzing with configurable scenario spaces (vehicles, pedestrians, traffic lights)
- Pluggable oracle system -- collision, speeding, wrong-way, red-light, lane violations, solid-line crossing
- Pluggable feedback metrics -- extensible for custom fitness functions
- DEAP integration -- ready for evolutionary / search-based fuzzing strategies
- Checkpoint & resume -- long-running campaigns survive restarts
- Multi-GPU support -- parallel Apollo containers with automatic GPU assignment
- Real-time visualization -- browser-based map + actor rendering via TrafficSandbox
- uv -- Python package manager
- Docker with NVIDIA Container Toolkit
- GPU (required for Apollo)
git clone --recurse-submodules git@github.com:MingfeiCheng/ApolloSimFuzz.git
cd ApolloSimFuzz
bash install.sh
bash build_apollo.shSee Installation Guide for details.
# Stop the build container first
docker stop apollo_dev_${USER}
# Run random fuzzing
bash scripts/random.shApolloSimFuzz/
├── start_fuzzer.py # Entry point
├── config.yaml # Global configuration (Hydra)
├── scripts/random.sh # Random fuzzing launch script
│
├── fuzzer/ # Fuzzing engine
│ ├── runner_base.py # Base fuzzer (checkpoint, evaluation, containers)
│ ├── runner_random.py # Random fuzzer implementation
│ ├── scenario_space.py # Scenario parameter space (ODD)
│ ├── configs/s1.yaml # Scenario configuration
│ ├── misc_common/ # Scenario sampling & mutation
│ ├── feedback/ # Feedback metric extraction
│ └── oracle/ # Offline violation detection
│
├── scenario_runner/ # Scenario execution
│ ├── manager.py # Scenario lifecycle management
│ ├── sandbox_operator.py # TrafficSandbox RPC client
│ ├── ctn_manager.py # Docker container management
│ └── config.py # Runtime configuration
│
├── scenario_elements/ # Agents and runtime criteria
│ ├── agents/ # Actor controllers
│ │ ├── waypoint_vehicle/ # NPC vehicle (PID + waypoint following)
│ │ ├── waypoint_walker/ # Pedestrian navigation
│ │ ├── static_obstacle/ # Static objects
│ │ └── traffic_light/ # Traffic light state machines
│ └── criteria/ # Runtime oracles
│ ├── runtime_collision.py # Collision detection
│ ├── runtime_speeding.py # Speed limit monitoring
│ ├── runtime_red_light.py # Traffic light compliance
│ ├── runtime_wrong_way.py # Wrong direction detection
│ ├── runtime_wrong_lane.py# Lane type violations
│ ├── runtime_solid_line.py# Solid line crossing
│ └── runtime_stuck.py # Stuck vehicle detection
│
├── scenario_corpus/ # Scenario format & execution
│ └── openscenario/ # OpenScenario-based scenarios
│ ├── openscenario_executor.py
│ └── sm_*.py # State machines (ego, NPC, traffic light)
│
├── apollo_bridge/ # Apollo communication layer
│ ├── common/
│ │ ├── container.py # Apollo Docker management
│ │ ├── publisher/ # Send perception to Apollo
│ │ └── subscriber/ # Receive control from Apollo
│ ├── instances/ # Apollo agent implementation
│ └── patches/ # Apollo build patches
│
├── TrafficSandbox/ # Traffic simulator (submodule)
├── registry/ # Plugin registry system
├── tools/ # Utility tools
└── documents/ # Documentation
| Parameter | Default | Description |
|---|---|---|
run_tag |
debug |
Run identifier |
debug |
true |
Enable debug logging |
resume |
true |
Resume from checkpoint |
output_root |
results |
Output directory |
sandbox_image |
drivora/sandbox:latest |
TrafficSandbox Docker image |
sandbox_fps |
20.0 |
Simulation FPS |
tester.type |
random |
Fuzzer type |
tester.time_budget |
4 |
Time budget in hours |
tester.config_path |
fuzzer/configs/s1.yaml |
Scenario space config |
Defines the operational design domain (ODD):
map_region_space-- Geographic region, map name, forbidden zonesego_space-- Ego vehicle route (start/end points)npc_vehicle_space-- NPC count, speed, route length, trigger timetraffic_light_space-- Light patterns (rule-based, force green), durationsoracle-- Which violations to detect (collision, speeding, etc.)
from registry import ENGINE_REGISTRY
from fuzzer.runner_base import Fuzzer
@ENGINE_REGISTRY.register("fuzzer.my_fuzzer")
class MyFuzzer(Fuzzer):
def _run(self, start_time):
while not self.termination_check(start_time):
seed = self.random_sample()
# ... your fuzzing logicfrom scenario_elements.criteria.base import CriteriaBase
class MyCustomCriteria(CriteriaBase):
def tick(self, snapshot):
# Check for violations in the current snapshot
...feedback_wrapper.register_metric("my_metric", my_metric_function)oracle_wrapper.register_check("my_check", my_check_function)results/{run_tag}/
├── results/
│ └── s_{step}/ # Per-scenario results
│ ├── scenario.json # Input scenario config
│ ├── result.json # Criteria results
│ └── observations/ # Trajectory data
├── fuzz_cache/ # Fuzzer checkpoints
├── overview.json # Summary of all scenarios
├── config.yaml # Config snapshot
└── run.log # Execution log
- Installation Guide -- Setup and build instructions
- Architecture -- System design and data flow
- API Reference -- Fuzzer and scenario APIs
If you use ApolloSimFuzz in your research, please cite:
@article{cheng2026drivora,
title = {Drivora: A Unified and Extensible Infrastructure for Search-based Autonomous Driving Testing},
author = {Cheng, Mingfei and Briand, Lionel and Zhou, Yuan},
journal = {arXiv preprint arXiv:2601.05685},
year = {2026}
}
@inproceedings{cheng2025decictor,
title = {Decictor: Towards Evaluating the Robustness of Decision-Making in Autonomous Driving Systems},
author = {Cheng, Mingfei and Xie, Xiaofei and Zhou, Yuan and Wang, Junjie and Meng, Guozhu and Yang, Kairui},
booktitle = {ICSE},
year = {2025}
}
@inproceedings{cheng2023behavexplor,
title = {Behavexplor: Behavior Diversity Guided Testing for Autonomous Driving Systems},
author = {Cheng, Mingfei and Zhou, Yuan and Xie, Xiaofei},
booktitle = {ISSTA},
year = {2023}
}More references: reference.bib
- Maintainer: Mingfei Cheng
- Email: snowbirds.mf@gmail.com
- Discord: Join
MIT License. See LICENSE for details.