feat(BEVFusion): release BEVFusion 2.6.x

[KSeangTan]

Summary

This PR officially releases the first version (2.6.x) of BEVFusion-LiDAR as the baseline for benchmarking performance of BEVFusion-LiDAR across versions. Note that JPNTaxi_base will be released in 2.7.x instead. This PR also fixes several bugs and update the training configs for the internal envinroment.

Major Changes

• Update the filename from 4xB8 to 8xb8 for 8 GPUs training. [1], [2], [3], [4], [5], [6], [7], [8], [9], [10]

• Add T4MetricV2 metric configs to experiments. [[11] (https://github.com/KSeangTan/AWML/blob/d2b3a528d1e8023a550296b1cfb5c2f35dbde0f0/projects/BEVFusion/configs/t4dataset/BEVFusion-L/bevfusion_lidar_voxel_second_secfpn_30e_8xb8_j6gen2_base_120m_t4metric_v2.py), [12], [13]

• Update learning rate from 1e-4 to 1.4141e-4 since batch_size is double in this config. [14], [15]

• Add SyncBatchNorm to fix batchnorm in DDP. [16], [17]

• Add detailed results about BEvFusion-LiDAR 2.6.x. [18], [19]

• Minor Changes

• Remove jpntaxi_base from evaluation. [20], [21]

• Update the spconv version to 2.3.8 with cuda-12.6. [22]

Performance

Summary

1. Consistent Performance Gains:

• Across all datasets where both versions were tested, version 2.6.1 consistently outperforms 2.6.0.
• Average mAP Increase: The mean Average Precision (mAP) typically saw an uplift of 1.4 to 2.0.
• Robustness: The improvements aren't localized to a single category; we see simultaneous gains in vehicle detection (cars/trucks) and vulnerable road users (pedestrians/bicycles).

2. Dataset-Specific Breakdown:

• LargeBus

  • Pedestrian detection saw a significant jump from 0.7669 to 0.8036 (+3.67), suggesting the 2.6.1 is much better at isolating smaller features in environments where large buses might otherwise cause occlusion or scale confusion.

• J6Gen2 & J6Gen2_Base:

  • Trucks: In the J6Gen2_Base set, truck detection improved from 0.7129 to 0.7409. This is a critical gain for heavy-vehicle safety.
  • The consistent delta between v2.6.0 and v2.6.1 here confirms that the model improvements are statistically significant and not just "noise" from a small sample.

• JPNTaxi_Gen2: The "Bus" Outlier

  • Bus: The performance for "bus" is a mere 0.5446, which is drastically lower than the ~0.85 seen in other datasets.
  • This suggests that we might need to tune JPNTaxi_Gen2 data even more for bus.

---

LargeBus

Eval Range: 0.0 - 121.0m

Model version	mAP	mAPH	car (28,895)	truck (2,806)	bus (539)	bicycle (1,288)	pedestrian (9,934)
BEVFusion-LiDAR base/2.6.0	0.7995	0.7514	0.8640	0.7788	0.8608	0.7272	0.7669
BEVFusion-LiDAR j6gen2_base/2.6.1	0.8198	0.7666	0.8690	0.8052	0.8756	0.7455	0.8036

---

J6Gen2

Eval Range: 0.0 - 121.0m

Model version	mAP	mAPH	car (114,558)	truck (12,414)	bus (4,190)	bicycle (1,004)	pedestrian (22,564)
BEVFusion-LiDAR base/2.6.0	0.7712	0.7223	0.8110	0.7129	0.8348	0.7458	0.7515
BEVFusion-LiDAR j6gen2_base/2.6.1	0.7851	0.7375	0.8166	0.7262	0.8481	0.7661	0.7687

---

JPNTaxi_Gen2

Eval Range: 0.0 - 121.0m

Model version	mAP	mAPH	car (94,831)	truck (54,396)	bus (8,596)	bicycle (4,652)	pedestrian (99,485)
BEVFusion-LiDAR base/2.6.0	0.7471	0.7176	0.8667	0.6928	0.5446	0.7710	0.8606

---

J6Gen2_Base

Eval Range: 0.0 - 121.0m

Model version	mAP	mAPH	car (143,453)	truck (15,220)	bus (4,729)	bicycle (2,292)	pedestrian (32,498)
BEVFusion-LiDAR base/2.6.0	0.7712	0.7223	0.8110	0.7129	0.8348	0.7458	0.7515
BEVFusion-LiDAR j6gen2_base/2.6.1	0.7903	0.7413	0.8266	0.7409	0.8510	0.7541	0.7790

---

Base

Eval Range: 0.0 - 121.0m

Model version	mAP	mAPH	car (238,284)	truck (69,616)	bus (13,325)	bicycle (6,944)	pedestrian (131,983)
BEVFusion-LiDAR base/2.6.0	0.7592	0.7227	0.8398	0.6994	0.6621	0.7595	0.8351

---

[Copilot]

Pull request overview

Releases BEVFusion-LiDAR 2.6.x as a new baseline in the repository by updating training/evaluation configs for 8-GPU runs, adding T4MetricV2 experiment variants, and publishing updated benchmark documentation and deployment config tweaks.

Changes:

• Update BEVFusion training configs/schedulers/pipelines for 8×GPU training (including LR scaling + SyncBN config knobs).
• Add T4MetricV2 evaluator experiment configs for BEVFusion-LiDAR variants.
• Update BEVFusion-LiDAR v2 documentation and adjust a few runtime/deploy/setup details.

Reviewed changes

Copilot reviewed 28 out of 28 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
tools/detection3d/train.py	Adds model structure logging before training.
projects/BEVFusion/setup.py	Updates BEVFusion extension install dependency (spconv CUDA variant/version).
projects/BEVFusion/docs/BEVFusion-L/v2/j6gen2.md	Large update to J6Gen2_base v2.6.1 benchmark/results documentation.
projects/BEVFusion/docs/BEVFusion-L/v2/base.md	Large update to base v2.6.0 benchmark/results documentation.
projects/BEVFusion/configs/t4dataset/default/schedulers/default_50e_8xb8_adamw_cosine.py	Adjusts LR for larger batch + sets train_gpu_size=8 + adds sync_bn config.
projects/BEVFusion/configs/t4dataset/default/schedulers/default_30e_8xb8_adamw_linear_cosine.py	Sets train_gpu_size=8 + adds sync_bn config.
projects/BEVFusion/configs/t4dataset/default/schedulers/default_30e_8xb8_adamw_cosine.py	Adjusts LR for larger batch + sets train_gpu_size=8 + adds sync_bn config.
projects/BEVFusion/configs/t4dataset/default/schedulers/default_20e_8xb8_adamw_linear_cosine.py	Sets train_gpu_size=8 + adds sync_bn config.
projects/BEVFusion/configs/t4dataset/default/pipelines/default_lidar_intensity_120m.py	Narrows filtered classes, removes min-points filters, adds city/vehicle_type meta keys.
projects/BEVFusion/configs/t4dataset/default/pipelines/default_lidar_120m.py	Reduces num_workers, narrows filtered classes, removes min-points filters, adds city/vehicle_type meta keys.
projects/BEVFusion/configs/t4dataset/default/pipelines/default_camera_lidar_intensity_120m.py	Removes min-points filters for camera+lidar intensity pipeline.
projects/BEVFusion/configs/t4dataset/BEVFusion-L/bevfusion_lidar_voxel_second_secfpn_50e_8xb8_base_120m_t4metric_v2.py	Adds a T4MetricV2 evaluator variant config for the base 50e run.
projects/BEVFusion/configs/t4dataset/BEVFusion-L/bevfusion_lidar_voxel_second_secfpn_50e_8xb8_base_120m.py	Switches scheduler to 8×GPU variant and updates experiment naming.
projects/BEVFusion/configs/t4dataset/BEVFusion-L/bevfusion_lidar_voxel_second_secfpn_30e_8xb8_jpntaxi_base_120m_t4metric_v2.py	Adds a T4MetricV2 evaluator variant config for JPNTaxi_base 30e run.
projects/BEVFusion/configs/t4dataset/BEVFusion-L/bevfusion_lidar_voxel_second_secfpn_30e_8xb8_jpntaxi_base_120m.py	Switches dataset/scheduler to JPNTaxi_base + 8×GPU naming; adds load_from placeholder line.
projects/BEVFusion/configs/t4dataset/BEVFusion-L/bevfusion_lidar_voxel_second_secfpn_30e_8xb8_j6gen2_base_120m_t4metric_v2.py	Adds a T4MetricV2 evaluator variant config for J6Gen2_base 30e run.
projects/BEVFusion/configs/t4dataset/BEVFusion-L/bevfusion_lidar_voxel_second_secfpn_30e_8xb8_j6gen2_base_120m.py	Switches scheduler to 8×GPU naming; adds load_from placeholder line.
projects/BEVFusion/configs/t4dataset/BEVFusion-L-offline/bevfusion_offline_lidar_voxel_second_secfpn_50e_8xb8_base_120m.py	Updates offline base config to 8×GPU scheduler + experiment naming.
projects/BEVFusion/configs/t4dataset/BEVFusion-L-offline/bevfusion_offline_lidar_voxel_second_secfpn_30e_8xb8_jpntaxi_gen2_base_120m.py	Updates offline JPNTaxi_gen2_base config to 8×GPU scheduler + experiment naming.
projects/BEVFusion/configs/t4dataset/BEVFusion-L-offline/bevfusion_offline_lidar_voxel_second_secfpn_30e_8xb8_j6gen2_base_120m.py	Updates offline J6Gen2_base config to 8×GPU scheduler + experiment naming.
projects/BEVFusion/configs/t4dataset/BEVFusion-CL/bevfusion_camera_lidar_voxel_second_secfpn_20e_8xb8_jpntaxi_base_120m.py	Switches camera+lidar intensity config to JPNTaxi_base and 8×GPU scheduler/naming.
projects/BEVFusion/configs/t4dataset/BEVFusion-CL/bevfusion_camera_lidar_voxel_second_secfpn_20e_8xb8_j6gen2_base_120m.py	Updates camera+lidar intensity config to 8×GPU scheduler/naming.
projects/BEVFusion/configs/t4dataset/BEVFusion-C/bevfusion_camera_swin_fpn_30e_8xb8_j6gen2_base_120m.py	Updates camera-only config to 8×GPU scheduler/naming.
projects/BEVFusion/configs/deploy/bevfusion_main_body_lidar_only_intensity_tensorrt_dynamic.py	Renames exported ONNX filename for lidar+intensity deployment.
projects/BEVFusion/bevfusion/bevfusion_head.py	Passes filter=False to bbox coder decode in target generation.
projects/BEVFusion/bevfusion/init.py	Exposes TransFusionBBoxCoder from the BEVFusion package.
autoware_ml/configs/detection3d/dataset/t4dataset/jpntaxi_base.py	Alters jpntaxi_base test groups (now points to gen2 test info).
autoware_ml/configs/detection3d/dataset/t4dataset/base.py	Removes jpntaxi_base from base dataset test groups.

Comments suppressed due to low confidence (2)

projects/BEVFusion/configs/t4dataset/BEVFusion-L/bevfusion_lidar_voxel_second_secfpn_30e_8xb8_j6gen2_base_120m.py:147

• load_from "<best_checkpoint>" is invalid Python/MMEngine config syntax and will raise a syntax error when the config is loaded. If you want to document a placeholder, make it a comment; if you want to actually set it, use load_from = "..." (or remove it and rely on --resume/--load-from).
  projects/BEVFusion/configs/t4dataset/BEVFusion-L/bevfusion_lidar_voxel_second_secfpn_30e_8xb8_jpntaxi_base_120m.py:147
• load_from "<best_checkpoint>" is invalid Python/MMEngine config syntax and will raise a syntax error when the config is loaded. If you want to document a placeholder, make it a comment; if you want to actually set it, use load_from = "..." (or remove it and rely on --resume/--load-from).

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[jacoblambert]

Great improvement! LGTM