Important
š Stay up to date at opendrivelab.com!
- A SOTA Decoder for End-to-End Autonomous Driving under BEV
- arXiv Paper (CVPR 2023)
- Installation
- Closed-Loop Evaluation in Carla
- Prepare Dataset
- Train Your Own Model
- Calibrations for Different Camera Settings (Optional)
Install the environment as in Installation, download our checkpoint (GoogleDrive or BaiduYun(ęåē m5di).) (189K frames training set), put it into open_loop_training/ckpt, and run:
## In the ThinkTwice/ directory CUDA_VISIBLE_DEVICES=0 nohup bash ./leaderboard/scripts/evaluation_town05long.sh 22023 22033 thinktwice_agent False True open_loop_training/ckpt/thinktwice.pth+open_loop_training/configs/thinktwice.py all_towns_traffic_scenarios_no256 thinktwice_town05long 2>&1 > thinktwice_town05long.log &
Check closed_loop_eval_log/eval_log to see how our model drives in Carla! š
(In case you have a screen to see the interface of Carla simulator, you could remove DISPLAY= in leaderboard/leaderboard/leaderboard_evaluator.py and then you could directly watch with Carla.)
We give the structure of our code. Note that we only introduce those folders/files are commonly used and modified.
ThinkTwice/
āāā agents # From Carla official
āāā camera_calibration # When you want to use cameras with different FOV
āāā closed_loop_eval_log # Save eval logs
āāā collect_data_json # Save data collection logs
āāā dataset # Data and metadata for training
āāā leaderboard # Code for Closed-Loop Evaluation
ā āāā data # Save routes and scenarios
ā āāā scripts # Run with Carla
ā āāā team_code # Your
| | āāā roach_ap_agent_data_collection.py # Data collection
ā | āāā thinktwice_agent.py # Interface for closed-loop evaluation of our model
ā āāā leaderboard # From Carla official
| | āāā leaderboard_evaluator.py # Entrance of closed-loop evaluation
āāā roach # Roach for data collection
āāā scenario_runner # From Carla official
āāā open_loop_training # Training and Neural Network
| āāā ckpt # Checkpoints
| āāā work_dirs # Training Log
| āāā code # Preprocessing, DataLoader, Model
| ā āāā apis # Training pipeline for mmdet3D
| ā āāā core # The hooks for mmdet3D
| ā āāā datasets # Preprocessing and DataLoader
| | | āāā pipelines # Functions of Preprocessing and DataLoader
| ā | āāā samplers # For DDP
| ā | āāā carla_dataset.py # Framework of Preprocessing and DataLoading
| ā āāā model_code # Neural Network
| | | āāā backbones # Module of Encoder
| | | āāā dense_heads # Module of Decoder and Loss Functions
| ā āāā encoder_decoder_framework.py # Entrance of Neural Network
| āāā train.py # Entrance of Training
All assets and code are under the Apache 2.0 license unless specified otherwise.
If this work is helpful for your research, please consider citing the following BibTeX entry.
@inproceedings{jia2023thinktwice,
title={Think Twice before Driving: Towards Scalable Decoders for End-to-End Autonomous Driving},
author={Jia, Xiaosong and Wu, Penghao and Chen, Li and Xie, Jiangwei and He, Conghui and Yan, Junchi and Li, Hongyang},
booktitle={CVPR},
year={2023}
}
Many thanks to the open-source community!
- End-to-end Autonomous Driving Survey (:rocket:Ours!)
- TCP (:rocket:Ours!)
- BEVFormer (:rocket:Ours!)
- UniAD (:rocket:Ours!)
- ST-P3 (:rocket:Ours!)
- Carla
- Roach
- Transfuser
- CARLA_GARGE
- LAV
- IBISCape