Code release plan
Vilonge opened this issue · 10 comments
Great work! Is there an open source plan for the code? About what time?
@Vilonge
Hello, thank you for your attention! I expect to release the code of data generatition samples (using GPT-4) before 6.15. I will release the training code before 7.1.
I integrated the code of llava into mmdet3d 1.0 during the development, but the mmdet3d 1.0 version does not support deepspeed, resulting in higher training memory.
So I plan to refactor on mmdet3d 1.1 to support deep speed.
Thanks for your reply. Looking forward to open source.
We release the caption & conversation data. Hope it will help you.
Thanks. Indeed.
Hi, where is the caption & conversation data? I could only find omnidrive_data.zip, where there is only annotation json files, but I do not know how the .json is connected to the actual image.
@Grace0413 Hi, please refer to this issue:
#2 (comment)
The name of json files are the sample_tokens in nuscenes dataset.
Thanks for your reply! I have read the code you provided:
sample_idx=info['token'] # ('sample_token' for nusc)
with open(vqa_path+results['sample_idx']+".json", 'r') as f:
data_qa = json.load(f)
Maybe this is a stupid question: what is info and what is results here? I'm sorry but I have no idea how to obtain info['token'] and results['sample_idx'] without knowing what info and results are.
@Grace0413
Do you have any experience with the nuscenes api? For each timestamp, you will have sample['token'], which you can use to index the corresponding image.
info['token'] and results['sample_idx'] = sample['token'] here.
You can refer to the StreamPETR dataset section from my previous work.
Hi, thanks for your great work! Could you provide the evaluation codes in advance, especially how to calculate the metrics such as Collision && Intersection? Maybe some essential files (e.g., gt_fut_trajs, gt_occ, results of ominidrive) are also needed.
Thanks, I will try it out!