Custom dataset training: Change paths

Question

Custom dataset training: Change paths

kirillkoncha opened this issue 2 years ago · 13 comments

Hi!

I would like to train SparseInst model on my own dataset, which is very alike COCO dataset. However, I wonder where I can change paths to dataset, so the script would find it. Unfortunately, "Training SparseInst with Custom Datasets" section is empty. I looked through tools/train_net.py and config files, however, I did not find, where I can change paths.

Should I change paths to folders with my dataset here?

DATASETS:
  TRAIN: ("coco_2017_train",)
  TEST:  ("coco_2017_val",)

Answer 1 · 2022-09-13T09:02:13.000Z

@Vincent630, maybe you could help?

Answer 2 · 2022-09-13T15:07:59.000Z

Hi @kirillkoncha, thanks for your interest in SparseInst!
As for using the custom dataset, we suggest:
(1) convert the dataset format into COCO format,
(2) register your own dataset,

from detectron2.data.datasets import register_coco_instances
register_coco_instances("my_dataset", {}, "json_annotation.json", "path/to/image/dir")

and then you can use your own dataset by setting the DATASETS.TRAIN or DATASETS.TEST.
You can find more details in the official guide for custom dataset.
Sorry for the late reply and if you have any other problems, please feel free to ask.

Answer 3 · 2022-09-16T12:35:02.000Z

So, I should add registration in train_net.py, change names in the config and then just run train_net.py? Or some other changes are need to be done as build_evaluator description suggests to built own evaluator for custom data...

Answer 4 · 2022-09-16T13:18:55.000Z

Hi @kirillkoncha, you only need to change the names in the config after you register the dataset. If you build the dataset in the coco format, there is no need to build extra evaluators.

Answer 5 · 2022-09-16T14:03:48.000Z

Thanks a lot for your answer! I also have one last question: when I training model on my own dataset, which have different new classes, should I use somehow already pretrained weights or train the model from scratch using the base config?

Answer 6 · 2022-09-16T15:29:49.000Z

@kirillkoncha, it's my pleasure.
It's ok to load the COCO pre-trained weights to initialize the model with a simple modification NUM_CLASSES. Using a pre-trained model can facilitate convergence and improve the overall performance.
If you have any questions, please feel free to ask~

Answer 7 · 2022-09-16T19:57:45.000Z

And how I can change NUM_CLASSES? I can't find this in base config, only in the training script. Or it will be done automatically based on the number of classes in dataset?

Answer 8 · 2022-09-17T04:40:39.000Z

Hi @kirillkoncha, the code will not automatically change the number of classes for models. You need to change the config:

MODEL.SPARSE_INST.DECODER.NUM_CLASSES

to specify the number of classes.

Answer 9 · 2022-09-17T14:10:20.000Z

Okay, thanks!!! But why Base-SparseInst.yaml does not have this parameter originally?

Answer 10 · 2022-09-18T08:20:15.000Z

Hi @kirillkoncha, it's ok to append the parameter to your config. The parameter mentioned above has been included in the config.py and has been registered.

Answer 11 · 2022-09-22T22:08:52.000Z

Thank you a lot! Could I ask one more question: is there any way to specifies the numbers of eight gpu's I want to use? (For example, I want to train model from 7 to 15 gpu's)

Answer 12 · 2022-09-23T06:37:14.000Z

Hi @kirillkoncha, it's simple, just like CUDA_VISIBLE_DEVICES=7,8,9,10,11,12,13,14,15 python train.py .....

Answer 13 · 2022-10-10T03:42:18.000Z

This issue will be closed, you can open a new issue if you have other problems or reopen it. Thanks for your interest in SparseInst 😊 . If you find it useful in your work or research, could you give us a star 🌟 or help us recommend SparseInst to your friends?