How to train Panoptic Segmentation on a custom dataset ?

Question

How to train Panoptic Segmentation on a custom dataset ?

david95j2 opened this issue 2 years ago · 3 comments

KeyError: 'thing_dataset_id_to_contiguous_id'

Hi guys, i am trying to register and train a custom panoptic dataset. I have converted the annotations to coco format.
I have only one concretcrack class and then the following is the format of the panoptic annotations

{
 "info": {...},
 "images": [
    {
      "id": 263,
      "width": 4032,
      "height": 3024,
      "file_name": "000263.jpg"
    }, ...
    ... 
  ],
  "annotations": [
    {
      "image_id": 263,
      "file_name": "000263_GT.png",
      "segments_info": [
        {
          "id": 39372,
          "category_id": 1,
          "bbox": [
            0.0,
            803.0,
            4032.0,
            2221.0
          ],
          "iscrowd": 0,
          "area" : 128846
        }
      ]
    }, ...
    ...
    "categories": [
       {
          "supercateory": "korad",
          "isthing" : 1,
          "id" : 1,
          "name" : "ConcreteCrack",
          "color": [
               0,
              153,
              204
          ]
       }
     ]
}

Instructions To Reproduce the Issue:

python3 train_net.py --config-file configs/coco/panoptic-segmentation/swin/maskformer2_swin_large_IN21k_384_bs16_100ep_custom.yaml --num-gpus 1 SOLVER.IMS_PER_BATCH 2

I modified train_net.py and The rest of code is same as the original code

if __name__ == "__main__":

    if 'ConcreteCrack_train' in DatasetCatalog.list():
        DatasetCatalog.remove('ConcreteCrack_train')
    if 'ConcreteCrack_val' in DatasetCatalog.list():
        DatasetCatalog.remove('ConcreteCrack_val')

    register_coco_panoptic("ConcreteCrack_train",{},"./datasets/crack/train2022/","./datasets/crack/panoptic_train2022/",
    "./datasets/crack/annotations/panoptic_train2022.json","./datasets/crack/annotations/instances_train2022.json")
    register_coco_panoptic("ConcreteCrack_val",{},"./datasets/crack/val2022/","./datasets/crack/panoptic_val2022/",
    "./datasets/crack/annotations/panoptic_val2022.json","./datasets/crack/annotations/instances_val2022.json")

    # load_coco_json('./datasets/crack/annotations/instances_train2022.json',
    # './datasets/crack/train2022/',
    # 'ConcreteCrack_train')

    # load_coco_json('./datasets/crack/annotations/instances_val2022.json',
    # './datasets/crack/val2022/',
    # 'ConcreteCrack_val')    

    args = default_argument_parser().parse_args()
    print("Command Line Args:", args)
    launch(
        main,
        args.num_gpus,
        num_machines=args.num_machines,
        machine_rank=args.machine_rank,
        dist_url=args.dist_url,
        args=(args,),
    )

However I got the following error:

Traceback (most recent call last):
  File "train_net.py", line 350, in <module>
    launch(
  File "/home/joo-tp/vsc/Mask2Former/seg/lib/python3.8/site-packages/detectron2/engine/launch.py", line 82, in launch
    main_func(*args)
  File "train_net.py", line 321, in main
    trainer = Trainer(cfg)
  File "/home/joo-tp/vsc/Mask2Former/seg/lib/python3.8/site-packages/detectron2/engine/defaults.py", line 378, in __init__
    data_loader = self.build_train_loader(cfg)
  File "train_net.py", line 178, in build_train_loader
    return build_detection_train_loader(cfg, mapper=mapper)
  File "/home/joo-tp/vsc/Mask2Former/seg/lib/python3.8/site-packages/detectron2/config/config.py", line 207, in wrapped
    explicit_args = _get_args_from_config(from_config, *args, **kwargs)
  File "/home/joo-tp/vsc/Mask2Former/seg/lib/python3.8/site-packages/detectron2/config/config.py", line 245, in _get_args_from_config
    ret = from_config_func(*args, **kwargs)
  File "/home/joo-tp/vsc/Mask2Former/seg/lib/python3.8/site-packages/detectron2/data/build.py", line 344, in _train_loader_from_config
    dataset = get_detection_dataset_dicts(
  File "/home/joo-tp/vsc/Mask2Former/seg/lib/python3.8/site-packages/detectron2/data/build.py", line 241, in get_detection_dataset_dicts
    dataset_dicts = [DatasetCatalog.get(dataset_name) for dataset_name in names]
  File "/home/joo-tp/vsc/Mask2Former/seg/lib/python3.8/site-packages/detectron2/data/build.py", line 241, in <listcomp>
    dataset_dicts = [DatasetCatalog.get(dataset_name) for dataset_name in names]
  File "/home/joo-tp/vsc/Mask2Former/seg/lib/python3.8/site-packages/detectron2/data/catalog.py", line 58, in get
    return f()
  File "/home/joo-tp/vsc/Mask2Former/seg/lib/python3.8/site-packages/detectron2/data/datasets/coco_panoptic.py", line 88, in <lambda>
    lambda: load_coco_panoptic_json(panoptic_json, image_root, panoptic_root, metadata),
  File "/home/joo-tp/vsc/Mask2Former/seg/lib/python3.8/site-packages/detectron2/data/datasets/coco_panoptic.py", line 51, in load_coco_panoptic_json
    segments_info = [_convert_category_id(x, meta) for x in ann["segments_info"]]
  File "/home/joo-tp/vsc/Mask2Former/seg/lib/python3.8/site-packages/detectron2/data/datasets/coco_panoptic.py", line 51, in <listcomp>
    segments_info = [_convert_category_id(x, meta) for x in ann["segments_info"]]
  File "/home/joo-tp/vsc/Mask2Former/seg/lib/python3.8/site-packages/detectron2/data/datasets/coco_panoptic.py", line 27, in _convert_category_id
    if segment_info["category_id"] in meta["thing_dataset_id_to_contiguous_id"]:
KeyError: 'thing_dataset_id_to_contiguous_id'

Environment:

----------------------  --------------------------------------------------------------------------------------
sys.platform            linux
Python                  3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0]
numpy                   1.23.3
detectron2              0.6 @/home/joo-tp/vsc/Mask2Former/seg/lib/python3.8/site-packages/detectron2
Compiler                GCC 9.4
CUDA compiler           CUDA 11.3
detectron2 arch flags   8.6
DETECTRON2_ENV_MODULE   <not set>
PyTorch                 1.12.0+cu113 @/home/joo-tp/vsc/Mask2Former/seg/lib/python3.8/site-packages/torch
PyTorch debug build     False
GPU available           Yes
GPU 0                   NVIDIA GeForce RTX 3090 (arch=8.6)
Driver version          470.141.03
CUDA_HOME               /usr/local/cuda-11.3
Pillow                  9.2.0
torchvision             0.13.0+cu113 @/home/joo-tp/vsc/Mask2Former/seg/lib/python3.8/site-packages/torchvision
torchvision arch flags  3.5, 5.0, 6.0, 7.0, 7.5, 8.0, 8.6
fvcore                  0.1.5.post20220512
iopath                  0.1.9
cv2                     4.6.0
----------------------  --------------------------------------------------------------------------------------
PyTorch built with:
  - GCC 9.3
  - C++ Version: 201402
  - Intel(R) Math Kernel Library Version 2020.0.0 Product Build 20191122 for Intel(R) 64 architecture applications
  - Intel(R) MKL-DNN v2.6.0 (Git Hash 52b5f107dd9cf10910aaa19cb47f3abf9b349815)
  - OpenMP 201511 (a.k.a. OpenMP 4.5)
  - LAPACK is enabled (usually provided by MKL)
  - NNPACK is enabled
  - CPU capability usage: AVX2
  - CUDA Runtime 11.3
  - NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86
  - CuDNN 8.2.1
    - Built with CuDNN 8.3.2
  - Magma 2.5.2
  - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.3, CUDNN_VERSION=8.3.2, CXX_COMPILER=/opt/rh/devtoolset-9/root/usr/bin/c++, CXX_FLAGS= -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -DEDGE_PROFILER_USE_KINETO -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Werror=cast-function-type -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.12.0, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=OFF, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF,

How can i fix the problem? thx :)

Answer 1 · 2022-09-28T05:32:27.000Z

and my config file
configs/coco/panoptic-segmentation/swin/maskformer2_swin_large_IN21k_384_bs16_100ep_custom.yaml

_BASE_: ../maskformer2_R50_bs16_50ep.yaml

DATASETS:
  PRECOMPUTED_PROPOSAL_TOPK_TEST: 1000
  PRECOMPUTED_PROPOSAL_TOPK_TRAIN: 2000
  PROPOSAL_FILES_TEST: []
  PROPOSAL_FILES_TRAIN: []
  TEST:
  - ConcreteCrack_val
  TRAIN:
  - ConcreteCrack_train
MODEL:
  BACKBONE:
    NAME: "D2SwinTransformer"
  SWIN:
    EMBED_DIM: 192
    DEPTHS: [2, 2, 18, 2]
    NUM_HEADS: [6, 12, 24, 48]
    WINDOW_SIZE: 12
    APE: False
    DROP_PATH_RATE: 0.3
    PATCH_NORM: True
    PRETRAIN_IMG_SIZE: 384
  WEIGHTS: "swin_large_patch4_window12_384_22k.pkl"
  PIXEL_MEAN: [123.675, 116.280, 103.530]
  PIXEL_STD: [58.395, 57.120, 57.375]
  MASK_FORMER:
    NUM_OBJECT_QUERIES: 200
  RETINANET:
    NUM_CLASSES: 1
  ROI_HEADS:
    NUM_CLASSES: 1
SOLVER:
  STEPS: (655556, 710184)
  MAX_ITER: 737500

Answer 2 · 2022-10-11T08:33:13.000Z

Hi, you need to assign "metadata" when registering your datasets.
Check "detectron2/data/datasets/builtin_meta.py", and there is a function "_get_builtin_metadata".
Key "thing_dataset_id_to_contiguous_id" could be created in this function.

Answer 3 · 2022-10-14T10:06:54.000Z

Can you tell me what dataset you used? I tried to use the crack dataset, but the crack dataset does not have a json file and only a mask file, so it does not work well for using it in the model.