PQ evaluation result on ADE20K
xiexinch opened this issue · 1 comments
xiexinch commented
Hi, Thanks for your great work!
I try to run an evaluation on the ADE20K dataset with the BestSeg Tiny checkpoint. I found that the mIoU and AP consist of your results, but PQ is not aligned. Could you help me identify the problem?
INFO:datasets.evaluation.segmentation_evaluation:OrderedDict([('sem_seg', {'mIoU': 25.138934771544356, 'fwIoU': 56.54702259148684, 'IoU-wall': 66.66384375012157, 'IoU-building': 61.80618454872132, 'IoU-sky': 93.03627440691773, 'IoU-floor': 54.56389385541954, 'IoU-tree': 64.13564186259205, 'IoU-ceiling': 77.29479183734789, 'IoU-road, route': 79.45689179433187, 'IoU-bed': 72.96180819185307, 'IoU-window ': 49.276455812945194, 'IoU-grass': 62.04998439647351, 'IoU-cabinet': 46.98513878773616, 'IoU-sidewalk, pavement': 55.164837905552744, 'IoU-person': 81.96654332757166, 'IoU-earth, ground': 0.0, 'IoU-door': 36.46062723798975, 'IoU-table': 28.028372398862402, 'IoU-mountain, mount': 40.54859414005354, 'IoU-plant': 17.151210866614175, 'IoU-curtain': 68.51877388456774, 'IoU-chair': 40.64549598364446, 'IoU-car': 75.92264676089647, 'IoU-water': 18.77329669322049, 'IoU-painting, picture': 8.316185301015338, 'IoU-sofa': 57.89617118568877, 'IoU-shelf': 29.76269778595836, 'IoU-house': 18.14527867328128, 'IoU-sea': 43.37431317680453, 'IoU-mirror': 59.25143146822302, 'IoU-rug': 21.110705561460193, 'IoU-field': 20.738062589728905, 'IoU-armchair': 27.52600064000787, 'IoU-seat': 20.41313024837476, 'IoU-fence': 27.28278320109175, 'IoU-desk': 6.503097748798531, 'IoU-rock, stone': 25.970516263564562, 'IoU-wardrobe, closet, press': 0.1846807453809998, 'IoU-lamp': 0.7250267466787069, 'IoU-tub': 52.04585362344179, 'IoU-rail': 8.451170913691197, 'IoU-cushion': 0.8200596407011419, 'IoU-base, pedestal, stand': 0.0, 'IoU-box': 13.988375531656741, 'IoU-column, pillar': 0.0, 'IoU-signboard, sign': 15.923987465222256, 'IoU-chest of drawers, chest, bureau, dresser': 4.93644486794556, 'IoU-counter': 3.323037256225233, 'IoU-sand': 39.24199391657301, 'IoU-sink': 53.1492515051619, 'IoU-skyscraper': 17.558166889843616, 'IoU-fireplace': 25.26774697327011, 'IoU-refrigerator, icebox': 55.05294079040842, 'IoU-grandstand, covered stand': 31.396726806009056, 'IoU-path': 6.703153377491354, 'IoU-stairs': 25.672768197014335, 'IoU-runway': 45.43841992927305, 'IoU-case, display case, showcase, vitrine': 0.0, 'IoU-pool table, billiard table, snooker table': 80.27526367967762, 'IoU-pillow': 0.13484984987869275, 'IoU-screen door, screen': 0.0, 'IoU-stairway, staircase': 0.3789820949091252, 'IoU-river': 11.185766890263418, 'IoU-bridge, span': 26.68986668917918, 'IoU-bookcase': 1.635284656019348, 'IoU-blind, screen': 27.46825264385997, 'IoU-coffee table': 1.7962906383315793, 'IoU-toilet, can, commode, crapper, pot, potty, stool, throne': 82.81076010282526, 'IoU-flower': 26.89008628784697, 'IoU-book': 35.18320239666976, 'IoU-hill': 0.0, 'IoU-bench': 31.55542601768192, 'IoU-countertop': 15.173809141969999, 'IoU-stove': 12.98205015198985, 'IoU-palm, palm tree': 3.563747397914385, 'IoU-kitchen island': 18.49397313213103, 'IoU-computer': 21.26985664127961, 'IoU-swivel chair': 20.40526039736884, 'IoU-boat': 71.28289053362035, 'IoU-bar': 0.04609290003238961, 'IoU-arcade machine': 13.801956630762666, 'IoU-hovel, hut, hutch, shack, shanty': 6.769738693649287, 'IoU-bus': 71.51411629003796, 'IoU-towel': 52.67571260631483, 'IoU-light': 10.874940000210218, 'IoU-truck': 18.308968818456865, 'IoU-tower': 3.8846859382760988, 'IoU-chandelier': 1.039063125144151, 'IoU-awning, sunshade, sunblind': 8.464720109362403, 'IoU-street lamp': 0.0, 'IoU-booth': 0.0, 'IoU-tv': 21.251801174226408, 'IoU-plane': 49.31738594020662, 'IoU-dirt track': 0.4271491891808845, 'IoU-clothes': 19.80153487917838, 'IoU-pole': 19.48645788874662, 'IoU-land, ground, soil': 0.2774159463904913, 'IoU-bannister, banister, balustrade, balusters, handrail': 0.6005131816565449, 'IoU-escalator, moving staircase, moving stairway': 0.0, 'IoU-ottoman, pouf, pouffe, puff, hassock': 21.542985891204715, 'IoU-bottle': 26.710336877651663, 'IoU-buffet, counter, sideboard': 0.09555932713171633, 'IoU-poster, posting, placard, notice, bill, card': 20.477183085814193, 'IoU-stage': 3.6712931019231108, 'IoU-van': 29.00911106040356, 'IoU-ship': 7.458659731394134, 'IoU-fountain': 16.93792762129684, 'IoU-conveyer belt, conveyor belt, conveyer, conveyor, transporter': 8.341539715134479, 'IoU-canopy': 0.0, 'IoU-washer, automatic washer, washing machine': 48.10438207779419, 'IoU-plaything, toy': 4.378879596393649, 'IoU-pool': 21.915560445632035, 'IoU-stool': 19.11406844106464, 'IoU-barrel, cask': 4.771511771759197, 'IoU-basket, handbasket': 15.027955178141156, 'IoU-falls': 34.95419449012886, 'IoU-tent': 71.33059398347683, 'IoU-bag': 15.193300948116296, 'IoU-minibike, motorbike': 62.37326771941579, 'IoU-cradle': 8.939156718393596, 'IoU-oven': 13.716478380604702, 'IoU-ball': 20.336170440554575, 'IoU-food, solid food': 43.168141456019306, 'IoU-step, stair': 0.09271339970316758, 'IoU-tank, storage tank': 17.602110706421065, 'IoU-trade name': 0.0, 'IoU-microwave': 80.83249143309624, 'IoU-pot': 8.95837787354705, 'IoU-animal': 65.3558342925042, 'IoU-bicycle': 47.88619345844731, 'IoU-lake': 0.0, 'IoU-dishwasher': 5.388336798677167, 'IoU-screen': 0.0, 'IoU-blanket, cover': 0.0, 'IoU-sculpture': 16.587339263108802, 'IoU-hood, exhaust hood': 0.0, 'IoU-sconce': 0.4015924011733482, 'IoU-vase': 23.626310871145257, 'IoU-traffic light': 24.812425480213438, 'IoU-tray': 6.6554194644082285, 'IoU-trash can': 19.14883613779226, 'IoU-fan': 6.708352822431747, 'IoU-pier': 27.35937044593413, 'IoU-crt screen': 5.049604189252767, 'IoU-plate': 21.94244018437383, 'IoU-monitor': 1.7734684944451007, 'IoU-bulletin board': 14.728702703084497, 'IoU-shower': 0.4926401990256038, 'IoU-radiator': 28.439220471239995, 'IoU-glass, drinking glass': 22.272511653951643, 'IoU-clock': 21.36579342404726, 'IoU-flag': 40.49251138551105, 'mACC': 40.569499444810965, 'pACC': 68.73159223690485, 'ACC-wall': 75.00736634097991, 'ACC-building': 68.57669111441295, 'ACC-sky': 95.63273189197791, 'ACC-floor': 59.757976224767575, 'ACC-tree': 92.34077795588281, 'ACC-ceiling': 89.35127228636448, 'ACC-road, route': 87.81631900386382, 'ACC-bed': 96.07369829673503, 'ACC-window ': 70.58632284693935, 'ACC-grass': 82.43561432559335, 'ACC-cabinet': 73.53039116272248, 'ACC-sidewalk, pavement': 73.39709750648018, 'ACC-person': 93.47207883771334, 'ACC-earth, ground': 0.0, 'ACC-door': 59.03325176174202, 'ACC-table': 49.13000241166902, 'ACC-mountain, mount': 55.726524253558, 'ACC-plant': 19.260336571944435, 'ACC-curtain': 88.2108816110385, 'ACC-chair': 52.78305442082364, 'ACC-car': 84.76829270902667, 'ACC-water': 23.58851803147177, 'ACC-painting, picture': 8.464140875846445, 'ACC-sofa': 83.19727622430966, 'ACC-shelf': 62.45517008717844, 'ACC-house': 82.99261107811003, 'ACC-sea': 65.22998750814483, 'ACC-mirror': 75.9656331517485, 'ACC-rug': 87.16420063917124, 'ACC-field': 44.32838478466386, 'ACC-armchair': 61.51674279239121, 'ACC-seat': 25.294529795433164, 'ACC-fence': 68.33004233403108, 'ACC-desk': 9.217060578819886, 'ACC-rock, stone': 80.14134453358818, 'ACC-wardrobe, closet, press': 0.18750462726011477, 'ACC-lamp': 0.7313907751753096, 'ACC-tub': 54.90131018701956, 'ACC-rail': 17.063962634011762, 'ACC-cushion': 0.8297668224687995, 'ACC-base, pedestal, stand': 0.0, 'ACC-box': 16.42992975333477, 'ACC-column, pillar': 0.0, 'ACC-signboard, sign': 16.680974773986584, 'ACC-chest of drawers, chest, bureau, dresser': 7.298404728266865, 'ACC-counter': 4.522503597325615, 'ACC-sand': 78.5309654979637, 'ACC-sink': 76.42303360469313, 'ACC-skyscraper': 18.593973495505224, 'ACC-fireplace': 38.415542283296865, 'ACC-refrigerator, icebox': 80.36443917573747, 'ACC-grandstand, covered stand': 48.92070676299068, 'ACC-path': 7.661385180390852, 'ACC-stairs': 41.16969922793024, 'ACC-runway': 61.60456632679726, 'ACC-case, display case, showcase, vitrine': 0.0, 'ACC-pool table, billiard table, snooker table': 85.14147567769025, 'ACC-pillow': 0.14709102357123177, 'ACC-screen door, screen': 0.0, 'ACC-stairway, staircase': 0.3908804987683872, 'ACC-river': 60.18685279307111, 'ACC-bridge, span': 91.48940963227, 'ACC-bookcase': 1.7295716153615495, 'ACC-blind, screen': 45.49583725804772, 'ACC-coffee table': 2.171676544142192, 'ACC-toilet, can, commode, crapper, pot, potty, stool, throne': 88.57000639791931, 'ACC-flower': 62.576209367420034, 'ACC-book': 53.07705546372753, 'ACC-hill': 0.0, 'ACC-bench': 62.2090848766363, 'ACC-countertop': 53.97093238884925, 'ACC-stove': 14.81976881160654, 'ACC-palm, palm tree': 3.90129710231016, 'ACC-kitchen island': 23.658562423857195, 'ACC-computer': 23.590959401135642, 'ACC-swivel chair': 54.51053688359166, 'ACC-boat': 84.5157004760545, 'ACC-bar': 0.05370413376683698, 'ACC-arcade machine': 14.322053974702028, 'ACC-hovel, hut, hutch, shack, shanty': 30.552171846217842, 'ACC-bus': 95.6193035949202, 'ACC-towel': 69.97914975225157, 'ACC-light': 64.74776622744498, 'ACC-truck': 77.98814563928875, 'ACC-tower': 6.134903306742333, 'ACC-chandelier': 1.1626590262105216, 'ACC-awning, sunshade, sunblind': 8.870710016787092, 'ACC-street lamp': 0.0, 'ACC-booth': 0.0, 'ACC-tv': 87.94082314414074, 'ACC-plane': 67.62691661931636, 'ACC-dirt track': 21.395178822216664, 'ACC-clothes': 46.82647250095726, 'ACC-pole': 24.199969080930664, 'ACC-land, ground, soil': 1.342942047443026, 'ACC-bannister, banister, balustrade, balusters, handrail': 1.4065128280502344, 'ACC-escalator, moving staircase, moving stairway': 0.0, 'ACC-ottoman, pouf, pouffe, puff, hassock': 30.51551038508793, 'ACC-bottle': 44.12124899262045, 'ACC-buffet, counter, sideboard': 0.2823001368343289, 'ACC-poster, posting, placard, notice, bill, card': 39.77516660258875, 'ACC-stage': 10.280593761417894, 'ACC-van': 34.128791328900626, 'ACC-ship': 8.036438615467821, 'ACC-fountain': 20.769331093975826, 'ACC-conveyer belt, conveyor belt, conveyer, conveyor, transporter': 35.06400740119194, 'ACC-canopy': 0.0, 'ACC-washer, automatic washer, washing machine': 48.34456072987988, 'ACC-plaything, toy': 17.207446042128925, 'ACC-pool': 39.923903234763294, 'ACC-stool': 48.73579710706946, 'ACC-barrel, cask': 57.04858345659347, 'ACC-basket, handbasket': 23.651472142144463, 'ACC-falls': 99.14987892214951, 'ACC-tent': 97.1923852633308, 'ACC-bag': 22.070884678834986, 'ACC-minibike, motorbike': 88.75300666809125, 'ACC-cradle': 26.168406573666324, 'ACC-oven': 74.7178956403201, 'ACC-ball': 22.763769713820047, 'ACC-food, solid food': 86.51076453274894, 'ACC-step, stair': 0.09411371821290193, 'ACC-tank, storage tank': 74.70981373983683, 'ACC-trade name': 0.0, 'ACC-microwave': 93.46128726097432, 'ACC-pot': 12.047977627389498, 'ACC-animal': 85.93243604308462, 'ACC-bicycle': 79.06879414072718, 'ACC-lake': 0.0, 'ACC-dishwasher': 8.238783926705175, 'ACC-screen': 0.0, 'ACC-blanket, cover': 0.0, 'ACC-sculpture': 65.21635914143977, 'ACC-hood, exhaust hood': 0.0, 'ACC-sconce': 0.45374864016323674, 'ACC-vase': 68.9828183382218, 'ACC-traffic light': 33.1904184354154, 'ACC-tray': 20.66102428737029, 'ACC-trash can': 22.464727360977815, 'ACC-fan': 9.924247942311895, 'ACC-pier': 39.18706427507897, 'ACC-crt screen': 10.554895203564191, 'ACC-plate': 32.65891890057357, 'ACC-monitor': 2.2339792399656915, 'ACC-bulletin board': 51.96720392257036, 'ACC-shower': 9.204919620375751, 'ACC-radiator': 29.92570733348865, 'ACC-glass, drinking glass': 27.533586416405427, 'ACC-clock': 33.470510146026925, 'ACC-flag': 51.60076343642118})])
INFO:datasets.evaluation.panoptic_evaluation:Writing all panoptic predictions to /tmp/panoptic_eval614xvyra ...
INFO:datasets.evaluation.panoptic_evaluation:Panoptic Evaluation Results:
| | PQ | SQ | RQ | #categories |
|:------:|:------:|:------:|:------:|:-------------:|
| All | 18.975 | 54.343 | 23.494 | 150 |
| Things | 16.644 | 56.361 | 21.067 | 100 |
| Stuff | 23.637 | 50.306 | 28.347 | 50 |
INFO:detectron2.evaluation.coco_evaluation:Preparing results for COCO format ...
INFO:detectron2.evaluation.coco_evaluation:Saving results to ../../data/output/test/coco_instances_results.json
INFO:detectron2.evaluation.coco_evaluation:Evaluating predictions with unofficial COCO API...
Loading and preparing results...
DONE (t=0.07s)
creating index...
index created!
INFO:detectron2.evaluation.fast_eval_api:Evaluate annotation type *bbox*
INFO:detectron2.evaluation.fast_eval_api:COCOeval_opt.evaluate() finished in 3.38 seconds.
INFO:detectron2.evaluation.fast_eval_api:Accumulating evaluation results...
INFO:detectron2.evaluation.fast_eval_api:COCOeval_opt.accumulate() finished in 0.45 seconds.
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.000
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.000
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.000
Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.000
Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.000
Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.000
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.000
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.000
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.000
Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.000
Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.000
Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.000
INFO:detectron2.evaluation.coco_evaluation:Evaluation results for bbox:
| AP | AP50 | AP75 | APs | APm | APl |
|:-----:|:------:|:------:|:-----:|:-----:|:-----:|
| 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 |
INFO:detectron2.evaluation.coco_evaluation:Per-category bbox AP:
| category | AP | category | AP | category | AP |
|:---------------------------|:------|:---------------------------------------------------------|:------|:------------------------------------------|:------|
| bed | 0.000 | window | 0.000 | cabinet | 0.000 |
| person | 0.000 | door | 0.000 | table | 0.000 |
| curtain | 0.000 | chair | 0.000 | car | 0.000 |
| painting, picture | 0.000 | sofa | 0.000 | shelf | 0.000 |
| mirror | 0.000 | armchair | 0.000 | seat | 0.000 |
| fence | 0.000 | desk | 0.000 | wardrobe, closet, press | 0.000 |
| lamp | 0.000 | tub | 0.000 | rail | 0.000 |
| cushion | 0.000 | box | 0.000 | column, pillar | 0.000 |
| signboard, sign | 0.000 | chest of drawers, chest, bureau, dresser | 0.000 | counter | 0.000 |
| sink | 0.000 | fireplace | 0.000 | refrigerator, icebox | 0.000 |
| stairs | 0.000 | case, display case, showcase, vitrine | 0.000 | pool table, billiard table, snooker table | 0.000 |
| pillow | 0.000 | screen door, screen | 0.000 | bookcase | 0.000 |
| coffee table | 0.000 | toilet, can, commode, crapper, pot, potty, stool, throne | 0.000 | flower | 0.000 |
| book | 0.000 | bench | 0.000 | countertop | 0.000 |
| stove | 0.000 | palm, palm tree | 0.000 | kitchen island | 0.000 |
| computer | 0.000 | swivel chair | 0.000 | boat | 0.000 |
| arcade machine | 0.000 | bus | 0.000 | towel | 0.000 |
| light | 0.000 | truck | 0.000 | chandelier | 0.000 |
| awning, sunshade, sunblind | 0.000 | street lamp | 0.000 | booth | 0.000 |
| tv | 0.000 | plane | 0.000 | clothes | 0.000 |
| pole | 0.000 | bannister, banister, balustrade, balusters, handrail | 0.000 | ottoman, pouf, pouffe, puff, hassock | 0.000 |
| bottle | 0.000 | van | 0.000 | ship | 0.000 |
| fountain | 0.000 | washer, automatic washer, washing machine | 0.000 | plaything, toy | 0.000 |
| stool | 0.000 | barrel, cask | 0.000 | basket, handbasket | 0.000 |
| bag | 0.000 | minibike, motorbike | 0.000 | oven | 0.000 |
| ball | 0.000 | food, solid food | 0.000 | step, stair | 0.000 |
| trade name | 0.000 | microwave | 0.000 | pot | 0.000 |
| animal | 0.000 | bicycle | 0.000 | dishwasher | 0.000 |
| screen | 0.000 | sculpture | 0.000 | hood, exhaust hood | 0.000 |
| sconce | 0.000 | vase | 0.000 | traffic light | 0.000 |
| tray | 0.000 | trash can | 0.000 | fan | 0.000 |
| plate | 0.000 | monitor | 0.000 | bulletin board | 0.000 |
| radiator | 0.000 | glass, drinking glass | 0.000 | clock | 0.000 |
| flag | 0.000 | | | | |
Loading and preparing results...
DONE (t=0.99s)
creating index...
index created!
INFO:detectron2.evaluation.fast_eval_api:Evaluate annotation type *segm*
INFO:detectron2.evaluation.fast_eval_api:COCOeval_opt.evaluate() finished in 3.73 seconds.
INFO:detectron2.evaluation.fast_eval_api:Accumulating evaluation results...
INFO:detectron2.evaluation.fast_eval_api:COCOeval_opt.accumulate() finished in 0.46 seconds.
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.101
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.180
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.099
Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.031
Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.119
Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.211
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.152
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.231
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.236
Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.082
Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.254
Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.397
INFO:detectron2.evaluation.coco_evaluation:Evaluation results for segm:
| AP | AP50 | AP75 | APs | APm | APl |
|:------:|:------:|:------:|:-----:|:------:|:------:|
| 10.118 | 18.028 | 9.900 | 3.135 | 11.858 | 21.123 |
INFO:detectron2.evaluation.coco_evaluation:Per-category segm AP:
| category | AP | category | AP | category | AP |
|:---------------------------|:-------|:---------------------------------------------------------|:-------|:------------------------------------------|:-------|
| bed | 40.379 | window | 14.416 | cabinet | 5.196 |
| person | 25.115 | door | 13.388 | table | 4.216 |
| curtain | 19.579 | chair | 13.900 | car | 30.626 |
| painting, picture | 2.294 | sofa | 34.544 | shelf | 2.526 |
| mirror | 36.965 | armchair | 17.870 | seat | 1.049 |
| fence | 2.379 | desk | 1.640 | wardrobe, closet, press | 0.069 |
| lamp | 3.607 | tub | 17.333 | rail | 0.478 |
| cushion | 3.489 | box | 2.804 | column, pillar | 0.845 |
| signboard, sign | 5.112 | chest of drawers, chest, bureau, dresser | 6.025 | counter | 0.291 |
| sink | 22.024 | fireplace | 4.248 | refrigerator, icebox | 53.591 |
| stairs | 6.187 | case, display case, showcase, vitrine | 0.015 | pool table, billiard table, snooker table | 45.625 |
| pillow | 0.395 | screen door, screen | 0.246 | bookcase | 0.242 |
| coffee table | 12.635 | toilet, can, commode, crapper, pot, potty, stool, throne | 50.129 | flower | 5.641 |
| book | 1.696 | bench | 3.321 | countertop | 3.413 |
| stove | 12.236 | palm, palm tree | 2.441 | kitchen island | 13.246 |
| computer | 0.816 | swivel chair | 4.634 | boat | 12.942 |
| arcade machine | 13.251 | bus | 35.889 | towel | 13.597 |
| light | 0.553 | truck | 10.119 | chandelier | 0.892 |
| awning, sunshade, sunblind | 3.094 | street lamp | 0.022 | booth | 0.206 |
| tv | 52.412 | plane | 21.707 | clothes | 1.826 |
| pole | 0.347 | bannister, banister, balustrade, balusters, handrail | 0.048 | ottoman, pouf, pouffe, puff, hassock | 10.297 |
| bottle | 8.790 | van | 10.889 | ship | 14.938 |
| fountain | 1.374 | washer, automatic washer, washing machine | 5.893 | plaything, toy | 0.103 |
| stool | 4.042 | barrel, cask | 15.875 | basket, handbasket | 3.537 |
| bag | 3.126 | minibike, motorbike | 19.403 | oven | 11.351 |
| ball | 7.071 | food, solid food | 0.624 | step, stair | 0.469 |
| trade name | 1.155 | microwave | 55.900 | pot | 0.650 |
| animal | 15.783 | bicycle | 8.503 | dishwasher | 3.732 |
| screen | 0.165 | sculpture | 5.731 | hood, exhaust hood | 0.000 |
| sconce | 0.672 | vase | 15.819 | traffic light | 5.216 |
| tray | 0.168 | trash can | 7.170 | fan | 0.648 |
| plate | 0.314 | monitor | 6.052 | bulletin board | 0.687 |
| radiator | 16.973 | glass, drinking glass | 7.016 | clock | 18.014 |
| flag | 7.855 | | | | |
INFO:__main__:{'ade20k_panoptic_val/sem_seg/mIoU': 25.138934771544356, 'ade20k_panoptic_val/panoptic_seg/PQ': 18.97506503772162, 'ade20k_panoptic_val/panoptic_seg/SQ': 54.342604302857666, 'ade20k_panoptic_val/panoptic_seg/RQ': 23.493577018336495, 'ade20k_panoptic_val/bbox/AP': 0.0, 'ade20k_panoptic_val/segm/AP': 10.118232565750764}MaureenZOU commented
For PQ, we use the threshold 0.4 instead of 0.8. I think that would be the problem.