Before running the code, make sure that the folder is named as MAD.
Also modify two variables hdd_dir and ImageNet_dir in common_flags.py to valid paths on your own machine.
Run python images-web-crawler/sample_ours.py under project root dir, or run python sample_ours.py under images-web-crawler.
Most files under images-web-crawler comes from this repo. Thanks to the authors for their great work!
Note that images-web-crawler only support Python2, while the rest of this repo support Python3.
Run python dataset/construct_dataset.py under project root dir, or run python construct_dataset.py under dataset.
First run python save_jsons.py under class_info.
Then run python construct_path.py under project root dir.
These two commands will construct and save multiple .json, .csv and .txt files, which mainly aims at showing the WordNet tree hierarchy.
All these files are already uploaded in this repo, so you can skip this step if you want.
For example, code_with_imgnet_id_readable.json shows the paths from WordNet root node to each ImageNet class leaf node.
Take the item with key="872" in code_with_imgnet_id_readable.json as an example, this item shows that the path from root node to leaf node "tripod", which has id=872 in ImageNet is:
"tripod; rack, stand; support; device; instrumentality, instrumentation; artifact, artefact; whole, unit; object, physical object; physical entity; entity".
Run python test_our_data.py to do prediction on the web-scale unlabeled dataset.
Run python compare_and_select.py to select MAD images.
After providing human annotations on all selected MAD images, run python global_ranking.py to get global ranking of all classifiers.