Getting bad results on the train set

[kishorepv]

I tried to evaluate the pretrained model (linked in the readme) on the EK-55 TRAIN split, following the suggested config in the readme. I get the following results:
{"noun_top1_acc": "0.48", "noun_top5_acc": "7.02", "split": "test_final", "verb_top1_acc": "34.10", "verb_top5_acc": "50.56"}

My config:

TRAIN:
  ENABLE: False
  DATASET: epickitchens
  BATCH_SIZE: 32
  EVAL_PERIOD: 2
  CHECKPOINT_PERIOD: 1
  CHECKPOINT_TYPE: caffe2
  AUTO_RESUME: True
  FINETUNE: True
DATA:
  NUM_FRAMES: 32
  SAMPLING_RATE: 2
  TRAIN_JITTER_SCALES: [256, 320]
  TRAIN_CROP_SIZE: 224
  TEST_CROP_SIZE: 256
  INPUT_CHANNEL_NUM: [3, 3]
SLOWFAST:
  ALPHA: 4
  BETA_INV: 8
  FUSION_CONV_CHANNEL_RATIO: 2
  FUSION_KERNEL_SZ: 7
RESNET:
  ZERO_INIT_FINAL_BN: True
  WIDTH_PER_GROUP: 64
  NUM_GROUPS: 1
  DEPTH: 50
  TRANS_FUNC: bottleneck_transform
  STRIDE_1X1: False
  NUM_BLOCK_TEMP_KERNEL: [[3, 3], [4, 4], [6, 6], [3, 3]]
  SPATIAL_STRIDES: [[1, 1], [2, 2], [2, 2], [2, 2]]
  SPATIAL_DILATIONS: [[1, 1], [1, 1], [1, 1], [1, 1]]
NONLOCAL:
  LOCATION: [[[], []], [[], []], [[], []], [[], []]]
  GROUP: [[1, 1], [1, 1], [1, 1], [1, 1]]
  INSTANTIATION: dot_product
BN:
  USE_PRECISE_STATS: True
  FREEZE: True
  NUM_BATCHES_PRECISE: 200
  MOMENTUM: 0.1
  WEIGHT_DECAY: 0.0
SOLVER:
  BASE_LR: 0.01
  LR_POLICY: steps_with_relative_lrs
  STEPS: [0, 20, 25]
  LRS: [1, 0.1, 0.01]
  MAX_EPOCH: 30
  MOMENTUM: 0.9
  WEIGHT_DECAY: 1e-4
  WARMUP_EPOCHS: -1
  WARMUP_START_LR: 0.01
  OPTIMIZING_METHOD: sgd
MODEL:
  NUM_CLASSES: [97, 300]
  ARCH: slowfast
  MODEL_NAME: SlowFast
  LOSS_FUNC: cross_entropy
  DROPOUT_RATE: 0.5
TEST:
  ENABLE: True
  DATASET: epickitchens
  BATCH_SIZE: 64
  NUM_SPATIAL_CROPS: 3
  NUM_ENSEMBLE_VIEWS: 10
  CHECKPOINT_FILE_PATH: /private/home/kishorepv/projects/retrieval-augmented-videos/extract-features/video_feature_extractor/model/SlowFast_EK100_EKOfficial_checkpoint.pyth

DATA_LOADER:
  NUM_WORKERS: 8
  PIN_MEMORY: True
EPICKITCHENS:
  TRAIN_PLUS_VAL: False
  VISUAL_DATA_DIR: /checkpoint/kishorepv/projects/retrieval-augmented-videos/data/EK-50/train
  ANNOTATIONS_DIR: /checkpoint/kishorepv/projects/retrieval-augmented-videos/data/EK-50-annotations
  TEST_LIST: EPIC_train_action_labels.pkl
  TEST_SPLIT: test2Caffe2OfficalCheckpointSpatialCrops3EnsembleViews10
NUM_GPUS: 8
NUM_SHARDS: 1
RNG_SEED: 0
OUTPUT_DIR: /checkpoint/kishorepv/projects/retrieval-augmented-videos/data/output/EK-50

Can you suggest what might be wrong?

[dimadamen]

The released model here is for EPIC-KITCHENS-100 and should not be used for EPIC-KITCHENS-55. The list of classes has changed between the two versions (please check the ArXiv paper for explanation of these changes). The class numbers thus do not correspond and thus the evaluation is incorrect.

If you want trained models on EPIC-KITCHENS-55, you should refer to the models released in 2019
https://github.com/epic-kitchens/epic-kitchens-55-action-models

[kishorepv]

Thanks!