Add some Instructions to run inferencing on custom dataset
Closed this issue · 9 comments
Hi,
Thank you for the work here.
Could you add some guidelines to allow users to run inferencing on custom dataset.
I have a list of .mp4 files which I like to run inference on.
Thank you
Hi @shamussim-ai,
Please follow the instructions here to achieve this. If you have any questions after trying the process there, feel free to make a new issue regarding those questions.
Best regards,
Akshay
``Hi @yahskapar
I have done so.
My CustomLoader.py file
import cv2
import numpy as np
import os
from .BaseLoader import BaseLoader
import glob
class CustomLoader(BaseLoader):
def __init__(self, name, data_path, config_data):
"""Initializes a custom dataloader.
Args:
data_path (str): path of a folder which stores raw video data.
name (str): name of the dataloader.
config_data (CfgNode): data settings (ref: config.py).
"""
self.name = name
self.data_path = data_path
self.config_data = config_data
super().__init__(name, data_path, config_data)
def get_raw_data(self, raw_data_paths):
"""Fetches a list of video file paths from specified directories."""
video_files = []
if isinstance(raw_data_paths, list):
for path in raw_data_paths:
files = glob.glob(os.path.join(path, "*.mp4"))
video_files.extend(files)
else:
video_files = glob.glob(os.path.join(raw_data_paths, "*.mp4"))
if not video_files:
raise ValueError(f"No MP4 files found in {raw_data_paths}.")
print(f"Found .mp4 files: {video_files}") # Debugging statement
return video_files
def preprocess_dataset(self, data_dirs, config_preprocess, begin, end):
"""Custom preprocessing tailored for inference on MP4 files."""
if isinstance(self.data_path, list):
for path in self.data_path:
video_files = self.get_raw_data(path)
self.read_video(video_files)
else:
video_files = self.get_raw_data(self.data_path)
self.read_video(video_files)
def preprocess_dataset(self, data_dirs, config_preprocess, begin, end):
"""Custom preprocessing tailored for inference on MP4 files."""
video_file = self.get_raw_data(self.data_path) # This should now correctly retrieve a list of file paths
frames = self.read_video(video_file[0])
return frames
@staticmethod
def read_video(video_file):
"""Reads video from an MP4 file and returns frames."""
cap = cv2.VideoCapture(video_file)
frames = []
while True:
ret, frame = cap.read()
if not ret:
break
frames.append(frame)
cap.release()
return np.array(frames)
In main.py I edited my code to be as shown below
if config.TOOLBOX_MODE == "train_and_test" or config.TOOLBOX_MODE == "only_test":
# test_loader
if config.TEST.DATA.DATASET == "UBFC-rPPG":
test_loader = data_loader.UBFCrPPGLoader.UBFCrPPGLoader
elif config.TEST.DATA.DATASET == "PURE":
test_loader = data_loader.PURELoader.PURELoader
elif config.TEST.DATA.DATASET == "SCAMPS":
test_loader = data_loader.SCAMPSLoader.SCAMPSLoader
elif config.TEST.DATA.DATASET == "MMPD":
test_loader = data_loader.MMPDLoader.MMPDLoader
elif config.TEST.DATA.DATASET == "BP4DPlus":
test_loader = data_loader.BP4DPlusLoader.BP4DPlusLoader
elif config.TEST.DATA.DATASET == "BP4DPlusBigSmall":
test_loader = data_loader.BP4DPlusBigSmallLoader.BP4DPlusBigSmallLoader
elif config.TEST.DATA.DATASET == "UBFC-PHYS":
test_loader = data_loader.UBFCPHYSLoader.UBFCPHYSLoader
elif config.TEST.DATA.DATASET == "iBVP":
test_loader = data_loader.iBVPLoader.iBVPLoader
elif config.TEST.DATA.DATASET == "Custom":
test_loader = data_loader.CustomLoader.CustomLoader
Modified a custom.yaml
BASE: [""]
TOOLBOX_MODE: "only_test" # "train_and_test" or "only_test"
TEST:
METRICS: ["MAE", "RMSE", "MAPE", "Pearson", "SNR", "BA"]
USE_LAST_EPOCH: True
DATA:
FS: 30
DATASET: Custom
DO_PREPROCESS: True # if first time, should be true
DATA_FORMAT: NDCHW
DATA_PATH: "/home/azureuser/cloudfiles/code/Users/rPPG-Toolbox/data/s1" # Raw dataset path, need to be updated
CACHED_PATH: "/home/azureuser/cloudfiles/code/Users/rPPG-Toolbox/PreprocessedData"
EXP_DATA_NAME: ""
BEGIN: 0.0
END: 1.0
PREPROCESS:
DATA_TYPE: ["DiffNormalized", "Standardized"]
LABEL_TYPE: DiffNormalized
DO_CHUNK: True
CHUNK_LENGTH: 180
CROP_FACE:
DO_CROP_FACE: True
BACKEND: "HC" # HC for Haar Cascade, RF for RetinaFace
USE_LARGE_FACE_BOX: True
LARGE_BOX_COEF: 1.5
DETECTION:
DO_DYNAMIC_DETECTION: False
DYNAMIC_DETECTION_FREQUENCY: 30
USE_MEDIAN_FACE_BOX: False # This should be used ONLY if dynamic detection is used
RESIZE:
H: 72
W: 72
DEVICE: cuda:0
NUM_OF_GPU_TRAIN: 1
LOG:
PATH: runs/exp
MODEL:
DROP_RATE: 0.2
NAME: DeepPhys
INFERENCE:
BATCH_SIZE: 4
EVALUATION_METHOD: FFT # "FFT" or "peak detection"
EVALUATION_WINDOW:
USE_SMALLER_WINDOW: False # Change this if you'd like an evaluation window smaller than the test video length
WINDOW_SIZE: 10 # In seconds
MODEL_PATH: "final_model_release/PURE_DeepPhys.pth"
Code to replicate
python main.py --config_file ./configs/infer_configs/custom1.yaml
Error
...config files content printed here...
Found .mp4 files: ['/home/azureuser/cloudfiles/code/Users/rPPG-Toolbox/data/s1/S1_Bright Place_No Compress_Chinese_High_HR.mp4']
Found .mp4 files: ['/home/azureuser/cloudfiles/code/Users/rPPG-Toolbox/data/s1/S1_Bright Place_No Compress_Chinese_High_HR.mp4']
Cached Data Path PreprocessedData/Custom_SizeW72_SizeH72_ClipLength180_DataTypeDiffNormalized_Standardized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_BackendHC_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse
File List Path PreprocessedData/DataFileLists/Custom_SizeW72_SizeH72_ClipLength180_DataTypeDiffNormalized_Standardized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_BackendHC_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse_0.0_1.0.csv
test Preprocessed Dataset Length: 0
===Testing===
Testing uses pretrained model!
Running model evaluation on the testing dataset!
0it [00:00, ?it/s]
There is the problem of the preprocessed dataset being of length zero.
Also, I am not sure what the Cached Data Path is supposed to be? Where should this path be pointing to?
Hi @ss8319,
You need to set your CACHED_PATH to something valid rather than commenting out and letting it be set to the default value. Try something like /home/azureuser/cloudfiles/code/Users/rPPG-Toolbox/PreprocessedData, similar to how the default config files in this repo have done it. I'm not totally sure if that will resolve the issue you're running into it, however - it's totally possible something is still going wrong on the preprocessing side of things here that isn't obvious from your terminal output.
Hi @yahskapar
Oops sorry, I was showing the wrong yaml file. My CACHED_PATH is a valid path. I have updated it now. I still get the same issue on the preprocessing side. Any idea what might be causing it.
I'd start off by carefully debugging your custom dataloader, perhaps by using print statements at key points (e.g., before returning from the function). Also, based on the paths you do have defined in your config, is nothing being saved there despite your terminal output not containing error messages, or does your terminal output contain error messages related to preprocessing?
I'm going to go ahead and close this issue based on the lack of activity, but please feel free to re-open it if your situation persists or make a new issue if you have any other questions or concerns.
Thanks,
Akshay
Hi @yahskapar
Please provide the source text you would like to have translated
Oops sorry, I was showing the wrong yaml file. My CACHED_PATH is a valid path. I have updated it now. I still get the same issue on the preprocessing side. Any idea what might be causing it?
Hello～I also need to upload my own video for inference. I found that the test data requires uploading a label to work. I tried to modify it myself but failed. I don't know if you can now use your own video for inference. Could you share the method? Thank you very much！
Hi @znygithub,
Can you make a new issue regarding your situation? A bit confused on the exact details, and whether or not it has to do with the original issue posted here. A new issue with more details may help clarify things so that you can get the help you need.