How do I merge datasets?

[Choihyeonhwa]

I want to create a model by merging the PURE dataset and the UBFC-rPPG dataset and dividing it into train/val/test. What is the best way to proceed?

[girishvn]

Hi @Choihyeonhwa,

The most straightforward method of doing this is by adding a custom FILE_LIST_PATH. The file list path (which you can see specified in config.py) holds a list of data file paths. There is a seperate file_list_path for train / val / test. This value generally defaults to os.path.join(config.TRAIN.DATA.CACHED_PATH, 'DataFileLists'). However, you can specify a custom FILE_LIST_PATH .csv path name.

In this .csv file list the files (prepocessed.npy) used for training / validating the model, one per row.
In this way you can append the PURE and UBFC preprocessed file all into the same file.

Hope that makes sense!