getCombinedDataset

[Entromorgan]

Hello, is there any updated version of the file 'Dataset.py'?

When I tried to run trainingLoop.py, it raised the following error:

Traceback (most recent call last): File "trainingLoop.py", line 26, in <module> testDatasetC = getCombinedDataset('countix/countix_test.csv', TypeError: getCombinedDataset() got an unexpected keyword argument 'frame_per_vid'

It seems that the original getCombineDataset function only receives three arguments, i.e.,

def getCombinedDataset(dfPath, videoDir, videoPrefix).

But when calling this function in trainingLoop.py, it gives five arguments, e.g.,

getCombinedDataset('countix/countix_test.csv', 'testvids', 'test', frame_per_vid=frame_per_vid, multiple=multiple).

I've also looked over the definition of other dataset-related functions like SyntheticDataset, BlenderDataset, unfortunately, none of them is suitable.

[confifu]

Does it work without the frame_per_vid and multiple parameters? I added these parameters while experimenting but they are not needed. Also get rid of the new parameters in SyntheticDataset. In BlenderDataset set the frame_per_vid parameter equal to 64.

[Entromorgan]

Thanks for the timely reply!

I'm not sure if removing these two parameters works because after that, I run into another problem, which traces back to the training_loop function that locates in trainLoop.py

Traceback (most recent call last): File "trainingLoop.py", line 73, in <module> trLoss, valLoss = training_loop( 10, File "/workspace/app/torch/RepNet-Pytorch-Modified/trainLoop.py", line 139, in training_loop for X, y in pbar: ValueError: too many values to unpack (expected 2)

Can it be something wrong with the initialization of the training set dataloader?

[Entromorgan]

I've tried to print out the size of the training set:

trainList = [trainDatasetC, trainDatasetS3] #, trainDatasetB]
random.shuffle(trainList)
trainDataset = ConcatDataset(trainList)

model = RepNet(frame_per_vid)
model = model.to(device)

print(len(trainDataset))
print(len(trainDataset[0]))
```.

The result is 7332 and 3 respectively so it does have two dimensions, which is a little bit confusing...

[Entromorgan]

Then I try to dig into the dataloader, and find out that in function getCombinedDataset, it calls the function dataset_with_indices, whose __getitem__ method returns three items.

So I change the original "for X, y in pbar" -> "for X, y, _ in pbar", the training process seems working just fine.

BTW, are the trainvids, testvids, and valvids belong to countix that used by the original RepNet paper?

[confifu]

That was because the synthetic dataset was returning X, y, index in the getitem function. It should only return X, y Now it should work. You can get rid of the indices altogether and only return X, y everywhere.

BTW, are the trainvids, testvids, and valvids belong to countix that used by the original RepNet paper?

Yes.

[Entromorgan]

Got it. Thx very much!

[fjptscottliu]

Got it. Thx very much!

can you reproduce the work?I think there are some mistakes on losses or Datasets.py