Batch execution listing experiment folders wrongly
brunofavs opened this issue · 2 comments
This line :
atom/atom_batch_execution/scripts/process_results
Lines 98 to 101 in 6d2230d
It's not doing what it's supposed to be doing.
['results/nig_0.1-ntfv_0.25_run001_fold001',
'results/nig_0.1-ntfv_0.25_run001_fold002',
'results/nig_0.1-ntfv_0.25_run001_fold003',
'results/nig_0.1-ntfv_0.25_run001_fold004',
'results/nig_0.1-ntfv_0.25_run001_fold005',
'results/nig_0.1-ntfv_0.25_run002_fold001',
'results/nig_0.1-ntfv_0.25_run002_fold002',
'results/nig_0.1-ntfv_0.25_run002_fold003',
'results/nig_0.1-ntfv_0.25_run002_fold004',
'results/nig_0.1-ntfv_0.25_run002_fold005',
'results/nig_0.1-ntfv_0.25_run003_fold001',
'results/nig_0.1-ntfv_0.25_run003_fold002',
'results/nig_0.1-ntfv_0.25_run003_fold003',
'results/nig_0.1-ntfv_0.25_run003_fold004',
'results/nig_0.1-ntfv_0.25_run003_fold005']
results/nig_0.1-ntfv_0.25_run001_fold001
['results/nig_0.15-ntfv_0.15_run001_fold001',
'results/nig_0.15-ntfv_0.15_run001_fold002',
'results/nig_0.15-ntfv_0.15_run001_fold003',
'results/nig_0.15-ntfv_0.15_run001_fold004',
'results/nig_0.15-ntfv_0.15_run001_fold005',
'results/nig_0.15-ntfv_0.15_run002_fold001',
'results/nig_0.15-ntfv_0.15_run002_fold002',
'results/nig_0.15-ntfv_0.15_run002_fold003',
'results/nig_0.15-ntfv_0.15_run002_fold004',
'results/nig_0.15-ntfv_0.15_run002_fold005',
'results/nig_0.15-ntfv_0.15_run003_fold001',
'results/nig_0.15-ntfv_0.15_run003_fold002',
'results/nig_0.15-ntfv_0.15_run003_fold003',
'results/nig_0.15-ntfv_0.15_run003_fold004',
'results/nig_0.15-ntfv_0.15_run003_fold005',
'results/nig_0.15-ntfv_0.1_run001_fold001',
'results/nig_0.15-ntfv_0.1_run001_fold002',
'results/nig_0.15-ntfv_0.1_run001_fold003',
'results/nig_0.15-ntfv_0.1_run001_fold004',
'results/nig_0.15-ntfv_0.1_run001_fold005',
'results/nig_0.15-ntfv_0.1_run002_fold001',
'results/nig_0.15-ntfv_0.1_run002_fold002',
'results/nig_0.15-ntfv_0.1_run002_fold003',
'results/nig_0.15-ntfv_0.1_run002_fold004',
'results/nig_0.15-ntfv_0.1_run002_fold005',
'results/nig_0.15-ntfv_0.1_run003_fold001',
'results/nig_0.15-ntfv_0.1_run003_fold002',
'results/nig_0.15-ntfv_0.1_run003_fold003',
'results/nig_0.15-ntfv_0.1_run003_fold004',
'results/nig_0.15-ntfv_0.1_run003_fold005']
This is the print of the said line. When grepping the nig 0.15 ntfv 0.15 experiment it grabs wrong things with it as well.
Seems to be "fixed".
What was happening is that
glob.glob(args['results_folder'] + '/' + experiment + '*') with the nig_0.15-ntfv_0.1 experiment was also grepping the nig_0.15-ntfv_0.15.
Adding a "_" should fixed all cases since AS LONG AS we make sure to not use underscores in the experiment names.
A experience named nig_0.15-ntfv_0.1_banana would break in the same way it was breaking now.
I'm not the biggest fan of this solution because it's prone to breaking sneakily. There's not a lot we can do. One will always be a substring of the other.
What we can do is rewrite any underscores in the naming to hyphens in the batch execution. Like a safeguard, leaving underscore as a escape character. That or substitute the underscore in _fold_run for a very unusual escape character, like «fold«run or something, albeit I agree it would look kinda funky.
What is your opinion on this @miguelriemoliveira ?
That or a way more complicated filtering mechanism, considering now the experience settings are also on a yaml file for each folder, which is more bulletproof than having the settings encoded in the name.