Omit List Working?

[nohelix]

Issue

It appears that the omit list is NOT working.

Details: Purple4 on 2-13-24 should have trial 1 omitted. This should result in a total of 44 trials but there are 45. The entry does show an omit of the 1st trial.

Desired Behavior

All stats in Calculate_Summary_Statistics should be calculated after omitting any trials in the omit list. Note that this is not true during import for Validate_Mat_Summary where we want to check the raw data.

Reproduce

Purple4_8kHz_60dB_0-0.5s_0catch_20240213-143605_BOX#002.mat.zip
Oddball Training_archive.csv.gz.zip

1. Checked .mat file (attached) which shows 45 trials completed.
2. Checked run_archive (attached) which shows 45 trials completed.
   run_archive %>% filter(rat_name == "Purple4" & date == 20240213) %>% pluck("stats", 1, "trial_count")
3. Confirmed that the omit list is correct and shows "1" for the 1st trial
   run_archive %>% filter(rat_name == "Purple4" & date == 20240213) %>% pluck("omit_list")
4. Checked the trials_archive and see there are 45 lines.
   filter(df, UUID == "20240213143605_1.68800130486488_0.0487271659541115") %>% nrow()

[tofof]

As far as I know, your original decision was that the omit list should have no automatic functionality and be there for manual data processing only. So, this is working as intended.

The summary statistics were intended to be calculated the same way as before PiedPiper was rolled out, so that they would compare 1:1.

In particular, you were very clear that 100% of trials should be recorded in the trials archive, regardless of whether they were flagged to be omitted or not, so that no data was ever permanently removed. So, point 4 above, in particular, is exactly as designed.

[nohelix]

Yes, this is how the system was designed. I simply forgot.