Unbalanced Designs

[Lubell]

Hi -
I've been delighted so far with the TFCE toolbox and I was just wanted to run something by you to make sure that it is valid.

I know that the ept_TFCE_ANOVA only works on balanced designs the way it is setup now. That was an issue for me because I've got two lesion groups and controls and I wasn't able to record an equal number of lesion patients. I dug around and found FMUT (https://github.com/ericcfields/FMUT) which is a branch/extension of the Mass Univariate Toolbox. FMUT has implemented cluster permutation for between group and within group unbalanced designs. I took the FMUT permutation function, which returns an F-distribution (nPermXchanXtime for each factor) and than rather than run the cluster-mass identification on the factor F-distributions, I ran each nPerm through the ept_mex_TFCE2D. Essentially it is the same thing that you do in ept_TFCE_ANOVA, except it builds a permuted F distribution first (which is slower) and then locates clusters. In ept_TFCE_ANOVA you return an F_Perm for each permutation and in that same loop run ept_mex_TFCE2D on it.

Am I crazy? The values I get back aren't insane and are in keeping with what I find using FMUT itself and reflect what the data looks like... Just wanted to check.

Also many thanks for the toolbox and your PhD Thesis which was actually an enjoyable read. Great piece of work.
J

[Mensen]

That all sounds completely reasonable.

I often do things like this by myself as well. Just replace the T-test, or ANOVAs with some other statistical calculation and then run the TFCE on those values etc. The only reason the balanced requirement is in the toolbox still is because its the only implementation I've found to be relatively fast enough to run in a non-frustrating amount of time.

How long does your analysis take using the FMUT function? Perhaps I should check it out.

I've generally been relying on mixed models now to do all my statistics so I can include as many factors as I like along with some more odd mixed designs for experiments... but the whole analysis often now takes hours to complete because of this. So didn't want to make that so publicly available since a lot can go wrong in your design (and in the worst case, not cause an error and still produce results that look believable).

[Lubell]

It's doesn't actually take that long, but I'm also unhappily relegated to 64 electrodes. For 5000 permutations with 46 subjects, 64 electrodes, and 225 samples the run time is about 4.5 minutes. There is a bit of permuting and whatnot to get the data into FMUT format (elec,time,condition,subject) but then the relevant FMUT function is perm_spANOVA, which is run on each factor individually. The dims argument that gets passed to perm_spANOVA was the only tricky part, but it's just the condition index plus 2 for main effect factors and [1 2]+2 for the interaction (1 and 2 being the condition indexes for a 3x2). perm_spANOVA is a wrapper function of sorts for the type of test, but its easy to dig down and find out what is exactly being done, i.e., if you wanted to speed things up even more. Thanks again for the toolbox and thesis.

…

On Thu, Apr 19, 2018 at 10:22 AM, Armand Mensen ***@***.***> wrote: That all sounds completely reasonable. I often do things like this by myself as well. Just replace the T-test, or ANOVAs with some other statistical calculation and then run the TFCE on those values etc. The only reason the balanced requirement is in the toolbox still is because its the only implementation I've found to be relatively fast enough to run in a non-frustrating amount of time. How long does your analysis take using the FMUT function? Perhaps I should check it out. I've generally been relying on mixed models now to do all my statistics so I can include as many factors as I like along with some more odd mixed designs for experiments... but the whole analysis often now takes hours to complete because of this. So didn't want to make that so publicly available since a lot can go wrong in your design (and in the worst case, not cause an error and still produce results that look believable). — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#19 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACYwTs0eXmgZ8HrA9_iu_vg6I7X0Syy3ks5tqEkugaJpZM4TZf9b> .

[Mensen]

Great tip!

I'll look into it for sure.