NaNs warning, failure to converge

[cbohil]

Hi Fabian,

I'm trying to use grt_wind_fit with some sparse confusion matrices. Sometimes I get a warning about NaNs, and sometimes I don't - for the same data sets! Below is an example with 5 confusion matrices - if I run it a few times I eventually get a solution without NaNs (on the 4th try below). But when I try to run a much larger set of confusion matrices, I get the NaNs problem every time.

Below is an example - 5 confusion matrices. The first 3 calls to grt_wind_fit fail, then finally on the 4th try it runs fine. Any input would be appreciated - CB

fitted_model <- grt_wind_fit(cmats)
Warning message:
In sqrt((results$fullpars[10]) * (results$fullpars[11])) : NaNs produced

plot(fitted_model)
Error in solve.default(model$covmat[[i]]) :
system is computationally singular: reciprocal condition number = 0

cmats
[[1]]
[,1] [,2] [,3] [,4]
[1,] 0 0 44 1
[2,] 0 0 17 28
[3,] 0 0 35 10
[4,] 0 0 19 26

[[2]]
[,1] [,2] [,3] [,4]
[1,] 41 4 0 0
[2,] 21 24 0 0
[3,] 37 8 0 0
[4,] 11 33 1 0

[[3]]
[,1] [,2] [,3] [,4]
[1,] 45 0 0 0
[2,] 41 3 1 0
[3,] 44 1 0 0
[4,] 36 8 1 0

[[4]]
[,1] [,2] [,3] [,4]
[1,] 9 1 32 3
[2,] 6 5 12 22
[3,] 3 0 27 15
[4,] 1 4 11 29

[[5]]
[,1] [,2] [,3] [,4]
[1,] 39 6 0 0
[2,] 15 30 0 0
[3,] 31 14 0 0
[4,] 8 37 0 0

fitted_model <- grt_wind_fit(cmats)
Warning message:
In sqrt((results$fullpars[10]) * (results$fullpars[11])) : NaNs produced

fitted_model <- grt_wind_fit(cmats)
Warning message:
In sqrt((results$fullpars[10]) * (results$fullpars[11])) : NaNs produced

fitted_model <- grt_wind_fit(cmats)

plot(fitted_model)

[fsotoc]

I'm checking this now, but while entering the data I noticed that for several participants there is either an error in the way these matrices were computed or the participant did not learn the task:

• participant 1 is 33% correct and only uses responses 3 and 4; never uses 1 and 2.
• participant 2 is 36% correct and almost exclusively uses responses 1 and 2, almost never 3 and 4
• participant 3 is 27% correct and seems to just respond "1" to almost everything
• participant 4 is 39% correct and is biased to use mostly responses 3 and 4
• participant 5 is 38% correct and uses responses 1 and 2 exclusively

If there are no problems with how the matrices were computed, then most of these participants should be eliminated from any model-based GRT analysis, I think. Participant 3 definitely offers no information. The other participants might offer some info regarding those two responses that they are biased to give, but I don't see how their data could be analyzed with traditional GRT analyses implemented in this toolbox.

[fsotoc]

You get this error because the model fitting procedure is finding a solution with some random starting configurations but not with others. This is probably due to the fact that the data is problematic. You should start by checking whether these confusion matrices were calculated properly, and if they were you should decide whether you want to run a GRT analysis with your data.

If you still want to run the analysis to see what you get, you can try the "shotgun" approach: run the analysis with grt_wind_parallel instead of grt_wind and set the parameter nreps to a relatively high value (actually, I'd recommend doing this every time you fit GRT-wIND). Try using nreps=20 for a start. It will take longer to finish, but essentially it tries running grt_wind with 20 different initial random configurations, gets rid of all the problematic ones (like those that you are getting) and picks the best solution out of those that finish without error. If you still get the error after that, you can try increasing the value for nreps.

[cbohil]

Thanks Fabian - will try this and will probably eliminate non-learners. I'll let you know if that solves it.

Corey

[fsotoc]

Great! I'm closing the issue for now but you can reopen it if this solution doesn't work.