Research topics that lead to coding questions

[peterhalpin]

Dealing with clustered data

1. How do to sample splitting with clustered data. There is a literature on resampling (e.g., bootstrapping) with clustered data, let's see if it has any lessons for how to cross-validation in this context

2. Power analyses for SEM with clustered data / robust GOF. Here is a nice article on power for SEM in R: https://doi.org/10.3758/s13428-020-01479-0 It notes that there are no analytical methods for power with multilevel data / robust GOF. In other literatures, design effects and effective sample sizes are used to address this issues. https://methods.sagepub.com/reference/encyclopedia-of-survey-research-methods/n153.xml. Playing loose and fast with this, we just need an ICC for the factor model....we might get this from the total score; or from a multilevel factor model (e.g., equation 8 of Muthen's 1991 paper on multilevel factor analysis)

Getting K:

1. One approach is a chi-square test of nested models. This suffers when then nesting model is incorrect, but power analysis is straightforward
2. Perhaps a better approach is equivalence tests. This uses the same test statistic as the standard tests, but difference critical regions. We would have to write our own code for power analysis, but it should be easy enough. These notes by Yuan are super helpful: https://www3.nd.edu/~kyuan/courses/sem/lectures/EquivalenceTesting.pdf