Significance of converegcne
Closed this issue · 3 comments
Hello,
I checked the document of rEDM and found this.
I do think that this statement is confusing since the fact that cross map skill with the largest library size is above correlation is not the sign of convergence.
Instead, only if there is significant improvement with increasing L, can we say that there is converegence.
Btw, can we compare two directions by simply comapring the cross map skill with largest L or shall we compare the rate pf convergence to say that which direction is stronger? (Rate of convergence=delta(cross map skill)/delta(Library size))
Any help would be appreciated!
The above picture is from this page of tutorial. https://ha0ye.github.io/rEDM/articles/rEDM.html#convergent-cross-mapping-ccm
I can see how that statement is confusing. Nonetheless, it seems accurate. The red curve indicates convergence, while blue does not. That the red cross map value (at full library size) is greater than cross correlation is not evidence of significance, but, that is not claimed. As mentioned, some people take that difference as a sign, perhaps even quantitative metric, of non linearity. This is what was done in the previous version of rEDM, so it remains in the current tutorial.
Personally, I feel that the cross map correlation difference can only be viewed as a potential indicator, since linear correlation can over/under predict when the data are non linear, Anscombes quartet. A better metric would be a mutual information nonlinearity discrimination. But, this comparison to linear correlation still has some value as an illustration of the Type III error presuming linearity where it should not be applied.
I'm sure others will have different opinions.
Rate of convergence: that's an interesting question, I'm not aware of validated studies that relate the rate of convergence with cross map "strength". My understanding is to use the value at the full library size, or, the value at which the cross map metric has converged.
What I take from Sugihara et al. is that convergence arises if the variables share simplex predictive power, shared state dynamics, as the library of states of provides more complete information of the system dynamics. Trying to say something about the "rate" of convergence is saying something about an incomplete, partial, view of the system dynamics. I could see that having value in perhaps assessing "how complete" information regarding an observed variable is represented in the library of states. Perhaps someone with a deeper understanding can clarify this...
Thanks for your detailed explaination!