ext.index

[MCristinaMiranda]

This function presents problems when I use apply or within a cicle. For example, for 1000x500 matrix (samples are coluns) it calculates the first 10 but no more. I've tried with a cicle in stead and I realize that sometimes it does not calculate but I don't understand why. Has any one tried the same problem? I wanted to compare it with other estimates, calculating the mean value for some number of replications...

[lbelzile]

Apologies Cristina for not seeing this earlier and thanks for filing the issue. If I understand you correctly, the function returns an error when using a for-loop or apply.

Is this the error you see?

Error in lm.wfit(x, y, w, offset = offset, singular.ok = singular.ok, : NA/NaN/Inf in 'x'

It was caused by an off-by-one error in the number of exceedances, resulting in an Inf weight being passed to the function lm. Now fixed.

If not, please provide a minimal reproducible example.

The following should work with the latest version on Github.

samp <- rmev(n = 1000L, d = 50L, param = 0.2, model = "log")
apply(samp, 2, function(x){ext.index(x = x, q = c(0.5, 0.6, 0.7), method= "wls"))})

[MCristinaMiranda]

Dear Léo, Thank you so much for your answer. I’m afraid that it still has some problems though… I send you my matrices D and C, with 500 samples (columns) of 5-dependent sequences (with extremal index 0.2). I used q= quantiles = seq(0.8, 0.99, length = 30). If I use the function with D[,1:2] it works. But with 15 columns, for example, I get gap=apply(D[,1:15], 2, function(x){ext.index(x = x, q = quantiles, method= "wls")}) Error in while (floor(theta * (N - 1)) != Nc.new && trials < 30) { : missing value where TRUE/FALSE needed In addition: There were 30 warnings (use warnings() to see them) With D, also doesn’t work, neither with C – contaminated sample so it has a cluster with many more exceedances than the rest of the sample. apply(D, 2, function(x){ext.index(x = x, q = quantiles, method= "wls")}) Error in while (floor(theta * (N - 1)) != Nc.new && trials < 30) { : missing value where TRUE/FALSE needed In addition: There were 30 warnings (use warnings() to see them) Thank you for all possible help. Kind regards Cristina Miranda (Professora Adjunta) [Logo Isca 02] De: Léo Belzile [mailto:notifications@github.com] Enviada: 10 de junho de 2019 11:51 Para: lbelzile/mev <mev@noreply.github.com<mailto:mev@noreply.github.com>> Cc: Cristina Miranda <cristina.miranda@ua.pt<mailto:cristina.miranda@ua.pt>>; Author <author@noreply.github.com<mailto:author@noreply.github.com>> Assunto: Re: [lbelzile/mev] ext.index (#4) Apologies Cristina for not seeing this earlier and thanks for filing the issue. If I understand you correctly, the function returns an error when using a for-loop or apply. Is this the error you see? Error in lm.wfit(x, y, w, offset = offset, singular.ok = singular.ok, : NA/NaN/Inf in 'x' It was caused by an off-by-one error in the number of exceedances, resulting in an Inf weight being passed to the function lm. Now fixed. If not, please provide a minimal reproducible example<https://stackoverflow.com/questions/5963269/how-to-make-a-great-r-reproducible-example>. The following should work with the latest version on Github. samp <- rmev(n = 1000L, d = 50L, param = 0.2, model = "log") apply(samp, 2, function(x){ext.index(x = x, q = c(0.5, 0.6, 0.7), method= "wls"))}) — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub<#4?email_source=notifications&email_token=AMCGKTCQ3LL3Q5N2WIXUXZ3PZYWZNA5CNFSM4HMYLBFKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXJRZDI#issuecomment-500374669>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AMCGKTDC5XPHG5JJUYNQH6DPZYWZNANCNFSM4HMYLBFA>.

[lbelzile]

The algorithm in Süveges (2007) is a bit unclear, but alternates two steps until convergence. It seems that the procedure doesn't always work in small samples. The problem comes from the fact that, sometimes, the number of spacings used to fit the exponential is 1, in which case the routine breaks down (the estimator is obtained by fitting least squares with an intercept and an additional parameter).

The weighted least square routine should work most of the time, but if your thresholds are too high, the fit will fail (and there are plenty of good reasons then for it to fail). There are also cases where there are no gaps to use to compute an estimator, regardless of the method. I now catch these "errors" and return NAs with warnings.

In summary,

1. Function now return NAs if (1) the routine is divergent (number of gaps used to fit the model is less than or equal to 1).
2. If there are only zero gaps, the function returns 0 with a warning
3. Return NA if there are less than one exceedance over the threshold (e.g., because of extrapolation with the largest values).