Lightning-AI/torchmetrics

Error in ERGAS metric

Andrewwango opened this issue · 5 comments

In the definition of ERGAS, it should be \ ratio rather than * ratio

For example see ERGAS defined in Du et al, Meng et al

ergas_score = 100 * ratio * torch.sqrt(torch.sum((rmse_per_band / mean_target) ** 2, dim=1) / c)

Hi! thanks for your contribution!, great first issue!

Hi @Andrewwango, thanks for raising this issue.
I do not agree that there is an error in our formulation. If you look at the original paper by Wald (page 163 in the pdf) you will see that ERGAS metric is defined by multiplying the fraction and not dividing it. If you look at the paper by Du et al that you refer to, in equation (1) they ERGAS metric they multiply the ratio h/l. I am not sure why it is different in the paper by Meng et al.

I am updating the metric to reference the original paper and the specific formula we are using for calculating the metric in #2426

I disagree @SkafteNicki . The Wald and Du papers multiply by the ratio of pixel lengths , which is the inverse of the ratio. For example for ratio=4 the pixel length ratio h/l=1/4. The Wald paper gives examples of h/l=1/4 and h/l=1/2. Azarang et al states that h/l=1/4 for common satellites Pleiades-1A, WorldView-2, and GeoEye-1.

Therefore you should be dividing by the integer ratio. Please reopen this issue @SkafteNicki @Borda 😄

Hi @Andrewwango, thanks for correcting me. I am not a expert on these metrics and just saw that the original paper defined it by multiplication and division. Sorry on my part.
I make sure to fix it.

Hi @SkafteNicki, any progress on this?