NDCG@10 fails when relevant documents are expressed in gain -> docs format
Closed this issue · 1 comments
Describe the bug
NDCG@10 fails when relevant documents are expressed in gain -> docs format
"relevant_documents": { "3": ["1658090", "8927398273", "9879878"] }
To Reproduce
Create a judgment set using the gain -> docs format for expressing judgments.
Expected behavior
The format should be correctly parsed by all RRE components
StackTrace
Caused by: java.lang.NullPointerException at io.sease.rre.core.domain.metrics.impl.NDCGAtTen.lambda$idealDcg$0(NDCGAtTen.java:75) at java.util.stream.Collectors.lambda$groupingBy$45(Collectors.java:907) at java.util.stream.ReduceOps$3ReducingSink.accept(ReduceOps.java:169) at java.util.Iterator.forEachRemaining(Iterator.java:116) at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801) at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481) at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471) at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708) at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499) at io.sease.rre.core.domain.metrics.impl.NDCGAtTen.idealDcg(NDCGAtTen.java:75) at io.sease.rre.core.domain.metrics.impl.NDCGAtTen.access$200(NDCGAtTen.java:24) at io.sease.rre.core.domain.metrics.impl.NDCGAtTen$1.value(NDCGAtTen.java:61) at io.sease.rre.core.domain.DomainMember.lambda$null$4(DomainMember.java:120) at java.util.Optional.ifPresent(Optional.java:159)
Fixed. The patch provides also some useful logging messages during the evaluation process.