qrel dir

[laura-dietz]

Here is design problem. Your code forces me to use a qrels directory -- I much rather give it a qrels file (since there are so many qrels it is a bit inconvenient to please each in its own directory)

dietz@jelly:~/datasci/BryanTuckerBahram/trec-eval$ java -jar target/trec-eval-1.0-SNAPSHOT.jar ../../test200-mock1.run ../../all.test200.cbor.hierarchical.qrels
Exception in thread "main" java.lang.IllegalArgumentException: The qrel directory, really needs to be a directory
at edu.unh.cs.trec.GroundTruth.(GroundTruth.java:21)
at edu.unh.cs.trec.Eval.main(Eval.java:19)

I am producing the same numbers as you predicted:

dietz@jelly:~/datasci/BryanTuckerBahram/trec-eval$ java -jar target/trec-eval-1.0-SNAPSHOT.jar ../../test200-mock1.run ../../
Metric Precision@5 = 0.07125110913930785         err: 0.11847349070615334
Metric Precision@10 = 0.07231588287488802        err: 0.08859674826283298
Metric Precision@15 = 0.07446504249964714        err: 0.07676506298155762
Metric Precision@20 = 0.07473883622766549        err: 0.07138910728664746
Metric Precision@100 = 0.07371820217967284       err: 0.06312748611877798
Metric R-Precision = 0.07394524340307009         err: 0.18823212470003367
Metric MAP = 0.16836729318149882         err: 0.1842514830441846

[tuckerowens]

I thought this would work like BitBucket, I don't actually like using hashtags