omitnorms

Question

omitnorms

matteogrolla opened this issue 8 years ago · 6 comments

Hi I'm working with Luwak 1.3.0 and I build the lucene document that I pass to luwak specifying that a particular field has omitNorms=true
When I look at the score explanation with explainingmatcher I see that fieldNorm for that field is not 1, as expected, but 0.375.
So lucene seems to be ignoring omitnorms.
Can you hint me on why this happens? And how to fix it?

Answer 1 · 2017-02-03T19:17:59.000Z

Hi, can you write a test case to show how you're using this?

Answer 2 · 2017-02-06T16:03:48.000Z

Hi Alan, what I see is that DocumentBatch$SingletonDocumentBatch calls memoryindex.freeze() which for every field callas MemoryIndex$Info.getNormDocValues() that (lazily) computes norms ignoring omitnorms seems an easy fix now that I understood how it works What I'm doing is trying to make luwak compute the same scores as solr so I created a custom similarity that reads idf values from the solr index. Did someone else do it? 2017-02-03 20:18 GMT+01:00 Alan Woodward <notifications@github.com>:

…

Hi, can you write a test case to show how you're using this? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#135 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABEJLLFIxm9MDAQR9Iy3RxJTmAzDpIF2ks5rY31ogaJpZM4L2nPc> .

Answer 3 · 2017-02-06T16:22:50.000Z

Ah, OK, so this sounds like it's a bug in MemoryIndex then. I'll open a lucene JIRA for it.

I'm afraid I don't know if anybody else has tried to build custom similarities in that way. Sounds like an interesting idea though

Answer 4 · 2017-02-06T16:24:40.000Z

Thanks for opening a jira I'll keep you posted then! 2017-02-06 17:22 GMT+01:00 Alan Woodward <notifications@github.com>:

…

Ah, OK, so this sounds like it's a bug in MemoryIndex then. I'll open a lucene JIRA for it. I'm afraid I don't know if anybody else has tried to build custom similarities in that way. Sounds like an interesting idea though — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#135 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABEJLPiWMtZWoGK4TQzsbVgIywuw2LYSks5rZ0jbgaJpZM4L2nPc> .

Answer 5 · 2017-02-21T14:44:45.000Z

This will be fixed by https://issues.apache.org/jira/browse/LUCENE-7679, in lucene 6.5

Answer 6 · 2017-04-04T08:15:47.000Z

Fixed by 1.5 release