Lookup service gives false negative - gives empty hits when there should be (v. 5.66-98.0)

Question

Lookup service gives false negative - gives empty hits when there should be (v. 5.66-98.0)

Closed this issue a month ago · 7 comments

Main issue

We saw that we got not hits using our local lookup service but did when we avoided it. We have become worried that we cannot trust all the results we've generated using the lookup service. Below is an example of a sequence we saw this with, but for a range of genomes we saw >50% proteins be left unannotated falsely and had to disable lookup service to get our annotations.

Version

We have relied on interproscan and lookup for version 5.66-98.0. (We are aware there is a new version and will also test that once we have it downloaded)

Sequence

We saw that we were missing annotations for the following sequence:

MKTKLTGVALLLAAVSLGSTQPVEACTRAVYIGPEQMVITGRTMDWKEDLHSNLYVFPRGIQRTGHNKEKTLNWTSKYGSIVATGYDIGTCDGMNEKGLVASLLFLPETIYSLPGDTRPVMGISIWTQYVLDNFATVREAVNELKKETFRIDAPRLPNGSESTLHLAITDETGNTAILEYLDGKLSIHEGKQYQVMTNSPRYEYQLAINDYWKEVGGLQMLPGTNRSSDRFVRASFYIHAIPQTSDAKIAVPSVLSVMRNVSVPFGITTPDKPYISSTRWRTVSDQKNKVYYFESTLTPNLFWLDLKKIDFSPKASIKKLSLANGEIYAGDAVKDLKDSKSFTFLFQTPVM

With the following MD5 350CBC1A5F0A7E36297C0497D69C197F

One can see there are multiple hits if I run this on your webserver: https://www.ebi.ac.uk/interpro/result/InterProScan/iprscan5-R20240423-140843-0558-6407259-p1m/
I get the same hits if I run interpro without a local lookup service.

Debug output:

Version

$ curl http://localhost:8085/version
SERVER:5.66-98.0

Test example from tutorial

$ curl http://localhost:8085/matches?md5=2E38C8D754C63117A4FA5F5E44F2194E
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<kvSequenceEntryXML>
    <matches>
        <match>
            <matchId>161415140</matchId>
            <proteinMD5>2E38C8D754C63117A4FA5F5E44F2194E</proteinMD5>
            <hit>GENE3D,4.3.0,G3DSA:1.20.58.900,2cxfA00,477,660,477-660-S,29.1,2.7E-6,[],6,168,174,477,660,27.8,7.0E-6,</hit>
            <hit>MOBIDB_LITE,2.0,mobidb-lite,mobidb-lite,814,862,814-862-S,0.0,0.0,,0,0,0,0,0,0.0,0.0,</hit>
            <hit>COILS,2.2.1,Coil,Coil,371,398,371-398-S,0.0,0.0,,0,0,0,0,0,0.0,0.0,</hit>
            <hit>PANTHER,18.0,PTHR43158,PTHR43158:SF2,20,243,20-243-S,146.9,2.2E-39,..,4,222,0,17,297,146.9,2.2E-39,AN6</hit>
            <hit>PROSITE_PROFILES,2023_05,PS50893,PS50893,25,254,25-254-S,9.250543,0.0,,0,0,0,0,0,0.0,0.0,9M7I54M6D41M8I11M8D26M11D74M</hit>
            <hit>SUPERFAMILY,1.75,SSF52540,0051025,17,239,17-239-S,0.0,5.93E-24,,0,0,284,0,0,0.0,5.93E-24,</hit>
            <hit>SMART,9.0,SM00382,SM00382,56,230,56-230-S,27.8,0.0015,[],1,92,92,0,0,27.8,0.0015,</hit>
            <hit>MOBIDB_LITE,2.0,mobidb-lite,mobidb-lite,674,727,674-727-S,0.0,0.0,,0,0,0,0,0,0.0,0.0,</hit>
            <hit>GENE3D,4.3.0,G3DSA:3.40.50.300,4huqA00,23,260,23-260-S,85.0,2.1E-23,[],3,233,290,23,260,83.5,6.2E-23,</hit>
            <hit>COILS,2.2.1,Coil,Coil,476,496,476-496-S,0.0,0.0,,0,0,0,0,0,0.0,0.0,</hit>
            <hit>PFAM,36.0,PF00005,PF00005,47,181,47-181-S,43.0,2.3E-7,[.,1,125,137,47,187,41.4,7.1E-7,</hit>
        </match>
    </matches>
</kvSequenceEntryXML>

The sequence in question

$ curl http://localhost:8085/matches?md5=350CBC1A5F0A7E36297C0497D69C197F
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<kvSequenceEntryXML>
    <matches/>
</kvSequenceEntryXML>

Main question

Is there a bug here?
If so, has it been addressed in the later versions?

Answer 1 · 2024-04-25T08:33:11.000Z

We've now downloaded the newest version and spun it up and it does indeed give results:

$ curl http://localhost:8085/version
SERVER:5.67-99.0
$ curl http://localhost:8085/matches?md5=350CBC1A5F0A7E36297C0497D69C197F
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<kvSequenceEntryXML>
    <matches>
        <match>
            <matchId>557607008</matchId>
            <proteinMD5>350CBC1A5F0A7E36297C0497D69C197F</proteinMD5>
            <hit>SUPERFAMILY,1.75,SSF56235,0045430,26,328,26-328-S,0.0,1.39E-95,,0,0,342,0,0,0.0,1.39E-95,</hit>
            <hit>PHOBIUS,1.01,NON_CYTOPLASMIC_DOMAIN,NON_CYTOPLASMIC_DOMAIN,26,351,26-351-S,0.0,0.0,,0,0,0,0,0,0.0,0.0,</hit>
            <hit>PANTHER,18.0,PTHR35527,PTHR35527:SF2,9,344,9-344-S,395.9,6.0E-115,..,6,346,0,1,348,395.9,6.0E-115,AN18</hit>
            <hit>PHOBIUS,1.01,SIGNAL_PEPTIDE_N_REGION,SIGNAL_PEPTIDE_N_REGION,1,4,1-4-S,0.0,0.0,,0,0,0,0,0,0.0,0.0,</hit>
            <hit>SIGNALP_EUK,4.1,SignalP-noTM,SignalP-noTM,1,25,1-25-S,0.765,0.0,,0,0,0,0,0,0.765,0.0,</hit>
            <hit>CDD,3.20,cd01902,cd01902,25,312,25-312-S,481.451,0.0,,0,0,0,0,0,481.451,0.0,</hit>
            <hit>SIGNALP_GRAM_POSITIVE,4.1,SignalP-TM,SignalP-TM,1,25,1-25-S,0.718,0.0,,0,0,0,0,0,0.718,0.0,</hit>
            <hit>PFAM,36.0,PF02275,PF02275,26,310,26-310-S,167.8,1.5E-45,[.,1,307,316,26,319,164.8,1.3E-44,</hit>
            <hit>PHOBIUS,1.01,SIGNAL_PEPTIDE_C_REGION,SIGNAL_PEPTIDE_C_REGION,18,25,18-25-S,0.0,0.0,,0,0,0,0,0,0.0,0.0,</hit>
            <hit>PHOBIUS,1.01,SIGNAL_PEPTIDE,SIGNAL_PEPTIDE,1,25,1-25-S,0.0,0.0,,0,0,0,0,0,0.0,0.0,</hit>
            <hit>SIGNALP_GRAM_NEGATIVE,4.1,SignalP-noTM,SignalP-noTM,1,25,1-25-S,0.719,0.0,,0,0,0,0,0,0.719,0.0,</hit>
            <hit>PHOBIUS,1.01,SIGNAL_PEPTIDE_H_REGION,SIGNAL_PEPTIDE_H_REGION,5,17,5-17-S,0.0,0.0,,0,0,0,0,0,0.0,0.0,</hit>
            <hit>GENE3D,4.3.0,G3DSA:3.60.60.10,3hbcA00,23,341,23-341-S,408.9,6.5E-122,[],2,318,320,23,341,408.7,7.4E-122,</hit>
        </match>
    </matches>
</kvSequenceEntryXML>

My main question then is whether there was an issue with the previous lookup service or if it somehow got "corrupted" at our end?

Answer 2 · 2024-05-01T08:00:45.000Z

Hi @cmunk,

Sorry for the late reply. I can reproduce the issue. The sequences is pre-calculated (and is tagged as such in the lookup service), yet no matches are returned.

Starting the service:

$ java -Xmx36000m -jar server-5.66-98.0-jetty-console.war --headless --port 8080

Checking the version (in another terminal):

$ curl 'http://localhost:8080/version'
SERVER:5.66-98.0

Checking whether the sequences has been pre-calculated:

$ curl 'http://localhost:8080/isPrecalculated?md5=350CBC1A5F0A7E36297C0497D69C197F'
350CBC1A5F0A7E36297C0497D69C197F

Getting pre-calculated matches:

$ curl 'http://localhost:8080/matches?md5=350CBC1A5F0A7E36297C0497D69C197F'
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<kvSequenceEntryXML>
    <matches/>
</kvSequenceEntryXML>

It was definitely not a corrupted file on your end, but something that went wrong on ours. I am very sorry about that.

Answer 3 · 2024-05-01T08:32:40.000Z

Thanks for your reply @matthiasblum -- is there a possibility to get a bit insights into the cause and whether we can be more confident that this will not happen in the future? We very much enjoy relying on the lookup service, but also need to be able to trust it.

Answer 4 · 2024-05-01T16:38:53.000Z

I asked @tgrego to look into it. He will update this issue.

Answer 5 · 2024-05-07T09:04:29.000Z

Hello @cmunk
Thank you for reporting this issue, we could confirm that not only version 5.66-98.0 but a few of our most recent versions were affected by it. The latest version 5.67-99.0 is not affected as far as I can tell.
The issue seems be related to newest uniparc sequences and affect a small number of proteins. The example sequence you provided is indeed a high uniparc ID.
To help us get a deeper insight of the issue could you provide us with a list of sequences/md5s that have been affected on your previous run? That would really help us if possible.

Thank you and best regards

Answer 6 · 2024-05-07T11:10:54.000Z

hi @tgrego -- Thank you for looking into this -- I sadly do not have the IDs of the false negatives as the data has now been overwritten without using lookup. But yes indeed, it was seemingly a smaller subset of data, though I did not deduce a pattern.

Answer 7 · 2024-08-19T09:34:52.000Z

Issue was missing data on the lookup service for a small but increasing number of high uniparc IDs, affecting versions starting 5.63-95.0 to 5.66-98.0.
Issue was fixed from version 5.64-96.0 onwards.