Wimmics/corese

Add configuration option equivalent to having "@skip kg:select" annotation in all queries

tokarenko opened this issue · 4 comments

tokarenko commented

Issue Description:

Source selection with bind is not disabled by 'FEDERATE_JOIN = false' in corese-properties.properties. It worked before as advised in #152. We are unable to get it working in new environment.

Bug Details:

Federated query is issued with BIND and EXISTS: select * \nwhere {\n bind (exists {select * \n where {\n http://test.domain/device.owl#core1_Gi0/1_ipv4_address ?p ?o .\n }\n limit 1 } as ?b0)\n}\nlimit 1000

Steps to Reproduce:

  1. Run Corese in docker with mounted corese-properties.properties containing FEDERATE_JOIN = false:
    docker run -d --rm --name CORESE -p 8095:8080 -v /home/kyzauver/corese/config:/usr/local/corese/config -v /home/kyzauver/corese/data:/usr/local/corese/data -v /home/kyzauver/corese/log:/usr/local/corese/log wimmics/corese
  2. Run federated query:
    SELECT * WHERE {
    http://test.domain/device.owl#core1_Gi0/1_ipv4_address ?p ?o
    }

Expected Behavior:

Query is issued without BIND and EXISTS.

Actual Behavior:

Query is issued with BIND and EXISTS:

2024-04-23 06:45:05 INFO webservice.SPARQLRestAPI.getTriplesXMLForGet:311 [] - getTriplesXMLForGet
2024-04-23 06:45:05 INFO webservice.SPARQLResult.getResultFormat:90 [] - Endpoint URL: http://10.112.17.134:8095/myfederation/federate
2024-04-23 06:45:05 INFO webservice.SPARQLResult.getResultFormat:96 [] - Query: SELECT * WHERE {
http://test.domain/device.owl#core1_Gi0/1_ipv4_address ?p ?o
}

2024-04-23 06:45:05 INFO webservice.EventManager.log:72 [] - Workflow Context:
federate : true
st:remoteHost : "10.112.31.71"
st:service : "http://ns.inria.fr/sparql-template/user"
mode : "("federate" )"^^dt:list
operation : "federate"
request : "[org.eclipse.jetty.server.Request:Request(GET http://10.112.17.134:8095/myfederation/federate?query=SELECT%20*%20WHERE%20%7B%0A%20%20%3Chttp%3A%2F%2Ftest.domain%2Fdevice.owl%23core1_Gi0%2F1_ipv4_address%3E%20%3Fp%20%3Fo%0A%7D%0A)@378a0b87]"^^dt:pointer
url : http://10.112.17.134:8095/myfederation/federate
user query: true
level: PRIVATE

2024-04-23 06:45:05 INFO webservice.EventManager.log:73 [] - {st:count="[Map: size=1]"^^dt:map, st:date="[Map: size=1]"^^dt:map, st:host="[Map: size=1]"^^dt:map, st:hostlite="[Map: size=1]"^^dt:map}
2024-04-23 06:45:05 INFO webservice.EventManager.log:74 [] - {"http://ns.inria.fr/sparql-template/user"=2}
2024-04-23 06:45:05 INFO webservice.EventManager.log:76 [] - {"10.112.31.71"=2}
2024-04-23 06:45:05 INFO webservice.TripleStore.federate:396 [] - metadata: Metadata:
@federation : [http://10.112.17.134:8095/myfederation/federate]

2024-04-23 06:45:05 INFO federate.FederateVisitor.option:447 [] - Metadata:
@federation : [http://10.112.17.134:8095/myfederation/federate]

2024-04-23 06:45:05 INFO query.ProviderService.getTimeout:919 [] - Timeout: 200000
2024-04-23 06:45:05 INFO load.Service.basicPost:253 [] - Post http://10.112.17.134:8083
2024-04-23 06:45:05 INFO load.Service.basicPost:253 [] - Post http://10.112.17.134:8225
2024-04-23 06:45:05 INFO load.Service.basicPost:286 [] - Response status: 400
2024-04-23 06:45:05 ERROR load.Service.basicPost:318 [] - http://10.112.17.134:8225
2024-04-23 06:45:05 ERROR load.Service.basicPost:319 [] - jakarta.ws.rs.client.ResponseProcessingException {
"exception": "Invalid SPARQL query: Built-in function "exists{selectwhere{http://test.domain/device.owl#core1_gi0/1_ipv4_address?p?o.}limit1}" not yet implemented; if you need it, just add it to SparqlQleverVisitor.cpp::visitTypesafe(Parser::BuiltInCallContext following the already implemented functions there",
"metadata": {
"line": 3,
"positionInLine": 8,
"query": "select * \nwhere {\n bind (exists {select * \n where {\n http://test.domain/device.owl#core1_Gi0/1_ipv4_address ?p ?o .\n }\n limit 1 } as ?b0)\n}\nlimit 1000 ",
"startIndex": 26,
"stopIndex": 140
},
"query": "select * \nwhere {\n bind (exists {select * \n where {\n http://test.domain/device.owl#core1_Gi0/1_ipv4_address ?p ?o .\n }\n limit 1 } as ?b0)\n}\nlimit 1000 ",
"resultsize": 0,
"status": "ERROR",
"time": {
"computeResult": 10,
"total": 10
}
}
2024-04-23 06:45:05 ERROR query.ProviderService.exception:507 [] - jakarta.ws.rs.client.ResponseProcessingException http://10.112.17.134:8225
2024-04-23 06:45:05 ERROR query.ProviderService.error:596 [] - service error: http://10.112.17.134:8225
2024-04-23 06:45:05 ERROR query.ProviderService.error:600 [] - {
"exception": "Invalid SPARQL query: Built-in function "exists{selectwhere{http://test.domain/device.owl#core1_gi0/1_ipv4_address?p?o.}limit1}" not yet implemented; if you need it, just add it to SparqlQleverVisitor.cpp::visitTypesafe(Parser::BuiltInCallContext following the already implemented functions there",
"metadata": {
"line": 3,
"positionInLine": 8,
"query": "select * \nwhere {\n bind (exists {select * \n where {\n http://test.domain/device.owl#core1_Gi0/1_ipv4_address ?p ?o .\n }\n limit 1 } as ?b0)\n}\nlimit 1000 ",
"startIndex": 26,
"stopIndex": 140
},
"query": "select * \nwhere {\n bind (exists {select * \n where {\n http://test.domain/device.owl#core1_Gi0/1_ipv4_address ?p ?o .\n }\n limit 1 } as ?b0)\n}\nlimit 1000 ",
"resultsize": 0,
"status": "ERROR",
"time": {
"computeResult": 10,
"total": 10
}
}
2024-04-23 06:45:05 ERROR query.ProviderService.error:602 [] - select *
where {
bind (exists {select *
where {
http://test.domain/device.owl#core1_Gi0/1_ipv4_address ?p ?o .
}
limit 1 } as ?b0)
}
limit 1000
2024-04-23 06:45:05 INFO load.Service.basicPost:286 [] - Response status: 200
2024-04-23 06:45:05 ERROR datatype.DatatypeMap.createLiteral:667 [] - Datatype error: http://www.w3.org/2001/XMLSchema#boolean
2024-04-23 06:45:05 INFO federate.Selector.process11:243 [] - Source Selection Join Test Success: true
2024-04-23 06:45:05 ERROR federate.FederateVisitor.error:977 [] - rewrite triple
2024-04-23 06:45:05 ERROR federate.FederateVisitor.error:982 [] - Undefined triple: http://test.domain/device.owl#core1_Gi0/1_ipv4_address ?p ?o .
2024-04-23 06:45:05 ERROR federate.FederateVisitor.error:316 [] - Query rewrite fail due to undefined triple

corese-properties.properties seem to load fine and contains 'FEDERATE_JOIN = false' (checked the file inside the container).

2024-04-23 06:44:08 INFO webservice.EmbeddedJettyServer.main:115 [] - Loading log4j configuration: file:/usr/local/corese/config/log4j2.xml
2024-04-23 06:44:08 INFO webservice.EmbeddedJettyServer.main:116 [] - To override log4j configuration add JVM option: -Dlog4j.configurationFile=file:/home/.../your_log4j2.xml
2024-04-23 06:44:08 INFO core.Corese.init:14 [] - Corese, Inria: Tue Apr 23 06:44:08 UTC 2024
2024-04-23 06:44:08 INFO webservice.EmbeddedJettyServer.main:231 [] - Profile: file:///usr/local/corese/config/corese-profile.ttl
2024-04-23 06:44:08 INFO webservice.EmbeddedJettyServer.main:282 [] - init = /usr/local/corese/config/corese-properties.properties
2024-04-23 06:44:08 INFO util.Property.defineFederation:967 [] - federation: /usr/local/corese/data/federation.ttl
federation: http://10.112.17.134:8095/myfederation/federate : "(http://10.112.17.134:8083/ http://10.112.17.134:8225/ )"^^dt:list
access: http://10.112.17.134:8083/
access: http://10.112.17.134:8225/

ocorby commented
tokarenko commented

Thank you, Olivier, for your quick reply! @Skip kg:select does not work as expected. Query still contains bind (exists {} as var). I would be grateful if you could resolve the issue.

2024-04-24 07:57:05 INFO  webservice.SPARQLRestAPI.getTriplesXMLForGet:311 [] - getTriplesXMLForGet
2024-04-24 07:57:05 INFO  webservice.SPARQLResult.getResultFormat:90 [] - Endpoint URL: http://10.112.17.134:8095/myfederation/federate
2024-04-24 07:57:05 INFO  webservice.SPARQLResult.getResultFormat:96 [] - Query: prefix onto: <http://test.domain/device.owl#> 
@Skip kg:select
SELECT ?ip WHERE {
  <http://test.domain/device.owl#core1_Gi0/1_ipv4_address> onto:IP ?ip
}
2024-04-24 07:57:05 INFO  webservice.EventManager.log:72 [] - Workflow Context:
federate : true
st:remoteHost : "10.112.31.71"
st:service : "http://ns.inria.fr/sparql-template/user"
mode : "("federate" )"^^dt:list
operation : "federate"
request : "[org.eclipse.jetty.server.Request:Request(GET [http://10.112.17.134:8095/myfederation/federate?query=prefix%20onto%3A%20%3Chttp%3A%2F%2Ftest.domain%2Fdevice.owl%23%3E%20%0A%40Skip%20kg%3Aselect%0ASELECT%20%3Fip%20WHERE%20%7B%0A%20%20%3Chttp%3A%2F%2Ftest.domain%2Fdevice.owl%23core1_Gi0%2F1_ipv4_address%3E%20onto%3AIP%20%3Fip%0A%7D)@44b1454]"^^dt:pointer](http://10.112.17.134:8095/myfederation/federate?query=prefix%20onto%3A%20%3Chttp%3A%2F%2Ftest.domain%2Fdevice.owl%23%3E%20%0A%40Skip%20kg%3Aselect%0ASELECT%20%3Fip%20WHERE%20%7B%0A%20%20%3Chttp%3A%2F%2Ftest.domain%2Fdevice.owl%23core1_Gi0%2F1_ipv4_address%3E%20onto%3AIP%20%3Fip%0A%7D)@44b1454%5d%22%5e%5edt:pointer)
url : <http://10.112.17.134:8095/myfederation/federate>
user query: true
level: PRIVATE

2024-04-24 07:57:05 INFO  webservice.EventManager.log:73 [] - {st:count="[Map: size=1]"^^dt:map, st:date="[Map: size=1]"^^dt:map, st:host="[Map: size=1]"^^dt:map, st:hostlite="[Map: size=1]"^^dt:map}
2024-04-24 07:57:05 INFO  webservice.EventManager.log:74 [] - {"http://ns.inria.fr/sparql-template/user"=3}
2024-04-24 07:57:05 INFO  webservice.EventManager.log:76 [] - {"10.112.31.71"=3}
2024-04-24 07:57:05 INFO  webservice.TripleStore.federate:396 [] - metadata: Metadata:
@Skip : [http://ns.inria.fr/corese/kgram/select]
@federation : [http://10.112.17.134:8095/myfederation/federate]

2024-04-24 07:57:05 INFO  federate.FederateVisitor.option:447 [] - Metadata:
@Skip : [http://ns.inria.fr/corese/kgram/select]
@federation : [http://10.112.17.134:8095/myfederation/federate]

2024-04-24 07:57:05 INFO  query.ProviderService.getTimeout:919 [] - Timeout: 200000
2024-04-24 07:57:05 INFO  load.Service.basicPost:253 [] - Post http://10.112.17.134:8224/
2024-04-24 07:57:05 INFO  load.Service.basicPost:286 [] - Response status: 400
2024-04-24 07:57:05 ERROR load.Service.basicPost:318 [] - http://10.112.17.134:8224/
2024-04-24 07:57:05 ERROR load.Service.basicPost:319 [] - jakarta.ws.rs.client.ResponseProcessingException {
    "exception": "Invalid SPARQL query: Built-in function \"exists{select*where{?sonto:ip?o.}limit1}\"  not yet implemented; if you need it, just add it to SparqlQleverVisitor.cpp::visitTypesafe(Parser::BuiltInCallContext following the already implemented functions there",
    "metadata": {
        "line": 5,
        "positionInLine": 8,
        "query": "prefix kg: <http://ns.inria.fr/corese/kgram/>\nprefix onto: <http://test.domain/device.owl#>\nselect * \nwhere {\n  bind (exists {select * \n    where {\n      ?s onto:IP ?o .\n    }\n    limit 1 } as ?b0)\n  bind (exists {select * \n    where {\n      <http://test.domain/device.owl#core1_Gi0/1_ipv4_address> onto:IP ?ip .\n    }\n    limit 1 } as ?b1)\n}\nlimit 1000 ",
        "startIndex": 113,
        "stopIndex": 183
    },
    "query": "prefix kg: <http://ns.inria.fr/corese/kgram/>\nprefix onto: <http://test.domain/device.owl#>\nselect * \nwhere {\n  bind (exists {select * \n    where {\n      ?s onto:IP ?o .\n    }\n    limit 1 } as ?b0)\n  bind (exists {select * \n    where {\n      <http://test.domain/device.owl#core1_Gi0/1_ipv4_address> onto:IP ?ip .\n    }\n    limit 1 } as ?b1)\n}\nlimit 1000 ",
    "resultsize": 0,
    "status": "ERROR",
    "time": {
        "computeResult": 0,
        "total": 0
    }
}
2024-04-24 07:57:05 ERROR query.ProviderService.exception:507 [] - jakarta.ws.rs.client.ResponseProcessingException http://10.112.17.134:8224/
2024-04-24 07:57:05 ERROR query.ProviderService.error:596 [] - service error: http://10.112.17.134:8224/
2024-04-24 07:57:05 ERROR query.ProviderService.error:598 [] - {
    "exception": "Invalid SPARQL query: Built-in function \"exists{select*where{?sonto:ip?o.}limit1}\"  not yet implemented; if you need it, just add it to SparqlQleverVisitor.cpp::visitTypesafe(Parser::BuiltInCallContext following the already implemented functions there",
    "metadata": {
        "line": 5,
        "positionInLine": 8,
        "query": "prefix kg: <http://ns.inria.fr/corese/kgram/>\nprefix onto: <http://test.domain/device.owl#>\nselect * \nwhere {\n  bind (exists {select * \n    where {\n      ?s onto:IP ?o .\n    }\n    limit 1 } as ?b0)\n  bind (exists {select * \n    where {\n      <http://test.domain/device.owl#core1_Gi0/1_ipv4_address> onto:IP ?ip .\n    }\n    limit 1 } as ?b1)\n}\nlimit 1000 ",
        "startIndex": 113,
        "stopIndex": 183
    },
    "query": "prefix kg: <http://ns.inria.fr/corese/kgram/>\nprefix onto: <http://test.domain/device.owl#>\nselect * \nwhere {\n  bind (exists {select * \n    where {\n      ?s onto:IP ?o .\n    }\n    limit 1 } as ?b0)\n  b
2024-04-24 07:57:05 ERROR query.ProviderService.error:602 [] - prefix kg: <http://ns.inria.fr/corese/kgram/>
prefix onto: <http://test.domain/device.owl#>
select * 
where {
  bind (exists {select * 
    where {
      ?s onto:IP ?o .
    }
    limit 1 } as ?b0)
  bind (exists {select * 
    where {
      <http://test.domain/device.owl#core1_Gi0/1_ipv4_address> onto:IP ?ip .
    }
    limit 1 } as ?b1)
}
limit 1000 
2024-04-24 07:57:05 INFO  federate.Selector.process11:243 [] - Source Selection Join Test Success: true
2024-04-24 07:57:05 ERROR federate.FederateVisitor.error:977 [] - rewrite triple
2024-04-24 07:57:05 ERROR federate.FederateVisitor.error:982 [] - Undefined triple: <http://test.domain/device.owl#core1_Gi0/1_ipv4_address> onto:IP ?ip .
2024-04-24 07:57:05 ERROR federate.FederateVisitor.error:316 [] - Query rewrite fail due to undefined triple
ocorby commented
tokarenko commented

It works as expected with lower case @Skip kg:select ! Thank you. I should have tried it before posting. I am sorry for this.

I would be grateful if you could add corresponding setting into corese-properties.properties. I would be nice to have this option set by default for all queries.