dbpedia/extraction-framework

Query doesn't return unless I manually paste it into my browser

dkylepierce opened this issue · 8 comments

Where did the problem occur (e.g. dbpedia.org/sparql, lookup, spotlight)?

dbpedia.org/sparql (see link at end)

Problem description

It seems that I can pass a URL+query string to your sparql endpoint and get a response, if and only if I am manually pasting it into my browser. If instead, I run a Windows app that does the HTTP request directly with a URL+query string, the query always fails. Maybe this is no longer permitted? For a long time, it used to work.

I wonder if there was a recent change (in the past few weeks) that was intended to defend against bots, or if there’s some maintenance issue. I’m not a bot but I want to be able to automate some queries that are run when the user interacts with the app.

Kyle state the nature of your technical emergency.
I have a Windows app that does the HTTP request directly with a URL+query string. The query always fails, even though it succeeds when I manually paste the URL+query string into my browser.

Expected behaviour

I want to pass a URL+query string to your sparql endpoint and get html results. I wrote a small Windows app that execs the default browser and passes the URL+query string as an argument to the command line.

Request/Reproduction

Give the link or request, so the problem can be reproduced. Ideally, this would be a unix curl command.

example URL+query:
http://dbpedia.org/sparql/?default-graph-uri=http%3A%2F%2Fdbpedia.org&query=PREFIX+dbo%3A+%3Chttp%3A%2F%2Fdbpedia.org%2Fontology%2F%3E%0D%0ASELECT+DISTINCT+%3Fbirth+%3Fperson+%3Fabstract++WHERE+%7B%0D%0A++++%3Fperson+dbo%3AbirthDate+%3Fbirth+.+++++%0D%0A%09%3Fperson+foaf%3Aname+%3Fname+.+++++%0D%0A%09%3Fperson+dbo%3Athumbnail+%3Fphoto+.%0D%0A%09%3Fperson+dbo%3Aabstract+%3Fabstract+.%0D%0A+++++FILTER+%28langMatches%28lang%28%3Fabstract%29%2C+%22EN%22%29%29+.%0D%0A%09+FILTER+%28%221861-02-11%22%5E%5Exsd%3Adate+%3C%3D+%3Fbirth+%26%26+%3Fbirth+%3C+%221861-02-17%22%5E%5Exsd%3Adate%29+.%0D%0A%7D++++%0D%0A%09+ORDER+BY+%3Fbirth&format=text%2Fhtml&CXML_redir_for_subjs=121&CXML_redir_for_hrefs=&timeout=30000&debug=on&run=+Run+Query+

I meant to make it clearer that the queries now failing are the same as the queries that used to work fine in my app that does the HTTP request directly with a query string. That stopped working a few weeks ago and since then, I can only paste the queries manually into my browser. I should note, this is the Chrome browser on Windows 10.

@dkylepierce,
thanks for reporting. HTTPS behaviour was changed. All HTTP requests are now upgraded to HTTPS:

curl -I http://dbpedia.org/sparql
HTTP/1.1 303 See Other
Server: nginx/1.18.0
Date: Wed, 10 Feb 2021 07:28:46 GMT
Content-Type: text/html
Content-Length: 153
Connection: keep-alive
Location: https://dbpedia.org/sparql

Your browser and most other apps can handle this by following the redirect. curl -L
Maybe you can switch the endpoint url from http://dbpedia.org/sparql to https://dbpedia.org/sparql. This should then work.

This one works fine:

curl  "https://dbpedia.org/sparql/?default-graph-uri=http%3A%2F%2Fdbpedia.org&query=PREFIX+dbo%3A+%3Chttp%3A%2F%2Fdbpedia.org%2Fontology%2F%3E%0D%0ASELECT+DISTINCT+%3Fbirth+%3Fperson+%3Fabstract++WHERE+%7B%0D%0A++++%3Fperson+dbo%3AbirthDate+%3Fbirth+.+++++%0D%0A%09%3Fperson+foaf%3Aname+%3Fname+.+++++%0D%0A%09%3Fperson+dbo%3Athumbnail+%3Fphoto+.%0D%0A%09%3Fperson+dbo%3Aabstract+%3Fabstract+.%0D%0A+++++FILTER+%28langMatches%28lang%28%3Fabstract%29%2C+%22EN%22%29%29+.%0D%0A%09+FILTER+%28%221861-02-11%22%5E%5Exsd%3Adate+%3C%3D+%3Fbirth+%26%26+%3Fbirth+%3C+%221861-02-17%22%5E%5Exsd%3Adate%29+.%0D%0A%7D++++%0D%0A%09+ORDER+BY+%3Fbirth&format=text%2Fhtml&CXML_redir_for_subjs=121&CXML_redir_for_hrefs=&timeout=30000&debug=on&run=+Run+Query+"

I am unable to help you further unless you give more debugging information, e.g. full request by yur app, including header or piece of code.

Hi @dkylepierce,
we need more information about the Windows App.

  1. if it is an existing app then please post the name
  2. if it is your own app, how do you create the HTTP request. Is it using HTTP1/1, GET ...?
    Otherwise we will close the issue and you should post it in the forum.dbpedia.org as a support question.