shilad/wikibrain

Updated Wikidata Toolkit

Yuropa opened this issue · 2 comments

Yuropa commented

I have tried a few times, and the loader seems unable to create the Wikidata database. Initially, the loader was unable to download the Wikidata dump file. Stepping through the debugger, it looks like the WmfDumpFileManager is unable to find any dump files.

To create a temporary fix, I downloaded the dump file manually and then the Wikidata loader was able to proceed. However, there seems to still be a bug since the following error keeps showing up in the debugger:

Method threw 'java.lang.RuntimeException' exception. Cannot evaluate org.wikidata.wdtk.datamodel.json.jackson.JacksonItemDocument.toString()

After some Googling, it seems like the cause of both these problems is the Wikidata Toolkit. I looked at the code for the WmfDumpFileManager and the latest version handles the new dumps format. The updated version should fix the loader so that dump files can be downloaded. It may also fix the processing error.

shilad commented

Thanks for your message. I was literally fixing this bug when you sent
this. I've published an updated, forked version of WikidataToolkit that
addresses this bug. I'll have a new Wikibrain release that links to it
momentarily.

On Sun, Feb 1, 2015 at 9:06 PM, Yuropa notifications@github.com wrote:

I have tried a few times, and the loader seems unable to create the
Wikidata database. Initially, the loader was unable to download the
Wikidata dump file. Stepping through the debugger, it looks like the
WmfDumpFileManager is unable to find any dump files.

To create a temporary fix, I downloaded the dump file manually and then
the Wikidata loader was able to proceed. However, there seems to still be a
bug since the following error keeps showing up in the debugger:

Method threw 'java.lang.RuntimeException' exception. Cannot evaluate org.wikidata.wdtk.datamodel.json.jackson.JacksonItemDocument.toString()

After some Googling, it seems like the cause of both these problems is the
Wikidata Toolkit. I looked at the code for the WmfDumpFileManager and the
latest version handles the new dumps format. The updated version should fix
the loader so that dump files can be downloaded. It may also fix the
processing error.

—
Reply to this email directly or view it on GitHub
#221.

shilad commented

@Yuropa: I just pushed release 0.5.2, which depends on a new custom build of Wikidata Toolkit. I've run some basic tests (and unit tests), and they seem to work, but please let me know if you notice anything odd.

Thanks for using WikiBrain!