shilad/wikibrain

Error in SpatialDataLoader

tobyli opened this issue · 7 comments

tobyli commented

@shilad @bjhecht @aaroniidx

Have you guys encountered any problem loading spatial data with the latest version? I encountered the following problem:

INFO: Spatial database tables have already been initialized
Jun 04, 2014 3:02:43 PM org.wikibrain.spatial.loader.SpatialDataLoader main
INFO: Beginning to extract geographic information from Wikidata
Exception in thread "main" java.lang.AssertionError: jdbc:hsqldb:file:/Users/toby/wikibrain/./.tmp/Geotools/Databases/HSQL/v7.9.0.1/EPSG;shutdown=true
    at org.geotools.referencing.factory.epsg.ThreadedHsqlEpsgFactory.createDataSource(ThreadedHsqlEpsgFactory.java:237)
    at org.geotools.referencing.factory.epsg.ThreadedEpsgFactory.createBackingStore0(ThreadedEpsgFactory.java:386)
    at org.geotools.referencing.factory.epsg.ThreadedEpsgFactory.createBackingStore(ThreadedEpsgFactory.java:434)
    at org.geotools.referencing.factory.DeferredAuthorityFactory.getBackingStore(DeferredAuthorityFactory.java:133)
    at org.geotools.referencing.factory.BufferedAuthorityFactory.isAvailable(BufferedAuthorityFactory.java:235)
    at org.geotools.referencing.factory.DeferredAuthorityFactory.isAvailable(DeferredAuthorityFactory.java:119)
    at org.geotools.factory.FactoryRegistry.isAvailable(FactoryRegistry.java:667)
    at org.geotools.factory.FactoryRegistry.isAcceptable(FactoryRegistry.java:501)
    at org.geotools.factory.FactoryRegistry$1.filter(FactoryRegistry.java:192)
    at javax.imageio.spi.FilterIterator.advance(ServiceRegistry.java:810)
    at javax.imageio.spi.FilterIterator.<init>(ServiceRegistry.java:804)
    at javax.imageio.spi.ServiceRegistry.getServiceProviders(ServiceRegistry.java:508)
    at org.geotools.factory.FactoryRegistry.getServiceProviders(FactoryRegistry.java:197)
    at org.geotools.referencing.ReferencingFactoryFinder.getFactories(ReferencingFactoryFinder.java:180)
    at org.geotools.referencing.ReferencingFactoryFinder.getCRSAuthorityFactories(ReferencingFactoryFinder.java:455)
    at org.geotools.referencing.DefaultAuthorityFactory.getBackingFactory(DefaultAuthorityFactory.java:88)
    at org.geotools.referencing.DefaultAuthorityFactory.<init>(DefaultAuthorityFactory.java:69)
    at org.geotools.referencing.CRS.getAuthorityFactory(CRS.java:257)
    at org.geotools.referencing.CRS.decode(CRS.java:514)
    at org.geotools.referencing.CRS.decode(CRS.java:442)
    at org.geotools.jdbc.SQLDialect.createCRS(SQLDialect.java:614)
    at org.geotools.jdbc.JDBCFeatureSource.buildFeatureType(JDBCFeatureSource.java:309)
    at org.geotools.jdbc.JDBCDataStore.createFeatureSource(JDBCDataStore.java:862)
    at org.geotools.data.store.ContentDataStore.getFeatureSource(ContentDataStore.java:387)
    at org.geotools.data.store.ContentDataStore.getFeatureSource(ContentDataStore.java:352)
    at org.geotools.data.store.ContentDataStore.getSchema(ContentDataStore.java:336)
    at org.wikibrain.spatial.core.dao.postgis.PostGISDB.getSchema(PostGISDB.java:399)
    at org.wikibrain.spatial.core.dao.postgis.PostGISSpatialDataDao.beginSaveGeometries(PostGISSpatialDataDao.java:231)
    at org.wikibrain.spatial.loader.SpatialDataLoader.loadWikidataData(SpatialDataLoader.java:204)
    at org.wikibrain.spatial.loader.SpatialDataLoader.main(SpatialDataLoader.java:367)

Tried running single-thread, cleaning m2 and lib, rebuilding database in psql. Doesn't work.

Will be trying to revert to an earlier version to see if it works.

tobyli commented

Update:

It works perfectly when I run SpatialDataLoader from IntelliJ, but not with wb-java.sh, which is super weird.

I'll try to look into it :)

shilad commented

Are assertions enabled in your IntelliJ VM settings ("-ea")?

On Wed, Jun 4, 2014 at 3:22 PM, Toby Li notifications@github.com wrote:

Update:

It works perfectly when I run SpatialDataLoader from IntelliJ, but not
with wb-java.sh, which is super weird.

I'll try to look into it :)

—
Reply to this email directly or view it on GitHub
#190 (comment).

tobyli commented

ooops that's the reason...

Then that's weird...this assertion must have been failing for a long time, since it was still popping out when I reverted to a 1-month-old version.

But everything seems to be working well with this failing assertion. Have u seen a similar failure?

shilad commented

I haven't, but I haven't installed spatial lately :) Does looking at the
source code generating the assertion (in geotools) provide any clues?

On Wed, Jun 4, 2014 at 3:39 PM, Toby Li notifications@github.com wrote:

ooops that's the reason...

Then that's weird...this assertion must have been failing for a long time,
since it was still popping out when I reverted to a 1-month-old version.

But everything seems to be working well with this failing assertion. Have
u seen a similar failure?

—
Reply to this email directly or view it on GitHub
#190 (comment).

tobyli commented

I'll check.

  • Sent from my phone. Please excuse any fat thumbed typos and abbreviations.
    On Jun 4, 2014 3:56 PM, "Shilad Sen" notifications@github.com wrote:

I haven't, but I haven't installed spatial lately :) Does looking at the
source code generating the assertion (in geotools) provide any clues?

On Wed, Jun 4, 2014 at 3:39 PM, Toby Li notifications@github.com wrote:

ooops that's the reason...

Then that's weird...this assertion must have been failing for a long
time,
since it was still popping out when I reverted to a 1-month-old version.

But everything seems to be working well with this failing assertion.
Have
u seen a similar failure?

—
Reply to this email directly or view it on GitHub
#190 (comment).

—
Reply to this email directly or view it on GitHub
#190 (comment).

w1tebear commented

I have had this problem as well in my unit test runs where I run with assertions on. I have found it to be caused by cases where ThreadedHsqlEpsgFactory.getDirectory() (getTemporaryDirectory() actually) returns null due to there already being a Geotools directory in my java.io.tmpdir directory for which I have no permissions.

When this is the case, createDataSource() creates the directory object using null as the parent. The database is created in the current working directory. It works without the assertion (although the database is created in the cwd, NOT the tmp directory) but File.equals() does not show the paths equal even though they have the same absolute path. It can be tested with:

public static void main(String[] args) throws IOException {
String sep = System.getProperty("file.separator");
String cwd = System.getProperty("user.dir");
File f2 = new File((File)null, "somefile");
File f3 = new File(cwd + sep + "somefile");
System.out.println(String.format("f2 (%s:%s) == f3 (%s:%s): %b",
f2.toString(), f2.getAbsolutePath(), f3.toString(), f3.getAbsolutePath(), f2.equals(f3)));
assert f2.equals(f3) : "assertion failed";
}

Maybe you are seeing the same thing?

shilad commented

Closing because it's stale.