[BUG] Serialization errors when attempting to build schema

Question

[BUG] Serialization errors when attempting to build schema

Closed this issue 3 years ago · 3 comments

Describe the bug

When connecting to an AWS Neptune instance through the SSH tunnel, serialization errors occur in shadow.org.apache.tinkerpop.shaded.kryo.serializers.FieldSerializer when calling databaseMetaData.getColumns(null, null, null, null) as per the example code.

This is without IAM Auth, over SSL.

The hosts file on my system has been modified to include 127.0.0.1 MY-ENDPOINT-URL.

[✅] Have you validated that this is not currently reported in
SQL Support and Limitations?

Steps to Reproduce

[❌] Is the JDBC Driver being used in a BI tool?

If being used outside of a BI Tool, fill in:

What vesion of the JDBC Driver are you using? 1.1.0
What language variant are you using (SPARQL, Gremlin, openCypher, SQL)? SQL
What database are you connecting to (Amazon Neptune, Neo4J, etc)? AWS Neptune
What engine version of the database are you using? Whatever is currently live on Neptune
What Java version are you using? OpenJDK 17
Please attach a code snippet of the code that is causing the error

package com.company;
import java.sql.*;
import java.util.Properties;

public class Main {
    public static void example() throws SQLException {
        String url = "jdbc:neptune:sqlgremlin://MY-SERVER-ENDPOINT.us-east-1.neptune.amazonaws.com;port=8182";

        final Properties properties = new Properties();
        properties.put("UseEncryption", "true");
        properties.put("LogLevel", "DEBUG");
        properties.put("AuthScheme", "None");

        Connection connection = DriverManager.getConnection(url, properties);
        Statement statement = connection.createStatement();
        DatabaseMetaData databaseMetaData = connection.getMetaData();

        final ResultSet getColumnsResultSet = databaseMetaData.getColumns(null, null, null, null);
        if (!getColumnsResultSet.next()) {
            throw new SQLException(
                    "This graph contains no columns (properties on any nodes with distinct label sets).");
        }

        do {
            // Grab the table name, column name, and data type of column. Look here for more information:
            // https://docs.oracle.com/javase/7/docs/api/java/sql/DatabaseMetaData.html#getColumns(java.lang.String,%20java.lang.String,%20java.lang.String,%20java.lang.String)
            final String tableName = getColumnsResultSet.getString("TABLE_NAME");
            final String columnName = getColumnsResultSet.getString("COLUMN_NAME");
            final String dataType = getColumnsResultSet.getString("DATA_TYPE");
            System.out.println(tableName + columnName + dataType);
        } while (getColumnsResultSet.next());


        connection.close();
    }

    public static void main(String[] args) {
	// write your code here
        try {
            example();
        } catch (Exception e) {
            e.printStackTrace();
        }

    }
}

If bug occurred during query execution attach the SQL Bug occurs when running above code
Attach debug logs if applicable/possible (please ensure it doesn't contain any sensitive information)

SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
java.util.concurrent.ExecutionException: java.util.concurrent.ExecutionException: shadow.io.netty.handler.codec.EncoderException: shadow.org.apache.tinkerpop.gremlin.driver.exception.ResponseException: An error occurred during serialization of this request [RequestMessage{, requestId=5e20fee2-8917-47ca-a216-db1e25edb9e0, op='bytecode', processor='traversal', args={gremlin=[[], [V(), label(), dedup()]], aliases={g=g}}}] - it could not be sent to the server - Reason: shadow.org.apache.tinkerpop.gremlin.driver.ser.SerializationException: java.lang.IllegalArgumentException: Unable to create serializer "shadow.org.apache.tinkerpop.shaded.kryo.serializers.FieldSerializer" for class: java.util.concurrent.atomic.AtomicLong
	at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122)
	at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:191)
	at shadow.org.twilmes.sql.gremlin.adapter.converter.schema.SqlSchemaGrabber.getSchema(SqlSchemaGrabber.java:65)
	at software.aws.neptune.common.gremlindatamodel.SchemaHelperGremlinDataModel.getGraphSchema(SchemaHelperGremlinDataModel.java:88)
	at software.aws.neptune.common.gremlindatamodel.MetadataCache.updateCache(MetadataCache.java:65)
	at software.aws.neptune.common.gremlindatamodel.MetadataCache.updateCacheIfNotUpdated(MetadataCache.java:80)
	at software.aws.neptune.gremlin.sql.SqlGremlinQueryExecutor.executeGetColumns(SqlGremlinQueryExecutor.java:117)
	at software.aws.neptune.jdbc.DatabaseMetaData.getColumns(DatabaseMetaData.java:953)
	at com.company.Main.example(Main.java:21)
	at com.company.Main.main(Main.java:43)
Caused by: java.util.concurrent.ExecutionException: shadow.io.netty.handler.codec.EncoderException: shadow.org.apache.tinkerpop.gremlin.driver.exception.ResponseException: An error occurred during serialization of this request [RequestMessage{, requestId=5e20fee2-8917-47ca-a216-db1e25edb9e0, op='bytecode', processor='traversal', args={gremlin=[[], [V(), label(), dedup()]], aliases={g=g}}}] - it could not be sent to the server - Reason: shadow.org.apache.tinkerpop.gremlin.driver.ser.SerializationException: java.lang.IllegalArgumentException: Unable to create serializer "shadow.org.apache.tinkerpop.shaded.kryo.serializers.FieldSerializer" for class: java.util.concurrent.atomic.AtomicLong
	at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122)
	at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:191)
	at shadow.org.twilmes.sql.gremlin.adapter.converter.schema.SqlSchemaGrabber$RunGremlinQueryVertices.call(SqlSchemaGrabber.java:145)
	at shadow.org.twilmes.sql.gremlin.adapter.converter.schema.SqlSchemaGrabber$RunGremlinQueryVertices.call(SqlSchemaGrabber.java:133)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: shadow.io.netty.handler.codec.EncoderException: shadow.org.apache.tinkerpop.gremlin.driver.exception.ResponseException: An error occurred during serialization of this request [RequestMessage{, requestId=5e20fee2-8917-47ca-a216-db1e25edb9e0, op='bytecode', processor='traversal', args={gremlin=[[], [V(), label(), dedup()]], aliases={g=g}}}] - it could not be sent to the server - Reason: shadow.org.apache.tinkerpop.gremlin.driver.ser.SerializationException: java.lang.IllegalArgumentException: Unable to create serializer "shadow.org.apache.tinkerpop.shaded.kryo.serializers.FieldSerializer" for class: java.util.concurrent.atomic.AtomicLong
	at shadow.io.netty.handler.codec.MessageToMessageEncoder.write(MessageToMessageEncoder.java:104)
	at shadow.io.netty.channel.AbstractChannelHandlerContext.invokeWrite0(AbstractChannelHandlerContext.java:717)
	at shadow.io.netty.channel.AbstractChannelHandlerContext.invokeWriteAndFlush(AbstractChannelHandlerContext.java:764)
	at shadow.io.netty.channel.AbstractChannelHandlerContext$WriteTask.run(AbstractChannelHandlerContext.java:1071)
	at shadow.io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
	at shadow.io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
	at shadow.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500)
	at shadow.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
	at shadow.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
	... 1 more
Caused by: shadow.org.apache.tinkerpop.gremlin.driver.exception.ResponseException: An error occurred during serialization of this request [RequestMessage{, requestId=5e20fee2-8917-47ca-a216-db1e25edb9e0, op='bytecode', processor='traversal', args={gremlin=[[], [V(), label(), dedup()]], aliases={g=g}}}] - it could not be sent to the server - Reason: shadow.org.apache.tinkerpop.gremlin.driver.ser.SerializationException: java.lang.IllegalArgumentException: Unable to create serializer "shadow.org.apache.tinkerpop.shaded.kryo.serializers.FieldSerializer" for class: java.util.concurrent.atomic.AtomicLong
	at shadow.org.apache.tinkerpop.gremlin.driver.handler.WebSocketGremlinRequestEncoder.encode(WebSocketGremlinRequestEncoder.java:60)
	at shadow.org.apache.tinkerpop.gremlin.driver.handler.WebSocketGremlinRequestEncoder.encode(WebSocketGremlinRequestEncoder.java:38)
	at shadow.io.netty.handler.codec.MessageToMessageEncoder.write(MessageToMessageEncoder.java:89)
	... 9 more
java.sql.SQLException: java.sql.SQLException: Error occurred during schema collection. 'java.util.concurrent.ExecutionException: shadow.io.netty.handler.codec.EncoderException: shadow.org.apache.tinkerpop.gremlin.driver.exception.ResponseException: An error occurred during serialization of this request [RequestMessage{, requestId=5e20fee2-8917-47ca-a216-db1e25edb9e0, op='bytecode', processor='traversal', args={gremlin=[[], [V(), label(), dedup()]], aliases={g=g}}}] - it could not be sent to the server - Reason: shadow.org.apache.tinkerpop.gremlin.driver.ser.SerializationException: java.lang.IllegalArgumentException: Unable to create serializer "shadow.org.apache.tinkerpop.shaded.kryo.serializers.FieldSerializer" for class: java.util.concurrent.atomic.AtomicLong'.
	at software.aws.neptune.jdbc.DatabaseMetaData.getColumns(DatabaseMetaData.java:961)
	at com.company.Main.example(Main.java:21)
	at com.company.Main.main(Main.java:43)
Caused by: java.sql.SQLException: Error occurred during schema collection. 'java.util.concurrent.ExecutionException: shadow.io.netty.handler.codec.EncoderException: shadow.org.apache.tinkerpop.gremlin.driver.exception.ResponseException: An error occurred during serialization of this request [RequestMessage{, requestId=5e20fee2-8917-47ca-a216-db1e25edb9e0, op='bytecode', processor='traversal', args={gremlin=[[], [V(), label(), dedup()]], aliases={g=g}}}] - it could not be sent to the server - Reason: shadow.org.apache.tinkerpop.gremlin.driver.ser.SerializationException: java.lang.IllegalArgumentException: Unable to create serializer "shadow.org.apache.tinkerpop.shaded.kryo.serializers.FieldSerializer" for class: java.util.concurrent.atomic.AtomicLong'.
	at shadow.org.twilmes.sql.gremlin.adapter.converter.schema.SqlSchemaGrabber.getSchema(SqlSchemaGrabber.java:71)
	at software.aws.neptune.common.gremlindatamodel.SchemaHelperGremlinDataModel.getGraphSchema(SchemaHelperGremlinDataModel.java:88)
	at software.aws.neptune.common.gremlindatamodel.MetadataCache.updateCache(MetadataCache.java:65)
	at software.aws.neptune.common.gremlindatamodel.MetadataCache.updateCacheIfNotUpdated(MetadataCache.java:80)
	at software.aws.neptune.gremlin.sql.SqlGremlinQueryExecutor.executeGetColumns(SqlGremlinQueryExecutor.java:117)
	at software.aws.neptune.jdbc.DatabaseMetaData.getColumns(DatabaseMetaData.java:953)
	... 2 more

Expected behavior

Expected to build the SQL schema necessary and list the columns

Screenshots

Environment

macOS Monterey, Apple Silicon, IntelliJ IDEA 2021.3.2 CE

Additional Context

This happens on both an empty and a populated graph.

Answer 1 · 2022-02-09T17:38:07.000Z

@jmlingeman Hey, this looks like the other issue we have reported. Both are boiling down to atomic long serialization. I am working on another project right now but this is on my radar and I plan to look at it as soon as I have a couple cycles.

Do you know if your database has AtomicLong's stored in it? It seems like the tinkerpop java driver does not support them.

Answer 2 · 2022-03-23T00:23:50.000Z

I looked into this in depth. This issue only exists when using OpenJDK 17 and TinkerPop 3.4.8.

So the solution seems to be to upgrade the JDBC Driver (and sql-gremlin) to 3.5.2.

I need to do proper testing before committing this change though, so I have made this ticket to track that since this exact issue was reported somewhere else as well.

See #159 going forward.

Answer 3 · 2022-03-25T19:21:36.000Z

A workaround would be for you to use a different JDK or alternatively you could likely inject a different serialization protocol into TinkerPop. Please let me know if you need assistance with the latter, as I can show you how it's done.