GSI Queries can fail with NPE when using an overriding DynamoDBMapperConfig

Question

GSI Queries can fail with NPE when using an overriding DynamoDBMapperConfig

kjl-dev opened this issue 5 years ago · 5 comments

Querying a Global Secondary Index will fail if using an overriding DynamoDBMapperConfig that doesnt specify a ConversionSchema and TypeConverterFactory.

Expected Behavior

Querying a Global Secondary Index will not fail if using an overriding DynamoDBMapperConfig that doesnt specify a ConversionSchema and TypeConverterFactory (perhaps by using reasonable defaults).

OR

An appropriate exception is thrown indicating an invalid configuration.

Actual Behavior

Querying a Global Secondary Index of a table when using a custom DynamoDBMapperConfig that does not specify a ConversionSchema and TypeConverterFactory fails with a NullPointerException. Searches on the regular table itself execute correctly.

Steps to Reproduce the Problem

Create the DynamoDB beans with a custom DynamoDBMapperConfig

  @Primary
  public DynamoDBMapperConfig dynamoDBMapperConfig() {

    String fullPrefix = tablePrefix + TABLE_DELIMITER + environment + TABLE_DELIMITER;
    TableNameOverride tableNameOverride = TableNameOverride.withTableNamePrefix(fullPrefix);

    return DynamoDBMapperConfig.builder()
        .withTableNameOverride(tableNameOverride)
        .build();
  }

  @Bean
  @Primary
  public DynamoDBMapper dynamoDBMapper(AmazonDynamoDB amazonDynamoDB, DynamoDBMapperConfig config,
      DynamoDBMapper mapper) {
    return new DynamoDBMapper(amazonDynamoDB, config);
  }

  @Bean
  public AmazonDynamoDB amazonDynamoDB() {
    return AmazonDynamoDBClientBuilder
        .standard()
        .withCredentials(amazonAWSCredentialsProvider())
        .withRegion(Regions.fromName(awsRegion))
        .build();
  }

Create the domain object that specifies the Global Secondary Index

@Data
@DynamoDBTable(tableName = "shopper")
public class Shopper{

  @DynamoDBHashKey(attributeName = "id")
  @DynamoDBAutoGeneratedKey
  private String id;

  @DynamoDBAttribute(attributeName = "address-hash" )
  @DynamoDBIndexHashKey(globalSecondaryIndexName = "idx-address-hash")
  private String addressHash;

  @DynamoDBTypeConvertedJson
  @DynamoDBAttribute(attributeName = "address")
  private Address address;

}

Create the Repository Bean with the custom find method

@Repository
@EnableScan
public interface ShopperRepository extends CrudRepository<Shopper, String> {
  
  Shopper findByAddressHash(String addressHash);
}

Invoke the method during runtime

Shopper result = repository.findByAddressHash(addressHash);

Observe the resulting stack trace

java.lang.NullPointerException: null
	at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:936) ~[na:1.8.0_66]
	at java.util.concurrent.ConcurrentHashMap.containsKey(ConcurrentHashMap.java:964) ~[na:1.8.0_66]
	at com.amazonaws.services.dynamodbv2.datamodeling.StandardModelFactories$StandardModelFactory.getTableFactory(StandardModelFactories.java:82) ~[aws-java-sdk-dynamodb-1.11.613.jar:na]
	at com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBMapper.getTableModel(DynamoDBMapper.java:410) ~[aws-java-sdk-dynamodb-1.11.613.jar:na]
	at org.socialsignin.spring.data.dynamodb.core.DynamoDBTemplate.getTableModel(DynamoDBTemplate.java:223) ~[spring-data-dynamodb-5.1.0.jar:5.1.0]
	at org.socialsignin.spring.data.dynamodb.repository.query.AbstractDynamoDBQueryCreator.create(AbstractDynamoDBQueryCreator.java:69) ~[spring-data-dynamodb-5.1.0.jar:5.1.0]
	at org.socialsignin.spring.data.dynamodb.repository.query.AbstractDynamoDBQueryCreator.create(AbstractDynamoDBQueryCreator.java:42) ~[spring-data-dynamodb-5.1.0.jar:5.1.0]
	at org.springframework.data.repository.query.parser.AbstractQueryCreator.createCriteria(AbstractQueryCreator.java:119) ~[spring-data-commons-2.1.10.RELEASE.jar:2.1.10.RELEASE]
	at org.springframework.data.repository.query.parser.AbstractQueryCreator.createQuery(AbstractQueryCreator.java:95) ~[spring-data-commons-2.1.10.RELEASE.jar:2.1.10.RELEASE]
	at org.springframework.data.repository.query.parser.AbstractQueryCreator.createQuery(AbstractQueryCreator.java:81) ~[spring-data-commons-2.1.10.RELEASE.jar:2.1.10.RELEASE]
	at org.socialsignin.spring.data.dynamodb.repository.query.PartTreeDynamoDBQuery.doCreateQuery(PartTreeDynamoDBQuery.java:56) ~[spring-data-dynamodb-5.1.0.jar:5.1.0]
	at org.socialsignin.spring.data.dynamodb.repository.query.AbstractDynamoDBQuery.doCreateQueryWithPermissions(AbstractDynamoDBQuery.java:81) ~[spring-data-dynamodb-5.1.0.jar:5.1.0]
	at org.socialsignin.spring.data.dynamodb.repository.query.AbstractDynamoDBQuery$SingleEntityExecution.execute(AbstractDynamoDBQuery.java:282) ~[spring-data-dynamodb-5.1.0.jar:5.1.0]
	at org.socialsignin.spring.data.dynamodb.repository.query.AbstractDynamoDBQuery.execute(AbstractDynamoDBQuery.java:311) ~[spring-data-dynamodb-5.1.0.jar:5.1.0]
	at org.springframework.data.repository.core.support.RepositoryFactorySupport$QueryExecutorMethodInterceptor.doInvoke(RepositoryFactorySupport.java:605) ~[spring-data-commons-2.1.10.RELEASE.jar:2.1.10.RELEASE]
	at org.springframework.data.repository.core.support.RepositoryFactorySupport$QueryExecutorMethodInterceptor.lambda$invoke$3(RepositoryFactorySupport.java:595) ~[spring-data-commons-2.1.10.RELEASE.jar:2.1.10.RELEASE]
	at org.springframework.data.repository.core.support.RepositoryFactorySupport$QueryExecutorMethodInterceptor.invoke(RepositoryFactorySupport.java:595) ~[spring-data-commons-2.1.10.RELEASE.jar:2.1.10.RELEASE]
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.1.9.RELEASE.jar:5.1.9.RELEASE]
	at org.springframework.data.projection.DefaultMethodInvokingMethodInterceptor.invoke(DefaultMethodInvokingMethodInterceptor.java:59) ~[spring-data-commons-2.1.10.RELEASE.jar:2.1.10.RELEASE]
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.1.9.RELEASE.jar:5.1.9.RELEASE]
	at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:93) ~[spring-aop-5.1.9.RELEASE.jar:5.1.9.RELEASE]
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.1.9.RELEASE.jar:5.1.9.RELEASE]
	at org.springframework.data.repository.core.support.SurroundingTransactionDetectorMethodInterceptor.invoke(SurroundingTransactionDetectorMethodInterceptor.java:61) ~[spring-data-commons-2.1.10.RELEASE.jar:2.1.10.RELEASE]
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.1.9.RELEASE.jar:5.1.9.RELEASE]
	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212) ~[spring-aop-5.1.9.RELEASE.jar:5.1.9.RELEASE]
	at com.sun.proxy.$Proxy78.findByAddressHash(Unknown Source) ~[na:na]
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_66]
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_66]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_66]
	at java.lang.reflect.Method.invoke(Method.java:497) ~[na:1.8.0_66]
	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343) ~[spring-aop-5.1.9.RELEASE.jar:5.1.9.RELEASE]
	at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198) ~[spring-aop-5.1.9.RELEASE.jar:5.1.9.RELEASE]
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) ~[spring-aop-5.1.9.RELEASE.jar:5.1.9.RELEASE]
	at org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:139) ~[spring-tx-5.1.9.RELEASE.jar:5.1.9.RELEASE]
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.1.9.RELEASE.jar:5.1.9.RELEASE]
	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212) ~[spring-aop-5.1.9.RELEASE.jar:5.1.9.RELEASE]
	at com.sun.proxy.$Proxy78.findByAddressHash(Unknown Source) ~[na:na]
...

Specifications

Spring Data DynamoDB Version: 5.1.0 (2.1)
Spring Data Version: 2.1.10.RELEASE
AWS SDK Version: 1.11.613
Java Version: 1.8.0_66 - Java HotSpot(TM) 64-Bit Server VM 25.66-b17
Platform Details: Mac OS X 10.14.5

Additional Information

I have enabled spring.main.allow-bean-definition-overriding
I was able to work around this issue by setting the ConversionSchema and TypeConverterFactory on my DynamoDBMapperConfig:

  @Bean
  @Primary
  public DynamoDBMapperConfig dynamoDBMapperConfig() {

    String fullPrefix = tablePrefix + TABLE_DELIMITER + environment + TABLE_DELIMITER;
    TableNameOverride tableNameOverride = TableNameOverride.withTableNamePrefix(fullPrefix);

    return DynamoDBMapperConfig.builder()
        .withTableNameOverride(tableNameOverride)
        .withConversionSchema(ConversionSchemas.V2)
        .withTypeConverterFactory(DynamoDBTypeConverterFactory.standard())
        .build();
  }

Answer 1 · 2019-10-25T09:06:18.000Z

I had a similar problem, adding .withTypeConverterFactory(DynamoDBTypeConverterFactory.standard()) seems to have solved the issue.

Answer 2 · 2020-01-22T11:09:59.000Z

@kjl-dev I tried both ways suggested above but still getting null pointer exception. Can you post your complete config?

Answer 3 · 2020-02-29T07:39:29.000Z

The error is because the table name is not getting overridden while querying. Meanwhile it is already taken care for other requests. So I was able to work around over that issue by registering a custom AWS Request handler.

   public RequestHandler2 requestHandler2() {
       return new RequestHandler2() {
           @Override
           public AmazonWebServiceRequest beforeExecution(AmazonWebServiceRequest request) {
               if (request instanceof QueryRequest)
                   ((QueryRequest) request).setTableName(tableName);
               return super.beforeExecution(request);
           }
       };
   }

Answer 4 · 2020-11-02T11:45:08.000Z

@kjl-dev I tried both ways suggested above but still getting null pointer exception. Can you post your complete config?

you also need to add
@EnableDynamoDBRepositories(dynamoDBMapperConfigRef = "dynamoDBMapperConfig") at AwsConfiguration class level.

Answer 5 · 2020-11-05T07:42:40.000Z

@CyberDracula had the same problem. Your hint was my solution :) Thank you!