roharon/spring-batch-redis

Spring Batch extension for Redis

Spring Batch Redis

ItemReader and ItemWriter implementations for Redis.

Getting Started

Maven

Add Spring Batch Redis dependency to your POM file:

pom.xml

<dependency>
    <groupId>com.redis</groupId>
    <artifactId>spring-batch-redis-core</artifactId>
    <version>4.3.2</version>
</dependency>

Gradle

Add Spring Batch Redis dependency to your build.gradle file

build.gradle

dependencies {
    implementation 'com.redis:spring-batch-redis-core:4.3.2'
}

Data Types

RedisItemReader<K,V,T> and RedisItemWriter<K,V,T> support a single data type called KeyValue that has the following fields: * key of type K representing the Redis key. * type of type String representing the type of that Redis key (string, hash, list, …​). * ttl of type long representing the key’s absolute expiration epoch time in milliseconds. * value of type: byte[] for dump reader. Object for struct reader. The type of the object depends on the type of the Redis key (V for string, Map<K,V> for hash, List<V> for list, …​). See Data Structures below for more details. * mem of type long representing the memory usage of that key in Redis. This is only populated when memUsageLimit on MemKeyValueRead operation is strictly greater than 0.

Data Structures

Values for struct readers RedisItemReader<K,V,T> are Java object representations of the underlying Redis data-structures:

• hash: java.util.Map<K,V>

• list: java.util.List<V>

• set: java.util.Set<V>

• stream: java.util.List<io.lettuce.core.StreamMessage<K,V>>

• string: V

• zset: java.util.Set<io.lettuce.core.ScoredValue<V>>

• json: V

• timeseries: java.util.List<com.redis.lettucemod.timeseries.Sample>>

These generic classes <K,V> depend on the RedisCodec used to initialize the reader: * StringCodec: java.lang.String * ByteArrayCodec: byte[]

Item Readers

RedisItemReader exposes 2 operation modes:

• Snapshot: relies on SCAN command to iterate over the keys whose values will be read to produce key/values.

• Live (AKA continuous): listens to changes in the keyspace via notifications and produces corresponding key/values.

Item Writers

RedisItemWriter can perform both inserts or deletes depending on the value and TTL in the incoming object. If value is null or TTL is -2 then the DEL command is called, otherwise a write is performed.

Item writers support two different payload (value) types:

Key Dump

The writer accepts key dumps (KeyValue with a byte[] value) and calls the RESTORE command with the byte array and TTL if any.

Data Structure

The writer takes KeyValue objects and calls the write command specific to the data type:

• hash: HSET

• json: JSON.SET

• list: RPUSH

• set: SADD

• stream: XADD

• string: SET

• zset: ZADD

• timeseries: TS.ADD

If TTL >= 0 then an additional call is made to the EXPIREAT command.

Usage

Refer to unit tests for usage examples.