/redis-smart-cache

Transparent SQL caching proxy for Redis

Primary LanguageJavaMIT LicenseMIT

Redis Smart Cache

Redis Smart Cache is a JDBC query cache for Redis Stack, Redis Cloud, and Redis Enterprise.


Build Status Download Coverage

Redis Smart Cache lets you add caching to your application without changing the code.

Implemented as a wrapper around your backend database’s JDBC driver, Redis Smart Cache can cache slow, repeated queries from Redis, bypassing expensive database calls and greatly improving response times.

Table of Contents

Background

Redis is an in-memory data store designed to serve data with the fastest possible response times. For this reason, Redis is frequently used for caching.

It’s not always easy to modify your code to utilize Redis as a cache. You need to think about a number of issues, including:

  • Result set serialization and deserialization

  • Fault-tolerance

  • Cache invalidation

This is why we built Redis Smart Cache. After adding a single dependency and setting some basic configuration, your JDBC application can take advantage of Redis as a cache without any changes to your code.

Quick start

To understand how Redis Smart Cache works, it’s best to try it for yourself.

This example showcases an application that continuously performs queries against a MySQL database and uses Redis Smart Cache to cache query results.

First, clone this git repository:

git clone https://github.com/redis-field-engineering/redis-smart-cache.git
cd redis-smart-cache

Next, use Docker Compose to launch containers for MySQL, Grafana, Redis Stack, and Redis Smart Cache example app instance:

docker-compose up

Once all of these containers have started, log in to Grafana by pointing your browser to http://localhost:3000. Use the following credentials:

Username: admin Password: admin

In the Grafana UI, click Add your first datasource.

In the filter box, enter Redis and click on the first entry (Redis data source).

In the wizard that shows up, enter redis://redis:6379 for the Redis address and click Save & test.

Redis Datasource

Now import the demo dashboard by clicking Dashboards/Import in the sidebar.

Import Dashboard

Next, click Upload JSON file and upload redis-smart-cache/demo/redis-smart-cache-demo/grafana/dashboard.json.

At the bottom of the page, select the Redis datasource that you just created and click Import.

Import Dashboard Data Source

You should see the following dashboard:

Dashboard

After a few minutes, the Redis cache will be populated, yielding dramatically improved response times.

Installation

To use Redis Smart Cache with an existing application, you’ll need to add the Redis Smart Cache JDBC driver as an application dependency.

Maven
<dependency>
    <groupId>com.redis</groupId>
    <artifactId>redis-smart-cache-jdbc</artifactId>
    <version>0.2.1</version>
</dependency>
Gradle
dependencies {
    implementation 'com.redis:redis-smart-cache-jdbc:0.2.1'
}

The next step is to configure Redis Smart Cache, as described below.

Usage

First, ensure that your application is using Redis Smart Cache as its JDBC driver:

com.redis.smartcache.Driver

Next, set your JDBC URI to the URI of your Redis instance prefixed by jdbc: for example:

jdbc:redis://cache.redis.cloud:6379

See Lettuce’s URI syntax for all of the possible URI parameters you can use here.

Next step is providing bootstrap configuration.

Bootstrap Configuration

Bootstrap configuration contains the information necessary to connect to Redis and the backend database and is specified using JDBC properties.

Property value types

Redis Smart Cache JDBC properties support different value types.

boolean

The properties of type boolean support two values, true or false.

data size

The properties of type data size support values that describe an amount of data, measured in byte-based units. These units are incremented in multiples of 1024, so one megabyte is 1024 kilobytes, one kilobyte is 1024 bytes, and so on. For example, the value 6MB describes six megabytes.

The data size type supports the following units:

  • B: Bytes

  • kB: Kilobytes

  • MB: Megabytes

  • GB: Gigabytes

double

The properties of type double support numerical values including decimals, such as 1.6. Double type values can be negative, if supported by the specific property.

duration

The properties of type duration support values describing an amount of time, using the syntax of a non-negative number followed by a time unit. For example, the value 7m describes seven minutes.

The duration type supports the following units:

  • ns: Nanoseconds

  • us: Microseconds

  • ms: Milliseconds

  • s: Seconds

  • m: Minutes

  • h: Hours

  • d: Days

A duration of 0 is treated as zero regardless of the unit that follows. For example, 0s and 0m both mean the same thing.

Properties of type duration also support decimal values, such as 2.25d. These are handled as a fractional value of the specified unit. For example, the value 1.5m equals one and a half minutes, or 90 seconds.

integer

The properties of type integer support whole numeric values, such as 5 and 1000. Negative values are supported as well, for example -7. Integer type values must be whole numbers, decimal values such as 2.5 are not supported.

Some integer type properties enforce their own minimum and maximum values.

string

The properties of type string support a set of values that consist of a sequence of characters. Allowed values are defined on a property-by-property basis, refer to the specific property for its supported and default values.

Backend database

smartcache.driver.class-name
  • Type: string

  • Required

Class name of the backend database JDBC driver, for example oracle.jdbc.OracleDriver.

smartcache.driver.url
  • Type: string

  • Required

JDBC URL for the backend database, for example jdbc:oracle:thin:@myhost:1521:orcl.

Additional properties

You can also include any property your backend JDBC driver requires, like username or password. These will be passed to the backend JDBC driver as is.

Redis

To further configure how Redis Smart Cache connects to Redis, set the following properties:

smartcache.redis.cluster
  • Type: boolean

  • Default value: false

Connect to a Redis Cluster.

smartcache.redis.tls
  • Type: boolean

  • Default value: false

Establish a secure TLS connection.

smartcache.redis.tls-verify
  • Type: string

  • Allowed values: NONE, CA, FULL

  • Default value: NONE

TLS verification mode. When set to NONE, no verification is performed. In CA mode the Certificate Authority and certificate are verified but not that the hostname matches. Use FULL mode for full certificate verification.

smartcache.redis.username
  • Type: string

Authenticate using the provided username. Overrides username in Redis URI. Requires password.

smartcache.redis.password
  • Type: string

Authenticate using the provided password. Overrides password in Redis URI.

smartcache.redis.keyspace
  • Type: string

  • Default value: smartcache

Prefix for all Redis keys used by Redis Smart Cache, such as cache entries, configuration, and metrics.

smartcache.redis.key-separator
  • Type: string

  • Default value: :

Delimiter to use between key elements.

smartcache.redis.pool.size
  • Type: integer

  • Default value: 8

Maximum number of connections that can be allocated by the pool at a given time. Use a negative value for no limit.

smartcache.redis.codec-buffer-size
  • Type: data size

  • Default value: 10MB

Maximum capacity of the buffer used to encode a result set.

Additional components

smartcache.metrics-step
  • Type: duration

  • Default value: 60s

Metrics publishing interval.

smartcache.config-step
  • Type: duration

  • Default value: 10s

Rule config refresh interval.

Rules

Redis Smart Cache uses rules to determine how SQL queries are cached. Rule configuration is stored in a Redis JSON document located at the key smartcache:config and can be modified at runtime. Redis Smart Cache will dynamically update to reflect changes made to the JSON document (see smartcache.config-step above to change the refresh rate).

Here is the default rule configuration:

{
  "rules": [
    {
      "tables": null,
      "tablesAny": null,
      "tablesAll": null,
      "regex": null,
      "ttl": 3600
    }
  ]
}

This default configuration contains a single passthrough rule where all SQL query results will be assigned a TTL of 3600 seconds.

Rules are processed in order and consist of criteria (conditions) and actions (results). Only the first rule with matching criteria will be considered, and its action applied.

Criteria

tables

Triggers if the given tables are exactly the same as the list in the SQL query (order does not matter).

tablesAny

Triggers if any of the given tables shows up in the SQL query.

tablesAll

Triggers if all the given tables show up in the SQL query.

regex

Triggers if regular expression matches the SQL query.

Action

ttl

Sets the time-to-live (in seconds) for the corresponding cache entry (default: 3600).

0

No caching

-1

No expiration

Examples

SELECT * FROM customers c, products p, orders o
Criteria Match
{ "tables": ["orders", "products"] }

Check

{ "tables": ["orders", "products", "customers"] }

Check

{ "tablesAny": ["transactions"] }

Check

{ "tablesAny": ["transactions", "orders"] }

Check

{ "tablesAll": ["transactions", "orders", "products"] }

Check

{ "tablesAll": ["orders", "products"] }

Check

{ "regex": "SELECT .+ FROM trans.*" }

Check

{ "regex": "SELECT .+ FROM cust.*" }

Check

Support

Redis Smart Cache is supported by Redis, Inc. on a good faith effort basis. To report bugs, request features, or receive assistance, please file an issue.

License

Redis Smart Cache is licensed under the MIT License. Copyright © 2023 Redis, Inc.