A Liquibase extension adding support for Google Cloud Spanner. Include this in your application project to run Liquibase database migration scripts against a Google Cloud Spanner database.
This extension is currently beta.
- Bug fix: Fixed a potential
ClassNotFoundExceptionforcom.google.spanner.admin.database.v1.DatabaseAdminGrpc
- Initial beta release.
Install Liquibase Community CLI from here. Alternatively, there are many other ways to install (e.g. brew on Mac OS/X). These examples were run with Liquibase 4.2.0.
Once Liquibase is installed, use the latest release or build your own liquibase-spanner-SNAPSHOT-all.jar and copy it into the Liquibase lib directory.
You can create a Spanner instance in the GCP console or use an emulator (with JDBC). You will also need to create a database for Liquibase to use.
Spanner CLI is a convenient way to access Spanner. It can be installed from here.
Configure the connection in the file liquibase.properties:
url: jdbc:cloudspanner:/projects/<project>/instances/<instance>/databases/<database>
Using the Liquibase CLI the following ChangeLogs are examples of using Spanner. Review Liquibase best practices. In this example, changelog.yaml is used as the master changelog.
Run:
liquibase --changeLog example/changelog.yaml
| Example | Description |
|---|---|
| create-schema.yaml | Create schema, including interleaved tables, column options, and indexes |
| load-data-singers.yaml | Load data into Singers table from CSV |
| load-update-data-singers.yaml | Insert or update data in Singers table from CSV |
| add-lookup-table-singers-countries.yaml | Create countries table as a foreign key from Country field in Singers |
| modify-data-type-singers-lastname.yaml | Alter STRING datatype in Singers LastName column |
| insert.yaml | Insert rows into Singers table |
| delete.yaml | Delete rows from Singers |
| update.yaml | Update rows in Singers |
The following Liquibase ChangeTypes are supported:
createTable, dropTable, addColumn, modifyDataType, addNotNullConstraint, dropColumn, createIndex, dropIndex, addForeignKeyConstraint, dropForeignKeyConstraint, dropAllForeignKeyConstraints, addLookupTable
The following Liquibase ChangeTypes are not allowed with Cloud Spanner:
addAutoIncrement, addDefaultValue, addPrimaryKey, addUniqueConstraint, dropUniqueConstraint, createProcedure, createSequence, createView, dropDefaultValue, dropNotNullConstraint, dropPrimaryKey, dropProcedure, dropSequence, dropView, renameColumn, renameSequence, renameTable, renameView, setColumnRemarks, setTableRemarks, alterSequence
The following data DML ChangeTypes are supported:
insert, update, loadData, loadUpdateData
Note:
- Column OPTIONS and table INTERLEAVE must be applied using modifySql.
- Instead of unique constraints use unique indexes.
See limitations.md for a full list of limitations and unsupported features.
Some Spanner specific SQL, such as INTERLEAVE'd tables or column OPTIONS, require using Liquibase's modifySql. See create-schema.yml for an example of doing this.
In order to limit the number of schema updates in a 7-day period, run Liquibase with small changeSets. Alternatively, use SQL change and batch the DDL using batch statements.
There are DML limits for the number of rows affected during DML. The recommendation is to use partitioned DML. Using Spanner JDBC driver this can be configured using the AUTOCOMMIT_DML_MODE.
This has been implemented in some of the changeSet types such as mergeColumns, but not in all changeSet types such as delete.
There are a number of features that Spanner does not have such as views and stored procedures. The Liquibase extension will throw an exception during analysis of the changeSet in most cases, but not all. For example, a DELETE without a WHERE clause will fail in Spanner but not in the Liquibase extension.
| Gradle target | Description |
|---|---|
| test | Run mock and Spanner emulator tests |
| build | Build extension and run above tests |
| integrationTest | Run Spanner in GCP tests |
| jibDocker | Build a local runnable docker container |
Testing requirements:
- Emulator requires testcontainers and its requirements installed.
- Spanner in GCP requires SPANNER_PROJECT and SPANNER_INSTANCE environment variables set to an active instance
- Spanner in GCP requires application default credentials set or GOOGLE_APPLICATION_CREDENTIALS environent set
There are two JARs built:
- build/libs/liquibase-spanner-VERSION.jar
- build/libs/liquibase-spanner-VERSION-all.jar
The first JAR is just the extension itself, while the second one includes all of the dependencies needed to run with Liquibase. Install the second one into the Liquibase lib directory.
For contributions please see contributing and our code of conduct.
If you have any questions, find a bug, or have a feature request please open an issue. Please note that this extension is not officially supported as part of the Cloud Spanner product.