Consul-Imex is a simple import/export tool for Consul key/value storage over Consul HTTP API.
- Export all the keys in a prefix, save it as a json file.
- Import the keys from a json file to Consul under a specified prefix.
- Copy keys:
- from a prefix to another prefix.
- from a server to another server under a specified prefix.
You can install Consul Imex in several ways:
- Executable phar file (requires PHP >=5.5.9)
- Docker image (requires Docker engine)
- Composer dependency (requires Git, Composer and PHP >=5.5.9)
- Single PHP project (requires Git, Composer and PHP >=5.5.9)
Download the phar file from https://github.com/gamegos/consul-imex/releases/download/1.0.0-rc1/consul-imex.phar, then assign execute permission for it.
Example:
wget -O /usr/local/bin/consul-imex https://github.com/gamegos/consul-imex/releases/download/1.0.0-rc1/consul-imex.phar
chmod +x /usr/local/bin/consul-imex
docker pull sozpinar/consul-imex
composer require gamegos/consul-imex
Install via composer:
composer create-project gamegos/consul-imex
Or clone/download and install manually:
git clone https://github.com/gamegos/consul-imex.git
cd consul-imex
composer install
consul-imex <command> [options] [arguments]
See Notices for Docker Usage for detailed information of docker usage.
docker run -t -v `pwd`:/consul-imex sozpinar/consul-imex <command> [options] [arguments]
composer exec -- consul-imex <command> [options] [arguments]
php scripts/consul-imex.php <command> [options] [arguments]
consul-imex export [options] <file>
- file: Output data file.
- --url (-u): Consul server url.
- --prefix (-p): Path prefix.
- --token (-c): Consul token.
consul-imex import [options] <file>
- file: Input data file.
- --url (-u): Consul server url.
- --prefix (-p): Path prefix.
- --token (-c): Consul token.
consul-imex copy [options] <source> <target>
- source: Source prefix.
- target: Target prefix.
- --source-server (-s): Source server URL.
- --target-server (-t): Target server URL. If omitted, source server is used as target server.
- --source-server-token (-c): Source server Consul token.
- --target-server-token: Target server Consul token. If omitted, source server token is used as target server token.
$ consul-imex export -u http://localhost:8500 -p /old/prefix my-data.json
93 keys are fetched.
Export using token key:
$ consul-imex export -c 4874874a-2a3s-7892-123e-597a4e1v87e1 -u http://localhost:8500 -p /old/prefix my-data.json
93 keys are fetched.
$ consul-imex import -u http://localhost:8500 -p /new/prefix my-data.json
93 keys are stored. (25 new directories are created.)
Import using token key:
$ consul-imex import -c 4874874a-2a3s-7892-123e-597a4e1v87e1 -u http://localhost:8500 -p /new/prefix my-data.json
93 keys are stored. (25 new directories are created.)
Copy keys from /old/prefix to /new/prefix:
$ consul-imex copy -s http://localhost:8500 /old/prefix /new/prefix
93 keys are fetched.
93 keys are stored. (25 new directories are created.)
Operation completed.
Copy keys from /old/prefix to /new/prefix using token:
$ consul-imex copy -s http://localhost:8500 -c 4874874a-2a3s-7892-123e-597a4e1v87e1 /old/prefix /new/prefix
93 keys are fetched.
93 keys are stored. (25 new directories are created.)
Operation completed.
Copy keys under /my/prefix to another server:
$ consul-imex copy -s http://localhost:8500 -t http://anotherhost:8500 /my/prefix /my/prefix
93 keys are fetched.
93 keys are stored. (25 new directories are created.)
Operation completed.
Copy keys under /my/prefix to another server using token:
$ consul-imex copy -s http://localhost:8500 -c 4874874a-2a3s-7892-123e-597a4e1v87e1 -t http://anotherhost:8500 --target-server-token 6242c15a-9w4x-2318-61a2-8as8c47317 /my/prefix /my/prefix
93 keys are fetched.
93 keys are stored. (25 new directories are created.)
Operation completed.
Copy all keys to another server:
$ consul-imex copy -s http://localhost:8500 -t http://anotherhost:8500
492 keys are fetched.
492 keys are stored. (108 new directories are created.)
Operation completed.
To use import and export commands with docker, the input/output files must be accessible by the container. The default working directory of the image is /consul-imex and input/output files are placed under this directory by default.
Example 1: Mount a host directory to the container for export operation, then the container will create my-data.json file in the host directory.
$ docker run -it -v `pwd`:/consul-imex sozpinar/consul-imex export -u 192.168.1.20:8500 -p /foo/bar my-data.json
93 keys are fetched.
Example 2: Copy the output file to your working directory after export operation.
$ docker run -it sozpinar/consul-imex export -u 192.168.1.20:8500 -p /foo/bar my-data.json
93 keys are fetched.
$ docker cp `docker ps -ql`:/consul-imex/my-data.json .
Example 1: Mount a host directory to the container for import operation, then the container will read my-data.json file from the host directory.
$ docker run -it -v `pwd`:/consul-imex sozpinar/consul-imex import -u 192.168.1.20:8500 -p /new/prefix my-data.json
93 keys are stored. (25 new directories are created.)
Example 2: Mount a file to the container and use it for import operation. This method does not require the input file to be placed in the default working directory.
$ docker run -it -v `pwd`/my-data.json:/my-data.json sozpinar/consul-imex import -u 192.168.1.20:8500 -p /new/prefix -v /my-data.json
93 keys are stored. (25 new directories are created.)
If your Consul service is in a private network or does not have a public URL, you may have to set up a custom network configuration for the docker container.
docker run -it --net=host sozpinar/consul-imex copy -s http://localhost:8500 -t http://anotherhost:8500
If the default docker network type is bridge then the running container does not recognize 'localhost'. So we simply add --net=host argument to make the container to use the network of the host machine.
Consul key/value storage allows a directory to have a value like an ordinary key. If a directory has a value, Consul Imex will ignore the value or the sub-keys; this depends how the keys are ordered.