This repo contains a library for generating data to push into Consul, a command line client for outputting that generated data and some terraform code to use that data and push it to Consul.
The generate subdirectory contains the top level package for all data generators. Subpackages are for generating a specific type of data.
go get github.com/mkeeler/consul-data/cmd/consul-data
Usage: consul-data [--version] [--help] <command> [<args>]
Available commands are:
generate Generate data for Consul
push Pushes generated data to consul
Usage: consul-data generate [OPTIONS] [output path]
Generate random data for consul.
By default the generated output is sent to the console but
an optional output path may be used to cause it to be written
to a file
Command Options
-config=<string>
Path to the configuration to use for generating data
-seed=<int>
Value to use to seed the pseudo-random number generator with
instead of the current time
Usage: consul-data push [OPTIONS]
Push data to Consul
By default this command will generate the random data and push it
to Consul. The data can be pregenerated if the -data flag is used
to specify a file which should be in the format outputted by the
consul-data generate command
Command Options
-ca-file=<value>
Path to a CA file to use for TLS when communicating with Consul.
This can also be specified via the CONSUL_CACERT environment
variable.
-ca-path=<value>
Path to a directory of CA certificates to use for TLS when
communicating with Consul. This can also be specified via the
CONSUL_CAPATH environment variable.
-client-cert=<value>
Path to a client cert file to use for TLS when 'verify_incoming'
is enabled. This can also be specified via the CONSUL_CLIENT_CERT
environment variable.
-client-key=<value>
Path to a client key file to use for TLS when 'verify_incoming'
is enabled. This can also be specified via the CONSUL_CLIENT_KEY
environment variable.
-config=<string>
Path to the configuration to use for generating data
-data=<string>
Path to data generated by consul-data generate to use as the data
source instead of generating new data
-datacenter=<value>
Name of the datacenter to query. If unspecified, this will default
to the datacenter of the queried agent.
-http-addr=<address>
The `address` and port of the Consul HTTP agent. The value can be
an IP address or DNS address, but it must also include the port.
This can also be specified via the CONSUL_HTTP_ADDR environment
variable. The default value is http://127.0.0.1:8500. The scheme
can also be set to HTTPS by setting the environment variable
CONSUL_HTTP_SSL=true.
-namespace=<default>
Specifies the namespace to query. If not provided, the namespace
will be inferred from the request's ACL token, or will default
to the `default` namespace. Namespaces are a Consul Enterprise
feature.
-output=<string>
Path to output the data file to if we generated it instead of
loading it in
-seed=<int>
Value to use to seed the pseudo-random number generator with
instead of the current time
-stale
Permit any Consul server (non-leader) to respond to this request.
This allows for lower latency and higher throughput, but can result
in stale data. This option has no effect on non-read operations.
The default value is false.
-tls-server-name=<value>
The server name to use as the SNI host when connecting via
TLS. This can also be specified via the CONSUL_TLS_SERVER_NAME
environment variable.
-token=<value>
ACL token to use in the request. This can also be specified via the
CONSUL_HTTP_TOKEN environment variable. If unspecified, the query
will default to the token of the Consul agent at the HTTP address.
-token-file=<value>
File containing the ACL token to use in the request instead of one
specified via the -token argument or CONSUL_HTTP_TOKEN environment
variable. This can also be specified via the CONSUL_HTTP_TOKEN_FILE
environment variable.
{
"KV": {
"NumEntries": 500,
"KeyType": "pet-name",
"ValueType": "random-b64",
"PetName": {
"Prefix": "",
"Segments": 3,
"Separator": "-",
},
"RandomB64": {
"MinSize": 64,
"MaxSize": 128,
}
},
"Catalog": {
"NumNodes": 250,
"MinServicesPerNode": 4,
"MaxServicesPerNode": 8,
"MinInstancesPerService": 1,
"MaxInstancesPerService": 1,
"MinMetaPerNode": 2,
"MaxMetaPerNode": 16,
"MinMetaPerService": 5,
"MaxMetaPerService": 7,
"NodeType": "pet-name",
"ServiceType": "pet-name",
"AddressType": "random-testing",
"MetaKeyType": "pet-name",
"MetaValueType": "random-b64",
"NodePetNames": {
"Prefix": "",
"Segments": 3,
"Separator": "-"
},
"ServicePetNames": {
"Prefix": "",
"Segments": 3,
"Separator": "-"
},
"MetaKeyPetNames": {
"Prefix": "",
"Segments": 3,
"Separator": "-"
},
"MetaValueRandomB64": {
"MinSize": 64,
"MaxSize": 128
}
}
}We can use the consul-data generate subcommand to generate the data file required by the terraform provider. After that you are a simple terraform apply away from pushing all the data into Consul. Note that for very large
numbers of resources terraform has some bad performance issues and the consul-data push command will need to be used instead.
consul_address- (Optional) Allows you to override the Consul HTTP API addresses used by the Consul Terraform providerconsul_token- (Optional) Allows you to set a Consul ACL token for use by the Consul Terraform provider.data- (Required) Path to the data generated by theconsul-data generatesubcommand.