A FHIR® Store with internal, fast CQL Evaluation Engine
The goal of this project is to provide a FHIR® Store with an internal CQL Evaluation Engine which is able to answer population wide aggregate queries in a timely manner to enable interactive, online queries.
The project is currently under active development. Essentially all official CQL Tests pass. Please report any issues you encounter during evaluation.
Latest release: v0.8.0-beta.3
In order to run Blaze just execute the following:
docker volume create blaze-data
docker run -p 8080:8080 -v blaze-data:/app/data samply/blaze:0.8.0-beta.3Blaze will create the directory db inside the blaze-data volume on its first start and use the same database directory on subsequent starts.
In case Docker isn't available, Blaze can be run on a machine having OpenJDK 11 installed. Blaze is tested with AdoptOpenJDK.
wget https://github.com/samply/blaze/releases/download/v0.8.0-beta.3/blaze-0.8.0-beta.3-standalone.jar
java -jar blaze-0.8.0-beta.3-standalone.jar -m blaze.coreBlaze will run with an in-memory, volatile database for testing and demo purposes.
Running Blaze with durable storage requires to set DB_DIR.
Under Linux/macOS:
DB_DIR=db java -jar blaze-0.8.0-beta.3-standalone.jar -m blaze.coreUnder Windows:
$Env:DB_DIR="db"
java -jar blaze-0.8.0-beta.3-standalone.jar -m blaze.coreThis will create a directory called db inside the current working directory.
Logging output should appear which prints the most important settings and system parameters like Java version and available memory.
In order to test connectivity, query the health endpoint:
curl http://localhost:8080/healthBlaze is configured solely through environment variables. There is a default for every variable. So all are optional.
The following table contains all of them:
| Name | Default | Since | Description |
|---|---|---|---|
| DB_DIR | – | v0.8 | The directory were the database files are stored. This directory must not exist on the first start of Blaze and will be created by Blaze. However the parent directory has to exist. The default is to use an in-memory, volatile database. |
| DB_BLOCK_CACHE_SIZE | 128 | v0.8 | The size of the block cache of the DB in MB. |
| DB_RESOURCE_CACHE_SIZE | 10000 | v0.8 | The size of the resource cache of the DB in number of resources. |
| DB_MAX_BACKGROUND_JOBS | 4 | v0.8 | The maximum number of the background jobs used for DB compactions. |
| DB_RESOURCE_INDEXER_THREADS | 4 | v0.8 | The number threads used for indexing resources. |
| DB_RESOURCE_INDEXER_BATCH_SIZE | 1 | v0.8 | The number of resources which are indexed in a batch. |
| PROXY_HOST | — | v0.6 | The hostname of the proxy server for outbound HTTP requests |
| PROXY_PORT | — | v0.6 | Port of the proxy server |
| PROXY_USER | — | v0.6.1 | Proxy server user, if authentication is needed. |
| PROXY_PASSWORD | — | v0.6.1 | Proxy server password, if authentication is needed. |
| CONNECTION_TIMEOUT | 5 s | v0.6.3 | connection timeout for outbound HTTP requests |
| REQUEST_TIMEOUT | 30 s | v0.6.3 | request timeout for outbound HTTP requests |
| TERM_SERVICE_URI | http://tx.fhir.org/r4 | v0.6 | Base URI of the terminology service |
| BASE_URL | http://localhost:8080 | The URL under which Blaze is accessible by clients. The FHIR RESTful API will be accessible under BASE_URL/fhir. |
|
| SERVER_PORT | 8080 | The port of the main HTTP server | |
| METRICS_SERVER_PORT | 8081 | v0.6 | The port of the Prometheus metrics server |
| LOG_LEVEL | info | v0.6 | one of trace, debug, info, warn or error |
| JAVA_TOOL_OPTIONS | — | JVM options (Docker only) | |
| FHIR_OPERATION_EVALUATE_MEASURE_THREADS | 4 | v0.8 | The maximum number of parallel $evaluate-measure executions. Not the same as the number of threads used for measure evaluation which equal to the number of available processors. |
Blaze heavily benefits from vertical scaling (number of CPU cores). Even with single clients, transactions and CQL queries scale with the number of CPU cores. Performance tests show that 32 cores can be utilized easily.
Blaze uses three main memory regions, namely the JVM heap memory, the RocksDB block cache and the OS page cache. The JVM heap memory is used for general operation and specifically for the resource cache which caches resources in their ready to use object form. The RocksDB block cache is separate from the JVM heap memory because it's implemented as off-heap memory region. The block cache stores resources and index segments in uncompressed blob form. After the resource cache, it functions as a second level cache which stores resources in a more compact form and is not subject to JVM garbage collection. The OS page cache functions as a third level of caching, because it contains all recently access database files. The database files are compressed and so store data in an even more compact form.
The first recommendation is to leave half of the available system memory for the OS page cache. For example if you have 16 GB of memory, you should only allocate 8 GB for JVM heap + RocksDB block cache. The ratio between the JVM heap size and the RocksDB block cache can vary. First you should monitor the JVM heap usage and garbage collection activity in your use case. Blaze provides a Prometheus metrics endpoint on port 8081 per default. Using half of the remaining system memory for the JVM heap and half for the RocksDB block cache will be a good start.
Depending on the capabilities of your disk I/O system, having enough OS page cache for holding most of the active database file size, can really make a huge difference in performance. In case all your database files fit in memory, you have essentially a in-memory database. Test with up to 128 GB system memory showed that effect.
Blaze is designed with local (NVMe) SSD storage in mind. That is the reason why Blaze uses RocksDB as underlying storage engine. RocksDB benefits from fast storage. The idea is that latency of local SSD storage is faster than the network. So remote network storage has no longer an advantage.
If you have only spinning disks available, a bigger OS page cache can help. Otherwise especially the latency of operations will suffer.
Blaze maintains indexes for FHIR search parameters and CQL evaluation. The indexing process can be executed in parallel if transactions contain more than one resource. Transaction containing only one resource and direct REST interactions like create, update or delete don't benefit from parallel indexing.
Depending on your system, indexing can be I/O or CPU bound. Performance tests show, that at least 20 CPU cores can be utilized for resource indexing. The default number of indexing threads is 4 but can be changed by setting DB_RESOURCE_INDEXER_THREADS to a different value. However only transactions with more than DB_RESOURCE_INDEXER_THREADS resources will benefit from it.
If your transactions contain way more resources as your number of CPU cores available, you could experiment with increasing the DB_RESOURCE_INDEXER_BATCH_SIZE which has a default value of 1. With a value of more than one, the resource indexer will create batches of resources which are indexed together by one thread. This batching lowers the coordination overhead. But be aware that a batch size of about your transaction size will prevent any parallelism.
In-deep deployment options of Blaze are described in the Deployment Section of the Blaze documentation.
The developers of Blaze uses the YourKit profiler to optimize performance. YourKit supports open source projects with innovative and intelligent tools for monitoring and profiling Java and .NET applications. YourKit is the creator of YourKit Java Profiler, YourKit .NET Profiler and YourKit YouMonitor.
Copyright 2019 The Samply Development Community
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.