This project is a conceptual Java EE application running on Open Liberty for a health records system, designed to showcase best in class integration of modern cloud technology running on OpenShift.
Example Health is a conceptual healthcare/insurance type company. It has been around a long time, and has 100s of thousands of patient records. Example's health records look very similar to the health records of most insurance companies.
Originally, Example Health used a monolithic application structure for their application. Their application structure was a full stack Java application running on WebSphere connected to a DB2 database on System z. Here's what the original architecture for Example Health looked like:
Recently, Example Health decided to modernize their application and break it up into microservices. They decided to move to a SQL database connected to a Java EE application running on Open Liberty for the business logic and a Node.js application for the Patient UI. In addition, Example Health also decided to bring these applications to Openshift in the Cloud. The new current architecture for Example Health looks like this:
Since moving to Openshift, Example Health has expanded to include new microservices that include an Admin application and an Analytics application. These along with the Patient UI can be found in seperate code patterns:
- User makes a call to one of the APIs for the Java EE application which is located in Openshift's application load balancer.
- The API in Openshift's application load balancer triggers the API endpoint code in the Java EE application that is running on an Open Liberty server in a Docker container on Openshift.
- The Java EE application queries the MySQL database to get the desired data.
- The MySQL database sends back the data to the Java EE application where it gets handled accordingly.
- The data gets configured into JSON format that gets returned to the API and User.
-
Install the following prerequisite tools.
- A Java 8 (or higher) JDK such as OpenJDK
- Maven
- Docker
- IBM Cloud CLI
- OpenShift (oc) CLI
- MySQL client
-
Sign up for an IBM Cloud account if you do not have one. You must have a Pay-As-You-Go or Subscription account to deploy this code pattern. See https://cloud.ibm.com/docs/account?topic=account-upgrading-account to upgrade your account.
-
Create a IBM Cloud Red Hat OpenShift Container Platform cluster
-
Create a Compose for MySQL database. After the database is provisioned, make note of its URL, port, user and password.
-
Clone this project.
git clone https://github.com/IBM/example-health-jee-openshift.git -
Create the database and tables using a MySQL client. Import the SQL schema for the the Synthea simulated patient record data using the SQL file at:
example-health-api/samples/health_schema.sql. -
Build the Java EE application.
cd example-health-api mvn package -
Build the Java EE docker image.
docker build -t ol-example-health:1 . -
Create a repository in your dockerhub account and push the Java EE docker image to it. (Substitute your account name into the commands.)
docker tag ol-example-health:1 YOURACCOUNT/ol-example-health:1 docker login -u YOURACCOUNT docker push YOURACCOUNT/ol-example-health:1 -
Create a project (like a namespace) in OpenShift. This will create a new project and set it as the working project where pods and services will get deployed.
oc new-project health -
Edit the file
example-health-api/kubernetes-openshift.yamlto change theimagekey to your docker image. -
Set the secret values for your MySQL cloud deployment in the
example-health-api/create-secrets.shscript. All the necesssary values can be found in the IBM Cloud MySQL service credentials page:
NOTE: The connection URL would resemble the following. Substitute the server name and port from the Cloud service page above. The value of
DB_NAMEuse the database created and populated with the tables in step 6 above.
--from-literal=db_host="jdbc:mysql://$HOST:$PORT/$DB_NAME?sessionVariables=sql_mode=''"
Run the script to load the secrets into the OpenShift project.
$ ./create-secrets.sh
secret/db-secrets created
The Open Liberty DataSource configuration in server.xml uses environment variables injected into the
container by the deployment yaml via Kubernetes secrets to set database access parameters:
<dataSource id="DefaultDataSource" jndiName="jdbc/health-api" jdbcDriverRef="mysql-driver"
type="javax.sql.ConnectionPoolDataSource" transactional="true">
<properties url="${ENV_MYSQL_URL}"
user="${ENV_MYSQL_USER}"
password="${ENV_MYSQL_PWD}"/>-
Deploy the application to your cluster.
oc apply -f kubernetes-openshift.yaml -
Create a route to expose the application to the internet.
oc expose svc example-health-api -
Verify that the application is working. First obtain the hostname assigned to the route.
oc get route example-health-api NAME HOST/PORT PATH SERVICES PORT TERMINATION WILDCARD example-health-api example-health-api-health.****.appdomain.cloud example-health-api http NoneIn a browser window, navigate to
<hostname>/openapi/ui/. An OpenAPI specification of the endpoints and operations supported by the Java EE application appears. -
Generate synthentic patient health records and populate the MySQL database by running the
generate.shscript ingenerate/. Refer to the script's README for instructions on how to run the script.NOTE: In our testing, the script populates the MySQL database at about 125 patients per hour.
NOTE: This directive in
liberty.xmlis necessary to allow long-running patient load operations via REST:<transaction totalTranLifetimeTimeout="3600s"/>. The default timeout of 120 seconds is too short to batch patients at 50 per call, as the current script is configured.
Once the application is up and running, the OpenAPI UI will allow you to browse the available APIs:
The SQL schema in for Synthea derived data imported into Example Health uses this logical pattern and maps tables to Java classes under src/main/java/com/ibm/examplehealth mapped using JPA annotation.
Allergy.java :@Table(name="Allergies")
Appointment.java :@Table(name="Appointments")
Provider.java :@Table(name="Providers")
Organization.java:@Table(name="Organizations")
Prescription.java:@Table(name="Prescriptions")
Observation.java: @Table(name="Observations")
Patient.java: @Table(name="Patients")
This project builds its OpenLiberty container based on the RedHat UBI, Universal Base Image: https://www.redhat.com/en/blog/introducing-red-hat-universal-base-image
To ensure that your Docker container works in the more security conscious environment of OpenShift, use Libert 19.0.0.5 or higher
see: https://openliberty.io/blog/2019/03/28/microprofile22-liberty-19003.html#docker. In addition, to install the JDBC driver, note
that chown option to ADD and the chmod needed to give the JVM permission to read the JDBC driver.
Part of the Dockerfile used to build the image downloads the MySQL JDBC driver to connect to our cloud MySQL instance.
ADD --chown=default:root https://repo1.maven.org/maven2/mysql/mysql-connector-java/8.0.16/mysql-connector-java-8.0.16.jar ${INSTALL_DIR}lib/mysql-connector-java-8.0.16.jar
RUN chmod 644 ${INSTALL_DIR}lib/mysql-connector-java-8.0.16.jar
COPY liberty-mysql/mysql-driver.xml ${CONFIG_DIR}configDropins/defaults/
The persistence.xml specifies the driver details that is injected as the default persistence context via
CDI:
<jdbcDriver id="mysql-driver"
javax.sql.XADataSource="com.mysql.cj.jdbc.MysqlXADataSource"
javax.sql.ConnectionPoolDataSource="com.mysql.cj.jdbc.MysqlConnectionPoolDataSource"
libraryRef="mysql-library"/>
<library id="mysql-library">
<fileset id="mysqlFileSet" dir="/opt/ol/wlp/lib"
includes="mysql-connector-java-8.0.16.jar"/>
</library>We don't need to specify any persistence context in this annotation because only one is defined:
@PersistenceContext
EntityManager entityManager;Because by default transactions are handled on a per call basis, when loading many records (100s of MBs) via generate, we noticed memory usage rose dramatically as the EntityManager instantiated Java objects representing each database table. Running clear() during batch
processing allowed memory to be reclaimed after entites were pushed to MySQL.
private void flushBatch(int size, int cnt, String type) {
if ( (cnt % batchSize == 0) || (size == cnt) ) {
entityManager.flush();
entityManager.clear();
}
}The default OpenShift timeout for the gateway is 30 seconds, too short for long running REST calls like the generate endpoint to load health data. It's necessary to set the route timeout to a longer value for the route defined for the health API:
# oc annotate route example-api --overwrite haproxy.router.openshift.io/timeout=60m
route.route.openshift.io/example-api annotated
We set up a Data Source to allow Open Liberty to manage our connections to the MySQL database via the MySQL JDBC driver. For more details, see this Open Liberty guide: https://openliberty.io/guides/jpa-intro.html
The persistence.xml specifies the driver details that is injected as the default persistence context via
CDI:
<jdbcDriver id="mysql-driver"
javax.sql.XADataSource="com.mysql.cj.jdbc.MysqlXADataSource"
javax.sql.ConnectionPoolDataSource="com.mysql.cj.jdbc.MysqlConnectionPoolDataSource"
libraryRef="mysql-library"/>
<library id="mysql-library">
<fileset id="mysqlFileSet" dir="/opt/ol/wlp/lib"
includes="mysql-connector-java-8.0.16.jar"/>
</library>We don't need to specify any persistence context in this annotation because only one is defined:
@PersistenceContext
EntityManager entityManager;Because by default transactions are handled on a per call basis, when loading many records (100s of MBs) via generate, we noticed memory usage rose dramatically as the EntityManager instantiated Java objects representing each database table. Running clear() during batch
processing allowed memory to be reclaimed after entites were pushed to MySQL.
private void flushBatch(int size, int cnt, String type) {
if ( (cnt % batchSize == 0) || (size == cnt) ) {
entityManager.flush();
entityManager.clear();
}
}The OpenShift dashboard is helpful in problem determination, in our case memory exhaustion due to the default 1GB JVM heap size during bulk loading via JPA:
Memory exhastion (hard limit at 1GB and rapid drop off as the call fails and cleanup occurs):
To solve this problem, we an option in jvm.options to increase the amount of memory available to the JVM above the default 1GB heap size:
-Xmx4096m
And added this to the image by adding this line in the Dockerfile:
COPY liberty/jvm.options $CONFIG_DIR
The OpenShift monitoring dashboard shows how container memory is able to exceed the 1GB threshold when necessary (in this case, the peak is caused by JSON-B parsing JSON inpout to the REST call that generates simulated health records):
This code pattern is licensed under the Apache License, Version 2. Separate third-party code objects invoked within this code pattern are licensed by their respective providers pursuant to their own separate licenses. Contributions are subject to the Developer Certificate of Origin, Version 1.1 and the Apache License, Version 2.