CoolRetailer

Microservices application with Istio, gRPC, Redis, BigQuery, Spring Boot, Spring Cloud and Stackdriver

Smart product finder with autocomplete feature
Distributed Stackdriver Tracing with Log correlation across gRPC calls
Istio based Stackdriver Service Monitoring
Automatic synthetic load generation with Locust
Complete CI/CD to GKE with Google Cloud Build
Redis in Master/Slave replication mode (image from Bitnami)
Autocomplete logic based on ZRANGEBYLEX Redis command
Tunable cache size
Dataset from BestBuy (converted to ndjson using JsonProcessor)
Developed completely with VSCode
Inspired by Next '18 Microservices Demo
Covering slides on Slideshare

Architecture:

Automatic Installation:

Guided one click installation from Google Cloud Shell. Complete stack (incl. all dashboards shown below + Istio fault simulation). No client tooling required.

Manual Installation:

Get the Best Buy products dataset and save it to a GCS bucket:
BQ requires the JSON to be new line delimited. Use the provided utility JSON processor for this:

java -jar src/queryservice/target/queryservice-1.0.0.jar \
    --spring.profiles.active=JSON \
    --input.json="$PWD"/products.json --exit

Load it to BQ as dataset coolretailer.products from GCS
Create a GCP service account with the following roles:

BigQuery Data Viewer, BigQuery Job User, Stackdriver Debugger Agent, Cloud Trace Agent, Context Graph Asserter, Error Reporting Admin, Logs Writer, Monitoring Metric Writer

and install the private key as app-gac.json as kubernetes secret app-gac.

apply Istio manifests in: setup/istio-manifests
set PROJECT_ID, TAG_NAME, REDIS_HOST in setup/kubernetes-manifests/ and apply them

Notes:

Spring Cloud Sleuth Stackdriver Trace and Logging enabled.
Complete CI/CD pipeline with Google Cloud Build.
Test locally using:

$mvn spring-boot:run \
    -DGOOGLE_APPLICATION_CREDENTIALS=path-to-key-file \
    -DPROJECT_ID=coolretailer \
    -DGOOGLE_CLOUD_PROJECT=coolretailer \
    -Dspring-boot.run.arguments=--spring.redis.host=localhost

Using docker from Cloud Shell:

(If tunneling to the redis service is required)

gcloud container clusters get-credentials coolretailer2 --zone europe-west4-a --project coolretailer \
 && kubectl port-forward $(kubectl get pod --selector="app=redis-master,role=master,tier=backend" --output jsonpath='{.items[0].metadata.name}') 6379:6379

gcloud auth configure-docker -q &&\
docker run \
    --network=host \
    -v /home/kbhattacharya:/etc/app-gac \
    -e GOOGLE_CLOUD_PROJECT=coolretailer \
    -e PROJECT_ID=coolretailer \
    -e GOOGLE_APPLICATION_CREDENTIALS=/etc/app-gac/app-gac.json \
    -e --spring.sleuth.sampler.probability=1.0 \
    -e --spring.application.name=ux-local \
    -e --spring.cloud.gcp.trace.enabled=true \
    -e --spring.cloud.gcp.logging.enabled=true \
    -e --spring.cloud.gcp.project-id=coolretailer \
    -it gcr.io/coolretailer/queryservice

Use Cloud Memorystore instead of the Redis deployments (src/setup/kubernetes-manifests/archive/redis):
Istio TLS setup
See setup/istio-manifests/secure for simple TLS configuration
The static content is recommended to be served from a GCS bucket with Cloud CDN. Just drop off the content of src/ui/static/ui.* and src/ui/static/lib in a GCS bucket and provide allUsers READER access.