Golang Microservices Example

An demonstration of Golang micro-services that expose a HTTP/JSON frontend and then leverages gRPC for inter-service communication.

Services written in Golang
gRPC for inter-service communication
Consul for service discovery
Jaeger for request tracing

The example application plots Hotel locations on a Google map:

The web page makes an HTTP request to the API Endpoint which in turn spawns a number of RPC requests to the backend services.

Data for each of the services is stored in JSON flat files under the data/ directory. In reality each of the services could choose their own specialty datastore. The Geo service for example could use PostGis or any other database specializing in geospacial queries.

Request Tracing

The Jaeger Tracing project is used for tracing inter-service requests. The tracing package is used initialize a new service tracer:

tracer, err := tracing.Init("serviceName", jaegeraddr)
if err != nil {
    fmt.Fprintf(os.Stderr, "%v\n", err)
    os.Exit(1)
}

View dashboard: http://localhost:16686/search

Service Discovery

Consul is used for service discovery. This allows each service to register with the registry and then discovery the IP addresses of the services they need to comunicate with. The registry package is used for registering services:

rc, err := registry.NewClient(consuladdr)
if err != nil {
    fmt.Fprintf(os.Stderr, "%v\n", err)
    os.Exit(1)
}

id, err := rc.Register("serviceName", port)
if err != nil {
    fmt.Fprintf(os.Stderr, "%v\n", err)
    os.Exit(1)
}
defer rc.Deregister(id)

View dashboard: http://localhost:8500/ui

Load balancing

Client side load balancing is handled with olivere/grpc pacakge. It uses Consul for service discovery and then the gRPC Dialer leverages a RoundRobin balancing strategy.

Additional Reading:

Installation

Setup

Docker is required for running the services https://docs.docker.com/engine/installation.

Protobuf v3 are required:

$ brew install protobuf

Install the protoc-gen libraries:

$ go get -u github.com/golang/protobuf/{proto,protoc-gen-go}

Clone the repository:

$ git clone git@github.com:harlow/go-micro-services.git

Run

To make the demo as straigforward as possible; Docker Compose is used to run all the services at once (In a production environment each of the services would be run (and scaled) independently).

$ make run

Vist the web page in a browser:

http://localhost:5000/

cURL the API endpoint and receive GeoJSON response:

$ curl "http://localhost:5000/hotels?inDate=2015-04-09&outDate=2015-04-10"

The JSON response:

{
	"type": "FeatureCollection",
	"features": [{
		"id": "5",
		"type": "Feature",
		"properties": {
			"name": "Phoenix Hotel",
			"phone_number": "(415) 776-1380"
		},
		"geometry": {
			"type": "Point",
			"coordinates": [-122.4181, 37.7831]
		}
	}, {
		"id": "3",
		"type": "Feature",
		"properties": {
			"name": "Hotel Zetta",
			"phone_number": "(415) 543-8555"
		},
		"geometry": {
			"type": "Point",
			"coordinates": [-122.4071, 37.7834]
		}
	}]
}

Protobufs

If changes are made to the Protocol Buffer files use the Makefile to regenerate:

$ make proto

Credits

Thanks to all the contributors. This codebase was heavily inspired by the following talks and repositories:

www.hward.com · GitHub @harlow · Twitter @harlow_ward

hariprasadraja/go-micro-services