A client-server application showcasing streaming of prime numeric values from a GRPC server.
mordor - prime-number-server
A GRPC protocol serving service that produce prime numbers in a streaming manner.
gondor - proxy-service
A http streaming service that serves clients requests and accepts numeric value which is then being forwarded to mordor service.
protobuf - GRPC contract protocol used by both services to communicate to each other
- Scala - main programing language used
- cats - functional library abstraction
- Http4s - Http web service library
- fs2.Streams - Streaming process library
- fs2 grpc - grpc implementation built on top of cats and fs2
- munit - unit test library
- pureConfig - typed configuration sourcing library
Internet Gondor Mordor
| Endpoint -> Service -> Repository -┐ |
clients <-> | |--> grpc <--| mordor.Service
| Endpoint <- Service <- Repository <┘ |
layer that is responsible for receiving and serving http request/response.
layer that is responsible for any business logic of the application (e.g chaining of events, change of sequence of call, some validation etc)
layer responsible for communication and other activities on external modules.
protocol used to communicate for both gondor and mordor service
layer responsible for the computation related to the request
Mordor prime-number-server
, is responsible for prime number generation that will be stream to the client. It's main function is to abstract any computation away from the proxy server a.k.a Gondor.
grpcurl -d '{"maxNumberRange":"17"}' -plaintext \
-import-path protobuf/src/main/protobuf -proto prime.proto \
127.0.0.1:9999 com.service.PrimeNumberService/GeneratePrimeNumber
sbt mordor
Gondor proxy-service
, is responsible for accepting request and response from clients, apply any necessary basic validation that can be extracted away from Mordor, and serving the response back to clients in a streaming fashion.
/prime/:intValue
- strictly accepts positive numeric value only
/prime/v2/:intValue
- [Optional solution]- accepts any numeric value, however gives you a response if fails the validation
/prime/v3/:intValue
- [Not part of the solution(yet)]- accepts any numeric value, however returns status 501 not implemented function response
request:
curl -v GET 'http://127.0.0.1:8081/prime/17'
response:
* Connected to 127.0.0.1 (127.0.0.1) port 8081 (#1)
> GET /prime/17 HTTP/1.1
> Host: 127.0.0.1:8081
> User-Agent: curl/7.64.1
> Accept: */*
>
< HTTP/1.1 200 OK
< Content-Type: text/plain; charset=UTF-8
< Date: Thu, 15 Jul 2021 12:01:00 GMT
< Transfer-Encoding: chunked
<
* Connection #1 to host 127.0.0.1 left intact
2,3,5,7,11,13,17
NOTE: response should be compressed with GZip
, Curl is not showing it, However its shown on the demo below.
sbt gondor
to create a gondor docker image
sbt dockerGondor
to create a mordor docker image
sbt dockerMordor
# docker run --rm -p 9999:9999 --name mordor <docker id/name>
# e.g
docker run --rm -p 9999:9999 --name mordor e9ab442f5b6d
# docker run --rm -p 8081:8081 --name gondor <docker id/name>
# e.g
docker run --rm -p 8081:8081 --name gondor e9ab442f5b6d
to run all gondor unit tests
sbt testGondor
to run all mordor unit tests
sbt testMordor
The tools and libraries I chosen and picked were mainly because of my taste, what I know that will do the job smoothly and my likeness on working in a functional code base.
As everything is expressed in functional way (I hope), It will be easy to comprehend with the code and extract/change/modify/chain any business request.
Also testing every layer of the application feels so much easier due to most of the function are composed, pure. additionally we are sure that all operation are executed at the end of the application.
Why I choose fs2.Stream over others? Aside from its being a purely functional streaming library, For me it's a great fit to the ecosystem since fs2.stream is a first class citizen of http4s. I would say integration is easy, along with their core functional abstraction under the hood, cats. also worth to mention that because of its pull based approach of streaming we mitigate any network and buffer overload also reduce any resource hungry operation. that said it fits perfectly on our requirement.
Why Effect pattern? by doing so. we build our application by wrapping behaviors/description in a effect, which then be evaluated latter of the application. and be sure that there are no operation that will execute directly that would introduce side effects, additionally we can change any execution model we like on our application.
Scala - Alternatively we can use any programming language/libraries where we are comfortable and with rich GRPC support such as (Go, Node[type-script/javascript], Java, Rust etc).
cats - ZIO, scalaZ
Http4s - Akka Http, ZIO Http, PlayFramework
Fs2.Streams - Akka Streams, ZStreams
Fs2 Grpc - Akka GRPC
Munit - Scala test, Specs2
PureConfig - Typesafe config
-
Give some love on the Mordor code base.
-
CI/Pipeline integration (Jenkins, Travis or Github actions etc)
-
Separation of this mono repo into 3 different repositories to enable a parallel contribution from other developers.
-
Enrich unit tests with more assertions and cases for both services.
-
Enhance test using property-based testing libraries such as ScalaCheck etc.
-
black box/functional testing.
-
web app (probably built on react/vue:type-script) for better stream response representation.
-
add logging libraries such as log4cats etc.
-
Apply Authentication and Authentication Layer on both Gondor and Mordor Services.
- TLS
- Session based/Cookie based
-
integrate refactoring and linting tool such as scalafix.
-
Tapir Integration for API documentation.
-
Expose health check APIs for both services.
-
performance testing suite using Gatling.io
-
Create a docker-compose yaml file
-
upgrade to a more standard response format such as JSON.