-
- 1.1 Queues
- 1.2 Auth service
- 1.3 File processor
- 1.4 File service
- 1.5 User service
-
- 2.1 Use cases
- 2.2 Error handling
Ok I know you look on diagram above a think "what the ...". That was my creativity but let me start from the beginning ...
Imagine system where authorized users can upload own images on the server and share it with another peoples. So first user need to create an account, and then he may share file for others via this app. Basically if you like AWS this is very tiny example of creation own S3 service (but please take it with a grain of salt).
On AWS S3 we can use Lambda's which listen for events and then make an actions. In this case
app file-processor is a simple example of lambda write in python 🐍. The main duty of this service is resize
and clear image from Exif (scroll bellow to know something more).
So to quickly summarize... user create account, sign in, upload image (*.jpg, *.jpeg, *.png)
and then fake Lambda start processing file. It's time for persisting data mechanism. Docker-compose
create a volume files which is shared between file-service, file-processor and root machine. It
literally shares a space with two catalogs unprocessed and for share. I want to avoid situation
where owner of file, doesn't have access for his asset because may be resized at this time. That's why image
is saved in unprocessed folder and only possessor has access to this resource by special token
generated by uuidv4 and saved in Redis for tiny amount of time.
Resized, without Exif file is saved in for-share folder after finish file-processor task, which
take around 200-300 ms. Everyone have access for this resource now. Old one is heavily removed
from unprocessed folder also by the same service, for cleaning.
System is build in microservice architecture and some services use database per service. RabbitMQ's
queues are used for TCP communication between apps using RPC that allows one program to directly
call procedures in another program either on the another container on the network... but it doesn't just do that.
Do you remember file-processor? What if will something happen with server where app was deployed
during processing big image? For this case app use durable queue. You should know that file-service send
message on file_processor_queue and this msg is permanent until some receivers will pull and ack this.
This means that after restarting the application, the message will be pulled
again (if it was not previously confirmed) and all actions will be make to the end.
This mechanism protects the application from zombies in the unprocessed folder.
Lorem ipsum
Lorem ipsum
Lorem ipsum
Lorem ipsum
Lorem ipsum
Lorem ipsum
Lorem ipsum
Lorem ipsum
Lorem ipsum
Lorem ipsum
Lorem ipsum
Lorem ipsum
Lorem ipsum
Lorem ipsum
Lorem ipsum
Lorem ipsum
Lorem ipsum
Lorem ipsum
@Catch(RpcException)
export class RpcPatternFilter implements RpcExceptionFilter<RpcException> {
constructor(private readonly loggerService: LoggerService) {
loggerService.setContext(RpcPatternFilter.name);
}
catch(exception: RpcException, host: ArgumentsHost): Observable<string> {
this.loggerService.warn(`Caught: [${exception.message}]`);
throw new Error(exception.message);
}
}Lorem ipsum ADD DIAGRAM
Lorem ipsum