Lambda and ECS?
brennanerbz opened this issue · 6 comments
Would it be possible to build a backed with microservices using ECS and then run them through Lambda and API gateway? Would that be recommended for a startup with a couple engs?
Lambda and ECS are for two different use cases. It's entirely possible that a startup would have good reasons to use both, but its important to understand the context of why you would use each.
Lambda is for when you want to run a small bit of code without provisioning infrastructure at all. It works fantastically as a way to do some ephemeral processing for some workflow, or as a way to power an endpoint that has very little traffic.
However, when it comes to running a high traffic service for long periods of time many companies will find that it is far more efficient to run a persistent server process. For example Lambda is billed per 100ms of execution time, rounded up to the nearest 100ms. So if your application code only takes 50ms to serve a request you would still be paying for 100ms of execution time with Lambda per request.
While you have very low request volume you will save money with Lambda because paying for an extra 50ms of execution time is still less than paying for an instance to run 24/7. But if you run a service that gets many requests all those extra 50ms will start to add up on your bill.
At that point you can get savings by running your own instance because the instance can be running those 50ms tasks all the time (perhaps even concurrently for Node.js) and the bill for running that instance 24/7 will be less than the bill for Lambda.
Many organizations will use both Lambda and ECS, perhaps using Lambda for simple low volume services, while using ECS for core computing services that have consistent medium to high volume that would be less efficient to run in Lambda.
Additionally the way Lambda works is such that your lambda functions will suffer extra overhead from setting up the prerequisites for your code to execute (connecting to any databases, downloading any data needed, etc). A long running container in ECS can connect to the database once on the first startup, and download any data it needs once, and then reuse that data to serve many, many requests far faster than a Lambda function could because the lambda function would suffer from the extra penalty of reestablishing these prerequisites so often.
I hope this helps. Definitely do some more research into what Lambda is versus what Docker containers and ECS is, and I think you will better understand what kind of use cases would be best for each service.
It really depends on your expectations around adoption and traffic for your service. So for example Lambda is great for getting a basic MVP up and running quickly, but eventually if you got a successful service that is getting consistent traffic you'd want to save money and run things more efficiently with a long running server on an EC2 instance so you only have to pay for the instance.
And initially for an MVP you can probably just run a monolithic architecture, but as you start to build out more features and more complexity you will need a microservices architecture to organize the complexity and keep it scalable, secure, and observable.
The goal as a startup engineer is to find that right balance between a solution that solves a problem quickly and efficiently in the short term, versus investing the effort in a solution that will last for a long time.
The stack you listed out is a great stack, and one of my favorite "endgame" stacks. You can't go too wrong with it except that it may take more time to set it up initially, and that extra engineer time initially can be hard for a startup that is seeking investment, etc. For this reason in a startup there is nothing wrong with standing up a quick prototype that runs on Lambda, or throwing together a monolith in the short term to get an MVP to test out your idea.
It sounds like you are on the right track with investigating architectures though and I'm confident you'll be able to make the right choice for your particular scenario!
I don't know of a specific example that has all the the things you mentioned already, but I'm confident you'll be able to assemble the pieces yourself from these:
http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/nodejs-dynamodb-tutorial.html
https://github.com/awslabs/lambda-refarch-webapp (Lambda but much code will remain the same)
Good luck and have fun!