Router - Start Lambda while prior Lambda is stopping

[huntharo]

Motivations

• True graceful shutdown means we are going to be waiting up to, say, a minute for some longer-lived Lambdas to complete all of their in-flight requests when it is time to shutdown
• This means that the Router will be down 1 instance for up to 1 minute for each Lambda that shuts down during a given 1 minute period
• This is VERY noticeable with small quantities of Lambdas
• This shows up as 5 second delays on requests in the best case when 2 of 2 Lambdas both gracefully exit at the same time
• Starting another Lambda while the prior Lambda is still stopping will:
  • Init a few more execution environments which will be helpful during traffic bursts
  • Allow RPS rates to remain steady when invokes are being rotated

To-Do

• Every time a Lambda is invoked, invoke 2 with 1 set to "init-only"?
• Return after init for the lambda marked as init-only
• If the lambda is already init'd, return immediately
• Do not count the init-only lambda as a starting or running instance
• Drop the closing lambda from the lambda invoke count as soon as the close initiates