minimaxir/gpt-2-cloud-run

Add rate limiting

minimaxir opened this issue · 2 comments

Since Cloud Run does unauthenticated HTTP requests, it would be good to add a simple rate limit by IP.

Unfortunately there's no simple implementation, and the simple implementations that exist are for Flask only.

Actually rate limiting might be dumb because the limits cannot be shared across containers.

This isn't possible anyways because the user IP is not exposed to the container.