sopel/botocross

Add graceful handling of region connection issues.

sopel opened this issue · 1 comments

A currently ongoing AWS outage in sa-east-1, which seems to affect various services (esp. those responsible for monitoring and dealing with things) yields cross region scripts to stall when connecting to sa-east-1, and eventually fail with:

socket.error: [Errno 104] Connection reset by peer.

This obviously asks for a parallel execution per region, which of course would be a good thing anyway to speed up cross region operations like backups or CloudFormation stack operations.

NOTE: The problem might be amplified due to an apparent upstream issue with socket connection timeout handling, see boto/boto#564 for details - unfortunately, applying the boto config file solution suggested in comment 3921067 hasn't made any difference right away (not thoroughly investigated though).

👍 for this solution