leafo/pgmoon

Add locking to query functions

leafo opened this issue · 1 comments

leafo commented

Since it's common to yield on network actions, in order to prevent multiple coroutines issuing queries at the same time a locking mechanism should be implemented over the duration of the query function.

Few notes:

  • Care should be taken to not leave connection object in locked state if some part of the query fails (although maybe the connection should be aborted when that happens anyway)
  • Consider a public interface for incrementing/decrementing the lock for clients that wish to add their own locking across multiple statements
  • It should probably be opt-in to not impact existing behavior with downsides of locking
leafo commented

Upon further testing, queries appear to be processed and returned in sequential order. I may do more stress test in the future to try to expose any issues.

If you send in 20 queries at once, they are returned in the order they were sent, which means that the correct response will be returned to the correct query. The throughput will just be limited to one query at a time. The query methods use an atomic send operation: they send the entire message as a single string on the socket. If we ever have a query operation that uses multiple socket write then we have to be careful that they might yield and another thread could try to send data in-between corrupt the request. Reads in pgmoon are not atomic but it doesn't matter since a single postgres connection will never returned an interleaved response as it would be impossible to parse.

Because of this, I don't believe it's necessary to add locking, it works as expected. A pgmoon connection is safe for concurrent access.

See https://github.com/leafo/lapis-concurrency-test for progress on implement concurrent connection pooling for Lapis & cqueues