zalando/logbook

[logbook-ktor-client] Dead locks on large request/response bodies

grassehh opened this issue · 2 comments

grassehh commented

Description

Using logbook-ktor-client plugin, if you send/receive request/response with large string bodies (approx ~4000+ characters), the request will never be sent or the response will never be received.
The issue seems to only occur when using streamed content serialized/deserialized through jackson.

Expected Behavior

Request/response with large content should be sent/received

Actual Behavior

Requests threads will be stuck and will never be sent.
Responses will timeout.

Possible Fix

Fix available here: https://github.com/grassehh/issues/blob/main/zalando-logbook-1822/src/test/kotlin/com/grassehh/logbook/LogbookClient.kt

In summary, as for the request, the root cause seems to reside in the Unconfined dispatcher used in the readBytes method. I don't have a clear explaination of what it does not work, but using the default or IO seems to fix it.

As for the response, after comparing with the official Ktor ObserverResponse plugin, simply executing the process inside a coroutine seems to fix the issue.
Another point to not is that it is recommanded to call the dicard() method on the ByteReadChannel once we are done with it.

In the fix, I eventually also placed the request process inside a coroutine aswell improve overall request/response times.

Steps to Reproduce

The test will fail. If you replace the LogbookClient with com.grassehh.logbook.LogbookClient, the test will pass.

Your Environment

  • Ktor 2.3.10
  • Logbook 3.8.0
kasmarian commented

Thank you for the well defined description of the issue, @grassehh ! It turned out that the only part that mattered was to use the coroutine scope from scope: HttpClient. I added the changes in #1832

kasmarian commented

The fix will be in the next release, thank you for the detailed issue

  • ❤️1