ActiveCampaign/postmark-java

Spring Boot random "SSLException: Connection Reset" with Java 8

amwakefield-olson opened this issue · 7 comments

Hello,

We have an app that is utilizing this app to make an api call to get message details from the postmark email that is giving us an SSLException: Connection Reset. Upon doing more digging it looks like there is a potential we are using all of our "Pools" and it is causing these api calls to fail.

This is a article on stack I found that seems to be relevant to this issue.

https://stackoverflow.com/questions/64810671/spring-boot-random-sslexception-connection-reset-in-kubernetes-with-jdk11

Here is the link provided in the answer from Jude Niroshan, https://howtodoinjava.com/spring-boot2/resttemplate/resttemplate-httpclient-java-config/

Here is a stack trace for what we are seeing on our end up until we blow up in our application.

Exception:
javax.net.ssl.SSLException
Message:
Connection reset
Stacktrace:
sun.security.ssl.Alert.createSSLException(Alert.java:127)
sun.security.ssl.TransportContext.fatal(TransportContext.java:331)
sun.security.ssl.TransportContext.fatal(TransportContext.java:274)
sun.security.ssl.TransportContext.fatal(TransportContext.java:269)
sun.security.ssl.SSLTransport.decode(SSLTransport.java:138)
sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1401)
sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1309)
sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:440)
org.apache.hc.client5.http.ssl.SSLConnectionSocketFactory.executeHandshake(SSLConnectionSocketFactory.java:303)
org.apache.hc.client5.http.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:275)
org.apache.hc.client5.http.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:251)
org.apache.hc.client5.http.impl.io.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:181)
org.apache.hc.client5.http.impl.io.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:447)
org.apache.hc.client5.http.impl.classic.InternalExecRuntime.connectEndpoint(InternalExecRuntime.java:162)
org.apache.hc.client5.http.impl.classic.InternalExecRuntime.connectEndpoint(InternalExecRuntime.java:172)
org.apache.hc.client5.http.impl.classic.ConnectExec.execute(ConnectExec.java:142)
org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
org.apache.hc.client5.http.impl.classic.ProtocolExec.execute(ProtocolExec.java:192)
org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
org.apache.hc.client5.http.impl.classic.HttpRequestRetryExec.execute(HttpRequestRetryExec.java:96)
org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
org.apache.hc.client5.http.impl.classic.ContentCompressionExec.execute(ContentCompressionExec.java:152)
org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
org.apache.hc.client5.http.impl.classic.RedirectExec.execute(RedirectExec.java:115)
org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
org.apache.hc.client5.http.impl.classic.InternalHttpClient.doExecute(InternalHttpClient.java:170)
org.apache.hc.client5.http.impl.classic.CloseableHttpClient.execute(CloseableHttpClient.java:245)
org.apache.hc.client5.http.impl.classic.CloseableHttpClient.execute(CloseableHttpClient.java:188)
org.apache.hc.client5.http.impl.classic.CloseableHttpClient.execute(CloseableHttpClient.java:162)
com.postmarkapp.postmark.client.HttpClient.execute(HttpClient.java:90)
com.postmarkapp.postmark.client.HttpClientHandler.execute(HttpClientHandler.java:61)
com.postmarkapp.postmark.client.HttpClientHandler.execute(HttpClientHandler.java:44)
com.postmarkapp.postmark.client.ApiClient.getTemplate(ApiClient.java:114)

Any help is much appreciated! I look forward to working with y'all to help debug this.

Allan

Hi @amwakefield-olson

thank you for reporting this, I will see to check it out. Could you share maybe more information how this happens? Do you run many Postmark http client requests when it does, and how many, etc?

The more details you could share, the more we can help out.

Thanks

@ibalosh - thanks for responding, apologies it took me a minute to get back. When we send or have sent email's, we go grab that information from your API and populate a message to a queue. We are calling your API (this is approximate apologies) 11,000 times a day to get Information from email transactions. I unfortunately do not have a guess on how many times as this is handled by a listener. If you have time I'd be more than happy to connect with you to give you a better understanding.

Thanks,

Allan

Hi @amwakefield-olson

I did not get a chance yet to take a look at this, but will see to investigate it bit more in week or so. Meanwhile if you get any additional details before that, please share them here.

Igor

Apologies - I am still looking to work with y'all at some point on this. I unfortunately don't have anymore details that I can share as far as amount of failures. It looks like we've had 308 failures in the last 90 days, we also have it built where we re-try to send the email 5 times before we send it to a log. If we could touch base at some point to go over this that would be great!

Thanks again,

Allan

Hi @amwakefield-olson

we have released a new version of the library 1.11.1 , which contains newer version of http client and some configuration tune ups. Let us know if it helps with using the client.

@ibalosh

Thank you! I will pull them into my services and monitor!

Allan