DatafileClient should specify read/write timeout on urlConnection
pdudchenok opened this issue · 3 comments
Recently we observed significant amount of OOM at our application due to the following cause
java.lang.OutOfMemoryErrorThread.java:730Could not allocate JNI Env
The issue is happening due to limitation on amount of threads per process.
We have noticed that a lot of threads are stuck at SSL_do_handshake.
While inspecting the source code we observed that
urlConnection created here doesn't have configured read/write socket timeout. By default the values are 0 which means infinity.
The full trace:
NativeCrypto.java:-2 com.android.org.conscrypt.NativeCrypto.SSL_do_handshake OpenSSLSocketImpl.java:357 com.android.org.conscrypt.OpenSSLSocketImpl.startHandshakeConnection.java:235 com.android.okhttp.Connection.connectTlsConnection.java:199 com.android.okhttp.Connection.connectSocketConnection.java:172 com.android.okhttp.Connection.connectConnection.java:367 com.android.okhttp.Connection.connectAndSetOwnerOkHttpClient.java:130 com.android.okhttp.OkHttpClient$1.connectAndSetOwnerHttpEngine.java:329 com.android.okhttp.internal.http.HttpEngine.connectHttpEngine.java:246 com.android.okhttp.internal.http.HttpEngine.sendRequestHttpURLConnectionImpl.java:457 com.android.okhttp.internal.huc.HttpURLConnectionImpl.executeHttpURLConnectionImpl.java:126 com.android.okhttp.internal.huc.HttpURLConnectionImpl.connectDelegatingHttpsURLConnection.java:89 com.android.okhttp.internal.huc.DelegatingHttpsURLConnection.connectHttpsURLConnectionImpl.java:-1 com.android.okhttp.internal.huc.HttpsURLConnectionImpl.connectHttpsURLConnectionExtension.java:111 com.newrelic.agent.android.instrumentation.HttpsURLConnectionExtension.connectDatafileClient.java:86 com.optimizely.ab.android.datafile_handler.DatafileClient$1.executeDatafileClient.java:70 com.optimizely.ab.android.datafile_handler.DatafileClient$1.executeClient.java:159 com.optimizely.ab.android.shared.Client.executeDatafileClient.java:115 com.optimizely.ab.android.datafile_handler.DatafileClient.requestDatafileLoader.java:104 com.optimizely.ab.android.datafile_handler.DatafileLoader$1.runThreadPoolExecutor.java:1133 java.util.concurrent.ThreadPoolExecutor.runWorkerThreadPoolExecutor.java:607 java.util.concurrent.ThreadPoolExecutor$Worker.runThread.java:761 java.lang.Thread.run
In other words - socket is connected but nothing responding on the TLS flow and since read/write timeout is infinity the thread is hang on forever.
Due to the fact that new thread is created periodically - sooner or later the app crashes when the backend has TLS problems.
The SDK has to specify read/write timeout instead of using infinity value.
@pdudchenok Thanks for sharing. We'll take a look into it.
Thanks @jaeopt
Do you have any timeline on possible fix?
@pdudchenok the fix has been released in 3.13.3. Let us know if you see more issues. Tx!