flumk sink hdfs to gcs, all gcs write thread blocked
Maplejw opened this issue · 0 comments
Maplejw commented
version: gcs-connector-hadoop2-2.2.12-shaded.jar
flume write data to gcs by using gcs-connector-hadoop.
jstack the flume proccess. All thread is Blocked, I don't know why and how to resolv it. I have attached the jstack file
Thread 16711: (state = BLOCKED)
- java.lang.Object.wait(long) @bci=0 (Compiled frame; information may be imprecise)
- java.io.PipedInputStream.read() @bci=142, line=326 (Compiled frame)
- java.io.PipedInputStream.read(byte[], int, int) @bci=43, line=377 (Compiled frame)
- com.google.cloud.hadoop.repackaged.gcs.com.google.api.client.util.ByteStreams.read(java.io.InputStream, byte[], int, int) @bci=43, line=172 (Compiled frame)
- com.google.cloud.hadoop.repackaged.gcs.com.google.api.client.googleapis.media.MediaHttpUploader.buildContentChunk() @bci=240, line=610 (Compiled frame)
- com.google.cloud.hadoop.repackaged.gcs.com.google.api.client.googleapis.media.MediaHttpUploader.resumableUpload(com.google.cloud.hadoop.repackaged.gcs.com.google.api.client.http.GenericUrl) @bci=90, line=380 (Compiled frame)
- com.google.cloud.hadoop.repackaged.gcs.com.google.api.client.googleapis.media.MediaHttpUploader.upload(com.google.cloud.hadoop.repackaged.gcs.com.google.api.client.http.GenericUrl) @bci=33, line=308 (Compiled frame)
- com.google.cloud.hadoop.repackaged.gcs.com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(boolean) @bci=71, line=539 (Compiled frame)
- com.google.cloud.hadoop.repackaged.gcs.com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed() @bci=2, line=466 (Compiled frame)
- com.google.cloud.hadoop.repackaged.gcs.com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute() @bci=1, line=576 (Compiled frame)
- com.google.cloud.hadoop.repackaged.gcs.com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call() @bci=11, line=85 (Compiled frame)
- java.util.concurrent.FutureTask.run() @bci=42, line=266 (Compiled frame)
- java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=95, line=1149 (Compiled frame)
- java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=624 (Compiled frame)
- java.lang.Thread.run() @bci=11, line=748 (Compiled frame)