prestodb/presto-hadoop-apache2

java.io.IOException: Filesystem closed

hellozeck opened this issue · 1 comments

hellozeck commented 
com.facebook.presto.spi.PrestoException: Error reading from hdfs://.... at position 3
	at com.facebook.presto.hive.orc.HdfsOrcDataSource.readInternal(HdfsOrcDataSource.java:76)
	at com.facebook.presto.orc.AbstractOrcDataSource.readFully(AbstractOrcDataSource.java:105)
	at com.facebook.presto.orc.CachingOrcDataSource.readCacheAt(CachingOrcDataSource.java:78)
	at com.facebook.presto.orc.CachingOrcDataSource.readFully(CachingOrcDataSource.java:96)
	at com.facebook.presto.orc.CachingOrcDataSource.readFully(CachingOrcDataSource.java:85)
	at com.facebook.presto.orc.StripeReader.readStripeFooter(StripeReader.java:365)
	at com.facebook.presto.orc.StripeReader.readStripe(StripeReader.java:109)
	at com.facebook.presto.orc.OrcRecordReader.advanceToNextStripe(OrcRecordReader.java:446)
	at com.facebook.presto.orc.OrcRecordReader.advanceToNextRowGroup(OrcRecordReader.java:399)
	at com.facebook.presto.orc.OrcRecordReader.nextBatch(OrcRecordReader.java:348)
	at com.facebook.presto.hive.orc.OrcPageSource.getNextPage(OrcPageSource.java:135)
	at com.facebook.presto.hive.HivePageSource.getNextPage(HivePageSource.java:197)
	at com.facebook.presto.operator.ScanFilterAndProjectOperator.processPageSource(ScanFilterAndProjectOperator.java:265)
	at com.facebook.presto.operator.ScanFilterAndProjectOperator.getOutput(ScanFilterAndProjectOperator.java:229)
	at com.facebook.presto.operator.Driver.processInternal(Driver.java:337)
	at com.facebook.presto.operator.Driver.lambda$processFor$6(Driver.java:241)
	at com.facebook.presto.operator.Driver.tryWithLock(Driver.java:614)
	at com.facebook.presto.operator.Driver.processFor(Driver.java:235)
	at com.facebook.presto.execution.SqlTaskExecution$DriverSplitRunner.processFor(SqlTaskExecution.java:622)
	at com.facebook.presto.execution.executor.PrioritizedSplitRunner.process(PrioritizedSplitRunner.java:163)
	at com.facebook.presto.execution.executor.LegacyPrioritizedSplitRunner.process(LegacyPrioritizedSplitRunner.java:23)
	at com.facebook.presto.execution.executor.TaskExecutor$TaskRunner.run(TaskExecutor.java:485)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
	Suppressed: java.lang.RuntimeException: java.io.IOException: Filesystem closed
		at com.google.common.base.Throwables.propagate(Throwables.java:240)
		at com.facebook.presto.hive.orc.OrcPageSource.close(OrcPageSource.java:177)
		at com.facebook.presto.hive.orc.OrcPageSource.closeWithSuppression(OrcPageSource.java:200)
		at com.facebook.presto.hive.orc.OrcPageSource.getNextPage(OrcPageSource.java:154)
		... 14 more
	Caused by: java.io.IOException: Filesystem closed
		at org.apache.hadoop.hdfs.DFSClient.checkOpen(DFSClient.java:808)
		at org.apache.hadoop.hdfs.DFSInputStream.close(DFSInputStream.java:710)
		at java.io.FilterInputStream.close(FilterInputStream.java:181)
		at com.facebook.presto.hive.orc.HdfsOrcDataSource.close(HdfsOrcDataSource.java:55)
		at com.facebook.presto.orc.CachingOrcDataSource.close(CachingOrcDataSource.java:125)
		at com.facebook.presto.orc.OrcRecordReader.close(OrcRecordReader.java:318)
		at com.facebook.presto.hive.orc.OrcPageSource.close(OrcPageSource.java:174)
		... 16 more
Caused by: java.io.IOException: Filesystem closed
	at org.apache.hadoop.hdfs.DFSClient.checkOpen(DFSClient.java:808)
	at org.apache.hadoop.hdfs.DFSInputStream.pread(DFSInputStream.java:1419)
	at org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:1410)
	at org.apache.hadoop.fs.FSInputStream.readFully(FSInputStream.java:78)
	at org.apache.hadoop.fs.FSDataInputStream.readFully(FSDataInputStream.java:107)
	at com.facebook.presto.hive.orc.HdfsOrcDataSource.readInternal(HdfsOrcDataSource.java:64)
	... 24 more

Hi , guys
I found with patch "File system finalizer #23" , when I start presto server first and do some query , It always happened exception like above . The reason I guess when privateCredentials not equals , the filesystem become PhantomReference and do clean , so i think we should cache filesystem .

electrum commented

This should be resolved now. Please re-open if you are still seeing the problem.

  • 👍1