GCS Connector fails with StackOverflowError during accessing hadoop credentials
daichi-m opened this issue · 0 comments
daichi-m commented
Similar to #219 - while accessing the Hadoop credential through spark, there is still a StackOverflowError.
Following spark properties are set during submitting spark job:
"spark.hadoop.javax.jdo.option.ConnectionDriverName": "com.mysql.jdbc.Driver",
"spark.hadoop.hadoop.security.credential.provider.path": "jceks://gs@bucket-name/path/to/credentials.jceks",
"spark.hadoop.javax.jdo.option.ConnectionURL": "jdbc:mysql://mysql-server.somedomain.net:3306/metastore",
"spark.hadoop.javax.jdo.option.ConnectionUserName": "username"Following is the stackoverflow error while executing the job:
Exception in thread "main" java.lang.StackOverflowError
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.getSubject(Subject.java:297)
at org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:577)
at org.apache.hadoop.fs.FileSystem$Cache$Key.(FileSystem.java:3565)
at org.apache.hadoop.fs.FileSystem$Cache$Key.(FileSystem.java:3555)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:3392)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:486)
at org.apache.hadoop.fs.Path.getFileSystem(Path.java:365)
at org.apache.hadoop.security.alias.JavaKeyStoreProvider.initFileSystem(JavaKeyStoreProvider.java:89)
at org.apache.hadoop.security.alias.AbstractJavaKeyStoreProvider.(AbstractJavaKeyStoreProvider.java:85)
at org.apache.hadoop.security.alias.JavaKeyStoreProvider.(JavaKeyStoreProvider.java:49)
at org.apache.hadoop.security.alias.JavaKeyStoreProvider.(JavaKeyStoreProvider.java:41)
at org.apache.hadoop.security.alias.JavaKeyStoreProvider$Factory.createProvider(JavaKeyStoreProvider.java:100)
at org.apache.hadoop.security.alias.CredentialProviderFactory.getProviders(CredentialProviderFactory.java:73)
at org.apache.hadoop.conf.Configuration.getPasswordFromCredentialProviders(Configuration.java:2386)
at org.apache.hadoop.conf.Configuration.getPassword(Configuration.java:2324)
at com.google.cloud.hadoop.repackaged.gcs.com.google.cloud.hadoop.util.HadoopConfigurationProperty.getPassword(HadoopConfigurationProperty.java:84)
at com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemConfiguration.getGcsOptionsBuilder(GoogleHadoopFileSystemConfiguration.java:490)
at com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemConfiguration.getGcsFsOptionsBuilder(GoogleHadoopFileSystemConfiguration.java:462)
at com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase.createGcsFs(GoogleHadoopFileSystemBase.java:1491)
at com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase.configure(GoogleHadoopFileSystemBase.java:1479)
at com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase.initialize(GoogleHadoopFileSystemBase.java:467)
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3376)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:126)
at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:3425)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:3393)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:486)
at org.apache.hadoop.fs.Path.getFileSystem(Path.java:365)
at org.apache.hadoop.security.alias.JavaKeyStoreProvider.initFileSystem(JavaKeyStoreProvider.java:89)
at org.apache.hadoop.security.alias.AbstractJavaKeyStoreProvider.(AbstractJavaKeyStoreProvider.java:85)
at org.apache.hadoop.security.alias.JavaKeyStoreProvider.(JavaKeyStoreProvider.java:49)
at org.apache.hadoop.security.alias.JavaKeyStoreProvider.(JavaKeyStoreProvider.java:41)
at org.apache.hadoop.security.alias.JavaKeyStoreProvider$Factory.createProvider(JavaKeyStoreProvider.java:100)
at org.apache.hadoop.security.alias.CredentialProviderFactory.getProviders(CredentialProviderFactory.java:73)
at org.apache.hadoop.conf.Configuration.getPasswordFromCredentialProviders(Configuration.java:2386)
at org.apache.hadoop.conf.Configuration.getPassword(Configuration.java:2324)
at com.google.cloud.hadoop.repackaged.gcs.com.google.cloud.hadoop.util.HadoopConfigurationProperty.getPassword(HadoopConfigurationProperty.java:84)
...
at com.google.cloud.hadoop.repackaged.gcs.com.google.cloud.hadoop.util.HadoopConfigurationProperty.getPassword(HadoopConfigurationProperty.java:84)
at com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemConfiguration.getGcsOptionsBuilder(GoogleHadoopFileSystemConfiguration.java:490)
at com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemConfiguration.getGcsFsOptionsBuilder(GoogleHadoopFileSystemConfiguration.java:462)
at com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase.createGcsFs(GoogleHadoopFileSystemBase.java:1491)
at com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase.configure(GoogleHadoopFileSystemBase.java:1479)
at com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase.initialize(GoogleHadoopFileSystemBase.java:467)
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3376)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:126)
Issue is seen in Cloud Dataproc version 2.0.x with Cloud Storage Connector version hadoop3-2.2.16