"Too many open files" when building Keycloak 19.0.1 image with singular-keycloak-database-federation provider

Question

"Too many open files" when building Keycloak 19.0.1 image with singular-keycloak-database-federation provider

Opened this issue a year ago · 3 comments

Hi, I'm having an issue when I try to execute "build command for keycloak 19.0.1.

First, I'm copying all jars to "providers" directory with following command:
buildah copy $buildercontainer company/keycloak/providers/singular_keycloak_database_federation "/opt/keycloak/providers/"

Then I execute build command like this:
buildah run $buildercontainer -- /opt/keycloak/bin/kc.sh "build --health-enabled=true --metrics-enabled=true --db postgres --http-relative-path /auth"

And I'm getting following errors:

ERROR: Failed to run 'build' command.
ERROR: io.quarkus.builder.BuildException: Build failure: Build failed due to errors
[error]: Build step io.quarkus.deployment.index.ApplicationArchiveBuildStep#build threw an exception: java.lang.RuntimeException: Failed to process /opt/keycloak/lib/../providers/oraclepki-19.3.0.0.jar
at io.quarkus.deployment.index.ApplicationArchiveBuildStep$2.apply(ApplicationArchiveBuildStep.java:315)
at io.quarkus.deployment.index.ApplicationArchiveBuildStep$2.apply(ApplicationArchiveBuildStep.java:309)
at java.base/java.util.HashMap.computeIfAbsent(HashMap.java:1134)
at io.quarkus.deployment.index.ApplicationArchiveBuildStep.handleJarPath(ApplicationArchiveBuildStep.java:309)
at io.quarkus.deployment.index.ApplicationArchiveBuildStep.createApplicationArchive(ApplicationArchiveBuildStep.java:213)
at io.quarkus.deployment.index.ApplicationArchiveBuildStep.scanForOtherIndexes(ApplicationArchiveBuildStep.java:154)
at io.quarkus.deployment.index.ApplicationArchiveBuildStep.build(ApplicationArchiveBuildStep.java:105)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at io.quarkus.deployment.ExtensionLoader$2.execute(ExtensionLoader.java:882)
at io.quarkus.builder.BuildContext.run(BuildContext.java:277)
at org.jboss.threads.ContextHandler$1.runWith(ContextHandler.java:18)
at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2449)
at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:[147]
at java.base/java.lang.Thread.run(Thread.java:829)
at org.jboss.threads.JBossThread.run(JBossThread.java:501)
Caused by: java.io.FileNotFoundException: /opt/keycloak/lib/../providers/oraclepki-19.3.0.0.jar (Too many open files)
at java.base/java.io.RandomAccessFile.open0(Native Method)
at java.base/java.io.RandomAccessFile.open(RandomAccessFile.java:345)
at java.base/java.io.RandomAccessFile.(RandomAccessFile.java:259)
at java.base/java.io.RandomAccessFile.(RandomAccessFile.java:214)
at java.base/java.util.zip.ZipFile$Source.(ZipFile.java:1305)
at java.base/java.util.zip.ZipFile$Source.get(ZipFile.java:1271)
at java.base/java.util.zip.ZipFile$CleanableResource.(ZipFile.java:733)
at java.base/java.util.zip.ZipFile$CleanableResource.get(ZipFile.java:850)
at java.base/java.util.zip.ZipFile.(ZipFile.java:248)
at java.base/java.util.zip.ZipFile.(ZipFile.java:177)
at java.base/java.util.jar.JarFile.(JarFile.java:350)
at java.base/java.util.jar.JarFile.(JarFile.java:321)
at java.base/java.util.jar.JarFile.(JarFile.java:287)
at io.quarkus.deployment.index.IndexingUtil.indexJar(IndexingUtil.java:73)
at io.quarkus.deployment.index.IndexingUtil.indexJar(IndexingUtil.java:56)
at io.quarkus.deployment.index.ApplicationArchiveBuildStep$2.apply(ApplicationArchiveBuildStep.java:313)
... 17 more
ERROR: Build failure: Build failed due to errors
[error]: Build step io.quarkus.deployment.index.ApplicationArchiveBuildStep#build threw an exception: java.lang.RuntimeException: Failed to process /opt/keycloak/lib/../providers/oraclepki-19.3.0.0.jar
at io.quarkus.deployment.index.ApplicationArchiveBuildStep$2.apply(ApplicationArchiveBuildStep.java:315)
at io.quarkus.deployment.index.ApplicationArchiveBuildStep$2.apply(ApplicationArchiveBuildStep.java:309)
at java.base/java.util.HashMap.computeIfAbsent(HashMap.java:1134)
at io.quarkus.deployment.index.ApplicationArchiveBuildStep.handleJarPath(ApplicationArchiveBuildStep.java:309)
at io.quarkus.deployment.index.ApplicationArchiveBuildStep.createApplicationArchive(ApplicationArchiveBuildStep.java:213)
at io.quarkus.deployment.index.ApplicationArchiveBuildStep.scanForOtherIndexes(ApplicationArchiveBuildStep.java:[154]
at io.quarkus.deployment.index.ApplicationArchiveBuildStep.build(ApplicationArchiveBuildStep.java:105)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at io.quarkus.deployment.ExtensionLoader$2.execute(ExtensionLoader.java:882)
at io.quarkus.builder.BuildContext.run(BuildContext.java:277)
at org.jboss.threads.ContextHandler$1.runWith(ContextHandler.java:18)
at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2449)
at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1478)
at java.base/java.lang.Thread.run(Thread.java:829)
at org.jboss.threads.JBossThread.run(JBossThread.java:501)
Caused by: java.io.FileNotFoundException: /opt/keycloak/lib/../providers/oraclepki-19.3.0.0.jar (Too many open files)
at java.base/java.io.RandomAccessFile.open0(Native Method)
at java.base/java.io.RandomAccessFile.open(RandomAccessFile.java:345)
at java.base/java.io.RandomAccessFile.(RandomAccessFile.java:259)
at java.base/java.io.RandomAccessFile.(RandomAccessFile.java:214)
at java.base/java.util.zip.ZipFile$Source.(ZipFile.java:1305)
at java.base/java.util.zip.ZipFile$Source.get(ZipFile.java:1271)
at java.base/java.util.zip.ZipFile$CleanableResource.(ZipFile.java:733)
at java.base/java.util.zip.ZipFile$CleanableResource.get(ZipFile.java:850)
at java.base/java.util.zip.ZipFile.(ZipFile.java:248)
at java.base/java.util.zip.ZipFile.(ZipFile.java:[177]
at java.base/java.util.jar.JarFile.(JarFile.java:350)
at java.base/java.util.jar.JarFile.(JarFile.java:321)
at java.base/java.util.jar.JarFile.(JarFile.java:287)
at io.quarkus.deployment.index.IndexingUtil.indexJar(IndexingUtil.java:73)
at io.quarkus.deployment.index.IndexingUtil.indexJar(IndexingUtil.java:56)
at io.quarkus.deployment.index.ApplicationArchiveBuildStep$2.apply(ApplicationArchiveBuildStep.java:313)
... 17 more
ERROR: Failed to process /opt/keycloak/lib/../providers/oraclepki-19.3.0.0.jar
ERROR: /opt/keycloak/lib/../providers/oraclepki-19.3.0.0.jar (Too many open files)

I don't know what is the issue here. How can I solve it?

Answer 1 · 2023-07-06T13:39:32.000Z

Check your ulimit -n
Increase it to a much larger value if need be.
e.g. see https://www.tutorialspoint.com/fixing-the-too-many-open-files-error-in-linux

Answer 2 · 2023-07-06T15:23:03.000Z

Check your ulimit -n Increase it to a much larger value if need be. e.g. see https://www.tutorialspoint.com/fixing-the-too-many-open-files-error-in-linux

I've tried it already, but hard limit in my case is set to 1024 and I can't set anything above that value.

Answer 3 · 2023-07-06T15:28:55.000Z

Then I don't see any way to solve your problem except to find a way to set it above that value (or ask your admin to do it, if that's the problem), or run the code from another computer (or VM, or Docker) that can have a higher value.