Add jdbc support for Oracle
teriiehina opened this issue · 42 comments
Hi,
it would really help advocate for Quarkus in my company if it was possible to use it with an Oracle database.
I didn't find any information on how to add support for a new jdbc driver and I'm pretty noob on this. I quickly had a look at the jdbc extensions but I didn't find a common pattern.
If somebody could give me a hint or two on how to add support for the Oracle driver, I will be happy to contribute :)
If you are satisfied with the JVM you can simply add the JDBC driver to your pom.
In the meanwhile I cloned a fork of Quarkus that contains a draft of the JDBC Extension:
https://github.com/Sanne/quarkus/tree/Oracle
but still is unable to native compile.
thank you @masini for your answer.
It looks like you are working on this feature so if you need any help (even some alpha-testing) please let me know.
Hi @masini @teriiehina . Any ideas if the native oracle extension now works ?
I'm interested too for a oracle jdbc 'native' solution
I'm interested too for Oracle native solution...
Does someone have more info on this?
Hi,
sorry, I found a way to use Oracle in a native executable and forgot to share it here 😅 I have been using two natiive applications in production for one month 🎉 , using the followings settings.
It is a complete solution for me so no problem if this issue is closed :)
EDIT : added the configuration for the quarkus-maven-plugin related to Oracle JDBC driver
EDIT 2 : added the reflection_config.json
pom.xml
<dependency>
<groupId>com.oracle</groupId>
<artifactId>ojdbc8</artifactId>
<version>12.2.0.1</version>
</dependency>
and
<plugin>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-maven-plugin</artifactId>
<version>${quarkus.version}</version>
<executions>
<execution>
<goals>
<goal>native-image</goal>
</goals>
<configuration>
<enableHttpUrlHandler>true</enableHttpUrlHandler>
<additionalBuildArgs>
<buildArg>-H:+ReportUnsupportedElementsAtRuntime</buildArg>
<buildArg>-H:+ReportExceptionStackTraces</buildArg>
<buildArg>-H:ReflectionConfigurationFiles=classes/reflection_config.json</buildArg>
<buildArg>-H:IncludeResourceBundles=oracle.net.jdbc.nl.mesg.NLSR</buildArg>
<buildArg>-H:EnableURLProtocols=http,https</buildArg>
<buildArg>-H:+JNI</buildArg>
<buildArg>--enable-all-security-services</buildArg>
<buildArg>--allow-incomplete-classpath</buildArg>
<buildArg>--initialize-at-run-time=org.hibernate.jpa.HibernatePersistenceProvider</buildArg>
<buildArg>--initialize-at-run-time=org.hibernate.secure.internal.StandardJaccServiceImpl</buildArg>
<buildArg>--initialize-at-run-time=oracle.jdbc.driver.BlockSource</buildArg>
<buildArg>--initialize-at-run-time=oracle.jdbc.driver.BlockSource$ThreadedCachingBlockSource$BlockReleaser</buildArg>
<buildArg>--initialize-at-run-time=oracle.jdbc.driver.NamedTypeAccessor$XMLFactory</buildArg>
<buildArg>--initialize-at-run-time=oracle.jdbc.driver.OracleTimeoutThreadPerVM</buildArg>
<buildArg>--initialize-at-run-time=oracle.jdbc.driver.SQLUtil$XMLFactory</buildArg>
<buildArg>--initialize-at-run-time=oracle.net.nt.TimeoutInterruptHandler</buildArg>
</additionalBuildArgs>
</configuration>
</execution>
</executions>
</plugin>
applications.properties
quarkus.datasource.url = jdbc:oracle:thin:@<#mydatabase.pf#>:1521:<#schema#>
quarkus.datasource.driver = oracle.jdbc.OracleDriver
quarkus.datasource.username = <#username#>
quarkus.datasource.password = <#password#>
quarkus.hibernate-orm.dialect = org.hibernate.dialect.Oracle9iDialect
Dockerfile
##### 1. BUILD THE NATIVE APP #####
FROM quay.io/quarkus/centos-quarkus-maven:19.1.1 AS build
COPY src /usr/src/app/src
COPY pom.xml /usr/src/app
COPY settings.xml /usr/src/app
USER root
RUN chown -R quarkus /usr/src/app
USER quarkus
RUN mvn -s /usr/src/app/settings.xml -f /usr/src/app/pom.xml -Pnative clean package
##### 2. EMBED IN RUNTIME CONTAINER #####
FROM myprivateregistry/openjdk8slim-w-containerpilot:20180905-1047
ARG ENV
COPY containerpilot/conf-${ENV}.json5 /etc/containerpilot.json5
COPY --from=build /usr/src/app/target/*-runner /work/application
RUN chmod 775 /work/application
EXPOSE 18008
CMD /usr/local/bin/containerpilot -config /etc/containerpilot.json5
reflection_config.json
[
{
"name" : "java.lang.Class",
"allDeclaredConstructors" : true,
"allPublicConstructors" : true,
"allDeclaredMethods" : true,
"allPublicMethods" : true
},
{
"name" : "java.util.Calendar[]",
"allDeclaredConstructors" : true,
"allPublicConstructors" : true,
"allDeclaredMethods" : true,
"allPublicMethods" : true
},
{
"name" : "java.net.URL[]",
"allDeclaredConstructors" : true,
"allPublicConstructors" : true,
"allDeclaredMethods" : true,
"allPublicMethods" : true
},
{
"name" : "java.io.File[]",
"allDeclaredConstructors" : true,
"allPublicConstructors" : true,
"allDeclaredMethods" : true,
"allPublicMethods" : true
},
{
"name" : "oracle.jdbc.driver.T4CDriverExtension",
"allDeclaredConstructors" : true,
"allPublicConstructors" : true,
"allDeclaredMethods" : true,
"allPublicMethods" : true
}, {
"name" : "org.hibernate.dialect.Oracle8iDialect",
"allDeclaredConstructors" : true,
"allPublicConstructors" : true,
"allDeclaredMethods" : true,
"allPublicMethods" : true
}, {
"name" : "org.hibernate.dialect.Oracle10gDialect",
"allDeclaredConstructors" : true,
"allPublicConstructors" : true,
"allDeclaredMethods" : true,
"allPublicMethods" : true
}, {
"name" : "oracle.sql.AnyDataFactory"
}, {
"name" : "oracle.sql.TypeDescriptorFactory"
}, {
"name" : "oracle.jdbc.driver.OracleDriver",
"allDeclaredConstructors" : true,
"allPublicConstructors" : true,
"allDeclaredMethods" : true,
"allPublicMethods" : true
}, {
"name" : "org.hibernate.boot.model.naming.ImplicitNamingStrategyJpaCompliantImpl",
"methods" : [{ "name" : "<init>", "parameterTypes" : [] }]
}
]
Hi!
Thank you for the explication @teriiehina; but I have an issue using quarkus 0.21.2 or 0.21.1 and graalvm 19.1.1
May I know which versions are you using? I will appreciate any hint.
Error: Class that is marked for delaying initialization to run time got initialized during image building: oracle.jdbc.driver.BlockSource. Try marking this class for build-time initialization with --initialize-at-build-time=oracle.jdbc.driver.BlockSource com.oracle.svm.core.util.UserError$UserException: Class that is marked for delaying initialization to run time got initialized during image building: oracle.jdbc.driver.BlockSource. Try marking this class for build-time initialization with --initialize-at-build-time=oracle.jdbc.driver.BlockSource
Hi @codecr, sorry for not giving the details.
I'm indeed using the quarkus 0.21.1 and GraalVM 19.1.1.
The solution is giving in the error log:
Try marking this class for build-time initialization with --initialize-at-build-time=oracle.jdbc.driver.BlockSource
If you look at the quarkus-maven-plugin plugin configuration in my pom.xml example, you will see the <additionalBuildArgs> needed.
hope it helps
Hi @teriiehina ,
Actually my pom is identical to your suggestion but if try to switch to --initialize-at-build-time I got another set of errors.
OK. sorry, I won't be of much help.
@teriiehina I'm trying the same as you but getting a "ClassNotFoundException: oracle.jdbc.OracleTypes" even after registering it for reflection.
I noticed you have the argument:
-H:ReflectionConfigurationFiles=classes/reflection_config.json
This is pointing to a file that you did not provide... Could you please provide me with the contents of that file? Or even better provide all your project source code?
Thanks
@Nunalves you are right, I forgot to put the reflection_config.json, sorry about that. I added with the others file contents in my previous response for clarity.
Just a heads-up that the Oracle JDBC drivers are in the central Maven repository: https://medium.com/@kuassimensah/oracle-jdbc-drivers-on-maven-central-64fcf724d8b
I'd love to work on this issue by providing a new Quarkus extension. @gsmet are there any legal issues regarding the Oracle JDBC driver in the Maven repository and Quarkus?
PR created in #4054 based on @Sanne's amazing job in Sanne@29025e7 and @teriiehina's comment in #1658 (comment). I'd appreciate if someone could give it a try:)
First of all, great work @gastaldi ! 👍
Does anyone know the release timeframe for this driver? Looks like an essential one if you want to adopt it in bigger enterprises.
Hello People, is there any official support in quarkus native build of Oracle JDBC?
Surprisingly that there's still no official support for Oracle JDBC. I really want to try out Quarkus in production but this issue is preventing me from doing this.
@duclm2609 you can simply add the Oracle JDBC driver in your pom.xml and it should work fine (unless you compile to native).
Well, not at all, you cannot create a native image with Oracle JDBC.
That's what I meant with the (unless you compile to native) 😉
Hello everyone, For those who need to use Oracle and would like a native image, I got my Quarkus application working out of the box as a native image with Oracle using the Helidon Oracle JDBC drivers. It's a "JDBC driver for Oracle database with native-image support". Didn't need any extra native-image build args nor any reflect-config.json. It's open source and distributed under Apache 2.0 license. I just added the following in my pom.xml (no direct dependency to Oracle jdbc):
<dependency>
<groupId>io.helidon.integrations.db</groupId>
<artifactId>ojdbc</artifactId>
<version>2.1.0</version>
</dependency>
It uses the ojdbc10 version 19.3.0.0 behind. If there are no technical issues or problem with licensing, maybe we could have a Quarkus Oracle extension using those driver instead (at least while waiting for the Oracle drivers release)? Or figure out how they got it working with native-image support?
Anyway, hope it helps.
IMHO we may still run into licensing issues since the Oracle's JDBC driver is resolved transitively. @Sanne?
IMHO we may still run into licensing issues since the Oracle's JDBC driver is resolved transitively. @Sanne?
Yes, possibly. I don't have deep knowledge about licenses, that's why I mentioned it :). I just suggested it in case it might help.
If a Quarkus extension is not possible with those drivers, I guess people can still use them with the following in their application.properties
quarkus.datasource.db-kind=other
quarkus.datasource.jdbc.driver=oracle.jdbc.driver.OracleDriver
It might help those who would want to use Quarkus but need a native image and are stuck with an Oracle DB. Unless of course those drivers pose any licensing issues themselves.
I don't think it would be a problem to depend on an helidon component (or anything else that is in Maven Central for that matter), but considering the next Oracle JDBC driver is going to be released soon - and is expected to feature GraalVM native-image support - I think I'd rather wait another couple of weeks.
I don't think it would be a problem to depend on an helidon component (or anything else that is in Maven Central for that matter), but considering the next Oracle JDBC driver is going to be released soon - and is expected to feature GraalVM native-image support - I think I'd rather wait another couple of weeks.
Sure, fair enough :). I guess the solution with Helidon drivers and quarkus.datasource.db-kind=other is still good to know for those (like me) who couldn't afford to wait another couple of weeks before the Quarkus solution was abandoned for production due to lack of native-image with Oracle DB. I'll use it and stay tuned for the official release and the Quarkus extension ;)
@loicmarchal this is very nice as a workaround.
Please be aware afaics this dependency has a compile dependency on https://mvnrepository.com/artifact/com.oracle.ojdbc/ojdbc10/19.3.0.0. Thus I'm surprised Helidon is able to ship this under ASL2 if they are literally reshading the oracle driver jar.
IANAL, but just be aware that you are most likey shipping bits that are under Oracle Free Use Terms and Conditions and not just ASL2.
Thanks @maxandersen, that's a good point! I was also unsure about the ASL2 since it's shipped with Oracle drivers, but again I really don't know much about all the legal aspects of licenses. Better to have it in mind though.
Also, I checked the jar that comes with the https://mvnrepository.com/artifact/io.helidon.integrations.db/ojdbc/2.1.0 dependency and there is only a transitive dependency to the Oracle ojdbc10 (19.3.0.0) and some config files. So I removed the dependency to Helidon JDBC drivers and used the official Oracle JDBC drivers with the same config as in the Helidon jar and my Quarkus native-image works just fine (compiled with the mvn package -Pnative). So that's another option if one wants to only have dependencies to the official Oracle drivers. Helidon guys did a great job figuring out how to configure the native-image compiler to work with Oracle drivers I must say :)
yes. You'll notice we've done similar for those with non-propritary licenses :)
As Sanne said we've worked with oracle to make the next release of oracle jdbc drivers native friendly.
You guys did an amazing job :) I was initially compiling to native images manually, but had to write the bash scripts, the Dockerfiles and sometimes had to run the native-image-agent to fix the config for the reflection. It's nice having it out of the box using Quarkus (and even the performances on the JVM are great, although I still wanted a native-image).
I first used Postgres for my own project, so indeed you guys did a great job integrating those non-proprietary :) The organization I'm helping now is using Oracle in production so I had to find a solution (that's how I found the workaround I shared on this thread). It's great you guys worked with Oracle to get it native friendly, thanks 👍 I'll change my workaround to a native Quarkus extension as soon as it's available ;)
are literally reshading the oracle driver jar.
they are not - I misread their setup. Its a wrapper jar; still triggers oracle license in use though.
Thus just something to be aware off.
In fact, the Helidon dependency doesn't pull any custom or modified driver. It's literally just the official Oracle JDBC driver (19.3.0.0) wrapped with a config file helping the native-image initialize the right classes at build/run time. That's why, as I mentioned in my comment here, you can use the Oracle driver directly. That's what I've done in my project and it works fine. I have
<dependency>
<groupId>com.oracle.database.jdbc</groupId>
<artifactId>ojdbc10</artifactId>
<version>19.7.0.0</version>
</dependency>
and the right quarkus.native.additional-build-args in my application.properties with a reflect-config.json and a resource-config.json. I don't have any more dependencies to the Helidon component.
I'm using Kotlin, Hibernate and Panache. It compiles and runs perfectly in Docker. IMHO it's an acceptable solution for those who have to use Oracle in production now, while waiting for the release of the next Oracle JDBC driver (hopefully with support for native-image OOB) and the Quarkus extension.
I just saw on Maven-Central that the new JDBC drivers from Oracle have been released in version 21 with GraalVM support (see here) yesterday. This seems to be the "regular", blocking JDBC driver, but as stated here, there should be a reactive version as well.
However, I tried to use them directly in a Quarkus v1.10.5 application without any modification like providing a reflect-config.json or resource-config.json, but compiling to native image using GraalVM v.20.2 aborts with an error:
Warning: class initialization of class oracle.jdbc.driver.SQLUtil$XMLFactory failed with exception java.lang.NoClassDefFoundError: oracle/xdb/XMLType. This class will be initialized at run time because option --allow-incomplete-classpath is used for image building. Use the option --initialize-at-run-time=oracle.jdbc.driver.SQLUtil$XMLFactory to explicitly request delayed initialization of this class.
Warning: class initialization of class oracle.jdbc.driver.NamedTypeAccessor$XMLFactory failed with exception java.lang.NoClassDefFoundError: oracle/xdb/XMLType. This class will be initialized at run time because option --allow-incomplete-classpath is used for image building. Use the option --initialize-at-run-time=oracle.jdbc.driver.NamedTypeAccessor$XMLFactory to explicitly request delayed initialization of this class.
[oracle-jdbc-1.0.0-SNAPSHOT-runner:66381] analysis: 31.017,35 ms, 2,36 GB
Error: Classes that should be initialized at run time got initialized during image building:
oracle.jdbc.OracleDriver the class was requested to be initialized at run time (from the command line). To see why oracle.jdbc.OracleDriver got initialized use -H:+TraceClassInitialization
java.sql.DriverManager the class was requested to be initialized at run time (from the command line). To see why java.sql.DriverManager got initialized use -H:+TraceClassInitialization
com.oracle.svm.core.util.UserError$UserException: Classes that should be initialized at run time got initialized during image building:
oracle.jdbc.OracleDriver the class was requested to be initialized at run time (from the command line). To see why oracle.jdbc.OracleDriver got initialized use -H:+TraceClassInitialization
java.sql.DriverManager the class was requested to be initialized at run time (from the command line). To see why java.sql.DriverManager got initialized use -H:+TraceClassInitialization
at com.oracle.svm.core.util.UserError.abort(UserError.java:68)
at com.oracle.svm.hosted.classinitialization.ConfigurableClassInitialization.checkDelayedInitialization(ConfigurableClassInitialization.java:526)
at com.oracle.svm.hosted.classinitialization.ClassInitializationFeature.duringAnalysis(ClassInitializationFeature.java:227)
at com.oracle.svm.hosted.NativeImageGenerator.lambda$runPointsToAnalysis$8(NativeImageGenerator.java:732)
at com.oracle.svm.hosted.FeatureHandler.forEachFeature(FeatureHandler.java:70)
at com.oracle.svm.hosted.NativeImageGenerator.runPointsToAnalysis(NativeImageGenerator.java:732)
at com.oracle.svm.hosted.NativeImageGenerator.doRun(NativeImageGenerator.java:555)
at com.oracle.svm.hosted.NativeImageGenerator.lambda$run$0(NativeImageGenerator.java:468)
at java.base/java.util.concurrent.ForkJoinTask$AdaptedRunnableAction.exec(ForkJoinTask.java:1407)
at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)
at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)
at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)
Error: Image build request failed with exit status 1
Unfortunately it seems the Oracle driver is including explicit GraalVM directives, but these are not compatible with other flags that other frameworks require. You can check the parameters they impose by looking into the resources in /META-INF/native-image/ (bundled within the driver).
In particular I noticed these as being problematic:
--allow-incomplete-classpathtends to hide serious problems at build time. And since it's a global flag, it's unwelcome as even if the Oracle team could claim that it's "ok" to use on their driver, this will hide problems in your whole application - and all other dependencies. I don't want this flag enabled if I have to be able to help others with problems, and in this case it's making it impossible to disable it.- Options such as
--enable-all-security-servicesand-H:EnableURLProtocols=http,-H:EnableURLProtocols=shttpshould really be conditional : depending on IFF their respective functionality is needed. This works against the minimalist approach of Quarkus, and doesn't allow us to take advantage of our Analysis phase. --initialize-at-run-time=oracle.jdbc.OracleDriver,java.sql.DriverManagerthis is a showstopper. They might have good reasons to force a runtime initialization of their driver (but I can't tell), however there are usually better alternatives, such as re-initializing just some fields. Quarkus in its current design is loading the driver class upfront, and since I can't change this flag it won't compile (as you noticed).
We will need :
- Reach out to the Oracle driver team in hope they could minimize these flags. I would prefer them to document recommendations rather than enforcing system-wide flags.
- Have the GraalVM team give us a way to override such flags; I predicted this problem would happen a long time ago: oracle/graal#1884
- or, we could explore some way in Quarkus to strip jars from their GraalVM resources, so to be able to parse/enrich/override them.
Both the Oracle Database team and the GraalVM team have been very helpful; it's not as polished as I'd hoped for but this works:
To be more polished IMO it should not rely on the above flags ^, but the specific order of the directives we generate seem effective enough to override the most critical ones. In parallel the Oracle JDBC team has suggested they will likely revisit the flags the jar is embedding, but that will have to wait for their next significant release.
The other flag that still concerns me is the --allow-incomplete-classpath; in my experience this tends to hide more problematic issues, but it seems it wasn't too bad to debug even with that flag being enforced so far, so hopefully I'm wrong about that and it might not be such a big deal.
I have been struggle with this problem, I have a service with Quarkus, Apache Camel and Jdbc Oracle when I try run in native mode receive Error: Classes that should be initialized at run time got initialized during image building:
oracle.jdbc.OracleDriver the class was requested to be initialized at run time (from the command line). To see why oracle.jdbc.OracleDriver got initialized use --trace-class-initialization=oracle.jdbc.OracleDriver
java.sql.DriverManager the class was requested to be initialized at run time (from the command line). To see why java.sql.DriverManager got initialized use --trace-class-initialization=java.sql.DriverManager
@DouglasGo8 you may want to report on https://github.com/apache/camel-quarkus/issues/new including proper steps to reproduce.
yes please report to Camel. @ppalaga let me know if you need help.. this was tricky.
It's probably related to some other component which is triggering a chain initialization at static init time; you'd either have to change the code to avoid this, or have such other component also reconfigured for runtime initialization, as I won't be able to change the Oracle driver code.
Reported at apache/camel-quarkus#2448