[Enhancement] Create a custom class loader for selected OCF connectors

Question

[Enhancement] Create a custom class loader for selected OCF connectors

mandy-chessell opened this issue 4 months ago · 1 comments

Please describe the new behavior that that will improve Egeria

We have seen builds of Egeria where XTDB will not come up due to a mismatch in the levels of lucene in the platform and the level it needs. The issue comes and goes from build to build, machine to machine. So there is a slight timing issue on the order of class loading.

We have seen similar issues with large stacks (such as HBASE under JanusGraph, or working with Hive Metastore).

This is a proposal to add an optional custom class loader to the OCF framework that connector writers can choose to use. It would be used by all repository connectors as a first implementation.

The OCF ConnectorProvider is the place where the connector's classes are loaded which is where the enhancement needs to be made.

Alternatives

No response

Any Further Information?

At the time of raising, the OCF now has a new class loader called IsolatedConnectorClassLoader. There is an extension to ConnectorProviderBase called IsolatedConnectorProviderBase that makes use of the custom class loader. Connector writers just need to inherit from IsolatedConnectorProvider to make use of the custom class loader. Below is the change to the repository connector's connector provider.

It "seems" to work fine for the inmemory connector. However, XTDB has problems in start up. This is the console log

/Library/Java/JavaVirtualMachines/temurin-17.jdk/Contents/Home/bin/java -agentlib:jdwp=transport=dt_socket,address=127.0.0.1:63446,suspend=y,server=n -Dstrict.ssl=false -Dloader.path=lib,extra -Dserver.port=9443 -javaagent:/Users/amandachessell/Library/Caches/JetBrains/IntelliJIdea2023.3/groovyHotSwap/gragent.jar -javaagent:/Users/amandachessell/Library/Caches/JetBrains/IntelliJIdea2023.3/captureAgent/debugger-agent.jar=file:/private/var/folders/rr/c417mvvj1w3bk98490743xz80000gn/T/capture.props -Dfile.encoding=UTF-8 -classpath /Applications/IntelliJ IDEA.app/Contents/lib/idea_rt.jar -jar /Users/amandachessell/Code/ODPi/egeria-oak/egeria/open-metadata-distribution/omag-server-platform/build/unpacked/egeria-platform-5.1-SNAPSHOT-distribution.tar.gz/assembly/platform/omag-server-platform-5.1-SNAPSHOT.jar -cp lib:extra
Connected to the target VM, address: '127.0.0.1:63446', transport: 'socket'
 Project Egeria - Open Metadata and Governance
    ____   __  ___ ___    ______   _____                                 ____   _         _     ___
   / __ \ /  |/  //   |  / ____/  / ___/ ___   ____ _   __ ___   ____   / _  \ / / __    / /  / _ /__   ____ _  _
  / / / // /|_/ // /| | / / __    \__ \ / _ \ / __/| | / // _ \ / __/  / /_/ // //   |  / _\ / /_ /  | /  _// || |
 / /_/ // /  / // ___ |/ /_/ /   ___/ //  __// /   | |/ //  __// /    /  __ // // /  \ / /_ /  _// / // /  / / / /
 \____//_/  /_//_/  |_|\____/   /____/ \___//_/    |___/ \___//_/    /_/    /_/ \__/\//___//_/   \__//_/  /_/ /_/

 :: Powered by Spring Boot (v3.1.4) ::

2024-07-18T10:09:35.133+01:00  INFO 90010 --- [           main] o.o.o.p.springboot.OMAGServerPlatform    : Starting OMAGServerPlatform using Java 17.0.9 with PID 90010 (/Users/amandachessell/Code/ODPi/egeria-oak/egeria/open-metadata-distribution/omag-server-platform/build/unpacked/egeria-platform-5.1-SNAPSHOT-distribution.tar.gz/assembly/platform/omag-server-platform-5.1-SNAPSHOT.jar started by amandachessell in /Users/amandachessell/Code/ODPi/egeria-oak/egeria/open-metadata-distribution/omag-server-platform/build/unpacked/egeria-platform-5.1-SNAPSHOT-distribution.tar.gz/assembly/platform)
2024-07-18T10:09:35.134+01:00  INFO 90010 --- [           main] o.o.o.p.springboot.OMAGServerPlatform    : No active profile set, falling back to 1 default profile: "default"
2024-07-18T10:09:38.805+01:00  INFO 90010 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat initialized with port(s): 9443 (https)
2024-07-18T10:09:41.552+01:00  INFO 90010 --- [           main] o.o.o.p.springboot.OMAGServerPlatform    : Working directory is: /Users/amandachessell/Code/ODPi/egeria-oak/egeria/open-metadata-distribution/omag-server-platform/build/unpacked/egeria-platform-5.1-SNAPSHOT-distribution.tar.gz/assembly/platform
2024-07-18T10:09:41.552+01:00  WARN 90010 --- [           main] o.o.o.p.springboot.OMAGServerPlatform    : Option strict.ssl is set to false! Invalid certificates will be accepted for connection!
2024-07-18T10:09:41.554+01:00  WARN 90010 --- [           main] o.o.o.p.springboot.OMAGServerPlatform    : Java trust store 'javax.net.ssl.trustStore' is null - this is needed by Tomcat - using 'server.ssl.trust-store'
2024-07-18T10:09:43.007+01:00  INFO 90010 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat started on port(s): 9443 (https) with context path ''
2024-07-18T10:09:43.019+01:00  INFO 90010 --- [           main] o.o.o.p.springboot.OMAGServerPlatform    : Started OMAGServerPlatform in 8.695 seconds (process running for 9.294)
2024-07-18T10:09:43.060+01:00  INFO 90010 --- [           main] o.o.o.p.springboot.OMAGServerPlatform    : Startup detected for servers: active-metadata-store,engine-host,integration-daemon,view-server,simple-metadata-store
Thu Jul 18 10:09:43 BST 2024 active-metadata-store Startup OMRS-AUDIT-0064 The Open Metadata Repository Services (OMRS) has initialized the audit log for the Metadata Access Store called active-metadata-store
Thu Jul 18 10:09:43 BST 2024 active-metadata-store Startup SERVER-OPS-0001 The active-metadata-store server is configured with a max page size of 1000
Thu Jul 18 10:09:43 BST 2024 active-metadata-store Startup OMRS-AUDIT-0001 The Open Metadata Repository Services (OMRS) is initializing the subsystems to support a new server
Thu Jul 18 10:09:43 BST 2024 active-metadata-store Startup OMRS-AUDIT-0002 Enterprise access through the Enterprise Repository Services is initializing
Thu Jul 18 10:09:43 BST 2024 active-metadata-store Startup OMRS-AUDIT-0003 The local repository is initializing the metadata collection named active-metadata-store with an id of c5409905-a9b6-4f30-85a2-9602b9b5b4b0
Thu Jul 18 10:09:43 BST 2024 active-metadata-store Startup OMRS-AUDIT-0029 The local repository outbound event manager is initializing
Thu Jul 18 10:09:43 BST 2024 active-metadata-store Startup OMRS-AUDIT-0030 Registering the Local Repository to Local Enterprise event consumer with the local repository outbound event manager
Thu Jul 18 10:09:43 BST 2024 active-metadata-store Startup OMRS-AUDIT-0044 The connector for the local repository is being started in mode Open Metadata Native using connector provider org.odpi.openmetadata.adapters.repositoryservices.xtdb.repositoryconnector.XTDBOMRSRepositoryConnectorProvider and configuration properties {xtdbConfig={xtdb.lucene/lucene-store={db-dir=data/servers/active-metadata-store/repository/xtdb-kv/lucene}, xtdb/tx-log={kv-store={db-dir=data/servers/active-metadata-store/repository/xtdb-kv/rdb-tx, xtdb/module=xtdb.rocksdb/->kv-store}}, xtdb/index-store={kv-store={db-dir=data/servers/active-metadata-store/repository/xtdb-kv/rdb-index, xtdb/module=xtdb.rocksdb/->kv-store}}, xtdb/document-store={kv-store={db-dir=data/servers/active-metadata-store/repository/xtdb-kv/rdb-docs, xtdb/module=xtdb.rocksdb/->kv-store}}}}
Thu Jul 18 10:09:43 BST 2024 active-metadata-store Startup OMRS-AUDIT-0045 The connector for the local repository has been initialized
SLF4J: No SLF4J providers were found.
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See https://www.slf4j.org/codes.html#noProviders for further details.
Thu Jul 18 10:09:43 BST 2024 active-metadata-store Startup OMRS-AUDIT-0046 The connector for the local repository is about to be started
Thu Jul 18 10:09:43 BST 2024 active-metadata-store Startup OMRS-XTDB-REPOSITORY-0001 A request to start an XTDB repository node has been received
Thu Jul 18 10:09:43 BST 2024 active-metadata-store Startup OMRS-XTDB-REPOSITORY-0002 An XTDB repository node is starting with a persistent store
2024-07-18T10:09:43.489+01:00 ERROR 90010 --- [           main] o.s.boot.SpringApplication               : Application run failed

java.lang.reflect.UndeclaredThrowableException: Failed to invoke event listener method
HandlerMethod details: 
Bean [org.odpi.openmetadata.platformchassis.springboot.OMAGServerPlatform$ApplicationContextListener]
Method [public void org.odpi.openmetadata.platformchassis.springboot.OMAGServerPlatform$ApplicationContextListener.applicationReady()]
Resolved arguments: 

	at org.springframework.context.event.ApplicationListenerMethodAdapter.doInvoke(ApplicationListenerMethodAdapter.java:365) ~[spring-context-6.0.12.jar!/:6.0.12]
	at org.springframework.context.event.ApplicationListenerMethodAdapter.processEvent(ApplicationListenerMethodAdapter.java:233) ~[spring-context-6.0.12.jar!/:6.0.12]
	at org.springframework.context.event.ApplicationListenerMethodAdapter.onApplicationEvent(ApplicationListenerMethodAdapter.java:165) ~[spring-context-6.0.12.jar!/:6.0.12]
	at org.springframework.context.event.SimpleApplicationEventMulticaster.doInvokeListener(SimpleApplicationEventMulticaster.java:174) ~[spring-context-6.0.12.jar!/:6.0.12]
	at org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:167) ~[spring-context-6.0.12.jar!/:6.0.12]
	at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:145) ~[spring-context-6.0.12.jar!/:6.0.12]
	at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:437) ~[spring-context-6.0.12.jar!/:6.0.12]
	at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:370) ~[spring-context-6.0.12.jar!/:6.0.12]
	at org.springframework.boot.context.event.EventPublishingRunListener.ready(EventPublishingRunListener.java:109) ~[spring-boot-3.1.4.jar!/:3.1.4]
	at org.springframework.boot.SpringApplicationRunListeners.lambda$ready$6(SpringApplicationRunListeners.java:80) ~[spring-boot-3.1.4.jar!/:3.1.4]
	at java.base/java.lang.Iterable.forEach(Iterable.java:75) ~[na:na]
	at org.springframework.boot.SpringApplicationRunListeners.doWithListeners(SpringApplicationRunListeners.java:118) ~[spring-boot-3.1.4.jar!/:3.1.4]
	at org.springframework.boot.SpringApplicationRunListeners.doWithListeners(SpringApplicationRunListeners.java:112) ~[spring-boot-3.1.4.jar!/:3.1.4]
	at org.springframework.boot.SpringApplicationRunListeners.ready(SpringApplicationRunListeners.java:80) ~[spring-boot-3.1.4.jar!/:3.1.4]
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:334) ~[spring-boot-3.1.4.jar!/:3.1.4]
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1309) ~[spring-boot-3.1.4.jar!/:3.1.4]
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1298) ~[spring-boot-3.1.4.jar!/:3.1.4]
	at org.odpi.openmetadata.platformchassis.springboot.OMAGServerPlatform.main(OMAGServerPlatform.java:130) ~[classes!/:na]
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) ~[na:na]
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
	at java.base/java.lang.reflect.Method.invoke(Method.java:568) ~[na:na]
	at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:49) ~[omag-server-platform-5.1-SNAPSHOT.jar:na]
	at org.springframework.boot.loader.Launcher.launch(Launcher.java:108) ~[omag-server-platform-5.1-SNAPSHOT.jar:na]
	at org.springframework.boot.loader.Launcher.launch(Launcher.java:58) ~[omag-server-platform-5.1-SNAPSHOT.jar:na]
	at org.springframework.boot.loader.PropertiesLauncher.main(PropertiesLauncher.java:467) ~[omag-server-platform-5.1-SNAPSHOT.jar:na]
Caused by: java.lang.ExceptionInInitializerError: null
	at xtdb.api.XtdbFactory.<clinit>(XtdbFactory.java:7) ~[xtdb-repository-connector-5.1-SNAPSHOT-jar-with-dependencies.jar:na]
	at xtdb.api.IXtdb.startNode(IXtdb.java:45) ~[xtdb-repository-connector-5.1-SNAPSHOT-jar-with-dependencies.jar:na]
	at org.odpi.openmetadata.adapters.repositoryservices.xtdb.repositoryconnector.XTDBOMRSRepositoryConnector.start(XTDBOMRSRepositoryConnector.java:157) ~[xtdb-repository-connector-5.1-SNAPSHOT-jar-with-dependencies.jar:na]
	at org.odpi.openmetadata.repositoryservices.localrepository.repositoryconnector.LocalOMRSRepositoryConnector.start(LocalOMRSRepositoryConnector.java:170) ~[repository-services-implementation-5.1-SNAPSHOT.jar!/:na]
	at org.odpi.openmetadata.repositoryservices.admin.OMRSOperationalServices.initializeCohortMember(OMRSOperationalServices.java:429) ~[repository-services-implementation-5.1-SNAPSHOT.jar!/:na]
	at org.odpi.openmetadata.serveroperations.server.OMAGServerOperationalServices.activateWithSuppliedConfig(OMAGServerOperationalServices.java:500) ~[server-operations-server-5.1-SNAPSHOT.jar!/:na]
	at org.odpi.openmetadata.serveroperations.server.OMAGServerOperationalServices.activateWithStoredConfig(OMAGServerOperationalServices.java:163) ~[server-operations-server-5.1-SNAPSHOT.jar!/:na]
	at org.odpi.openmetadata.serveroperations.server.OMAGServerOperationalServices.activateServerListWithStoredConfig(OMAGServerOperationalServices.java:105) ~[server-operations-server-5.1-SNAPSHOT.jar!/:na]
	at org.odpi.openmetadata.platformchassis.springboot.OMAGServerPlatform.autoStartConfig(OMAGServerPlatform.java:215) ~[classes!/:na]
	at org.odpi.openmetadata.platformchassis.springboot.OMAGServerPlatform$ApplicationContextListener.applicationReady(OMAGServerPlatform.java:322) ~[classes!/:na]
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) ~[na:na]
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
	at java.base/java.lang.reflect.Method.invoke(Method.java:568) ~[na:na]
	at org.springframework.context.event.ApplicationListenerMethodAdapter.doInvoke(ApplicationListenerMethodAdapter.java:348) ~[spring-context-6.0.12.jar!/:6.0.12]
	... 25 common frames omitted
Caused by: java.lang.IllegalStateException: Attempting to call unbound fn: #'clojure.core/refer
	at clojure.lang.Var$Unbound.throwArity(Var.java:45) ~[xtdb-repository-connector-5.1-SNAPSHOT-jar-with-dependencies.jar:na]
	at clojure.lang.AFn.invoke(AFn.java:32) ~[xtdb-repository-connector-5.1-SNAPSHOT-jar-with-dependencies.jar:na]
	at clojure.lang.Var.invoke(Var.java:384) ~[xtdb-repository-connector-5.1-SNAPSHOT-jar-with-dependencies.jar:na]
	at clojure.lang.RT.doInit(RT.java:485) ~[xtdb-repository-connector-5.1-SNAPSHOT-jar-with-dependencies.jar:na]
	at clojure.lang.RT.init(RT.java:467) ~[xtdb-repository-connector-5.1-SNAPSHOT-jar-with-dependencies.jar:na]
	at clojure.java.api.Clojure.<clinit>(Clojure.java:97) ~[xtdb-repository-connector-5.1-SNAPSHOT-jar-with-dependencies.jar:na]
	... 40 common frames omitted

2024-07-18T10:09:43.493+01:00  INFO 90010 --- [           main] o.o.o.p.springboot.OMAGServerPlatform    : Temporarily deactivating any auto-started servers '[active-metadata-store, engine-host, integration-daemon, view-server, simple-metadata-store]'
Thu Jul 18 10:09:43 BST 2024 OMAG Server Platform shutdown requested. Shutting down auto-started servers (if running): [active-metadata-store, engine-host, integration-daemon, view-server, simple-metadata-store]
Thu Jul 18 10:09:43 BST 2024 active-metadata-store Shutdown SERVER-OPS-0005 The active-metadata-store server has begun the shutdown process
Thu Jul 18 10:09:43 BST 2024 active-metadata-store Shutdown OMRS-AUDIT-0010 The Open Metadata Repository Services (OMRS) is disconnecting the open metadata repositories
Thu Jul 18 10:09:43 BST 2024 active-metadata-store Shutdown OMRS-AUDIT-0011 The Open Metadata Repository Services (OMRS) has disconnected from the open metadata repositories
Thu Jul 18 10:09:43 BST 2024 active-metadata-store Shutdown SERVER-OPS-0006 The active-metadata-store server has completed shutdown
Disconnected from the target VM, address: '127.0.0.1:63446', transport: 'socket'

Process finished with exit code 1

Answer 1 · 2024-07-18T09:46:05.000Z

The problem shown above could be:

something missing from the XTDB jar, or a limitation in the multi-language bridge between Java and Closure.
a problem in the new class loader.

The next step is to try this with the JanusGraph repository connector, and do further testing with the in-memory to check it is really working in the simple case.

The code for IsolatedClassLoader and IsolatedConnectorProvider is included in the main build (PR in prgress) but it is disabled. This is to allow ongoing work by different people without distrupting other 5.1 work.