Lightblue plugins have classpath issues in JBossAS

Question

Lightblue plugins have classpath issues in JBossAS

Opened this issue 9 years ago · 5 comments

2016-04-17 10:00:37,689 [https_executor-threads - 18] ERROR [com.redhat.lightblue.util.Error] {"objectType":"error","context":"rest/FindCommand/myEntity","errorCode":"metadata:ConfigurationNotValid","msg":"com.redhat.lightblue.hook.audit.AuditHookConfigurationParser from [Module \"deployment.lightblue-rest-crud-cert-auth-2.0.0.war:main\" from Service Module Loader]"}
2016-04-17 10:00:37,690 [https_executor-threads - 18] ERROR [com.redhat.lightblue.rest.crud.hystrix.AbstractRestCommand] {"objectType":"error","context":"rest/FindCommand/myEntity","errorCode":"metadata:ConfigurationNotValid","msg":"com.redhat.lightblue.hook.audit.AuditHookConfigurationParser from [Module \"deployment.lightblue-rest-crud-cert-auth-2.0.0.war:main\" from Service Module Loader]"}: {"objectType":"error","context":"rest/FindCommand/myEntity","errorCode":"metadata:ConfigurationNotValid","msg":"com.redhat.lightblue.hook.audit.AuditHookConfigurationParser from [Module \"deployment.lightblue-rest-crud-cert-auth-2.0.0.war:main\" from Service Module Loader]"}
    at com.redhat.lightblue.util.Error.get(Error.java:110)
    at com.redhat.lightblue.config.AbstractMetadataConfiguration.initializeFromJson(AbstractMetadataConfiguration.java:97)
    at com.redhat.lightblue.mongo.config.MongoMetadataConfiguration.initializeFromJson(MongoMetadataConfiguration.java:139)
    at com.redhat.lightblue.config.LightblueFactory.initializeMetadata(LightblueFactory.java:196)
    at com.redhat.lightblue.config.LightblueFactory.getMetadata(LightblueFactory.java:309)
    at com.redhat.lightblue.config.LightblueFactory.initializeMediator(LightblueFactory.java:120)
    at com.redhat.lightblue.config.LightblueFactory.getMediator(LightblueFactory.java:338)
    at com.redhat.lightblue.rest.crud.hystrix.AbstractRestCommand.getMediator(AbstractRestCommand.java:81)
    at com.redhat.lightblue.rest.crud.hystrix.FindCommand.run(FindCommand.java:77)
    at com.redhat.lightblue.rest.crud.hystrix.FindCommand.run(FindCommand.java:36)
    at com.netflix.hystrix.HystrixCommand$1.call(HystrixCommand.java:294)
    at com.netflix.hystrix.HystrixCommand$1.call(HystrixCommand.java:289)
    at rx.Observable$1.call(Observable.java:144)
    at rx.Observable$1.call(Observable.java:136)
    at rx.Observable$1.call(Observable.java:144)
    at rx.Observable$1.call(Observable.java:136)
    at rx.Observable$1.call(Observable.java:144)
    at rx.Observable$1.call(Observable.java:136)
    at rx.Observable$1.call(Observable.java:144)
    at rx.Observable$1.call(Observable.java:136)
    at rx.Observable$1.call(Observable.java:144)
    at rx.Observable$1.call(Observable.java:136)
    at rx.Observable$1.call(Observable.java:144)
    at rx.Observable$1.call(Observable.java:136)
    at rx.Observable$1.call(Observable.java:144)
    at rx.Observable$1.call(Observable.java:136)
    at rx.Observable$1.call(Observable.java:144)
    at rx.Observable$1.call(Observable.java:136)
    at rx.Observable$1.call(Observable.java:144)
    at rx.Observable$1.call(Observable.java:136)
    at rx.Observable$1.call(Observable.java:144)
    at rx.Observable$1.call(Observable.java:136)
    at rx.Observable$1.call(Observable.java:144)
    at rx.Observable$1.call(Observable.java:136)
    at rx.Observable.unsafeSubscribe(Observable.java:7466)
    at com.netflix.hystrix.AbstractCommand$1.call(AbstractCommand.java:398)
    at com.netflix.hystrix.AbstractCommand$1.call(AbstractCommand.java:377)
    at rx.Observable$1.call(Observable.java:144)
    at rx.Observable$1.call(Observable.java:136)
    at rx.Observable$1.call(Observable.java:144)
    at rx.Observable$1.call(Observable.java:136)
    at rx.Observable$1.call(Observable.java:144)
    at rx.Observable$1.call(Observable.java:136)
    at rx.Observable.unsafeSubscribe(Observable.java:7466)
    at com.netflix.hystrix.AbstractCommand$ObservableCommand$1.call(AbstractCommand.java:1135)
    at com.netflix.hystrix.AbstractCommand$ObservableCommand$1.call(AbstractCommand.java:1131)
    at rx.Observable$1.call(Observable.java:144)
    at rx.Observable$1.call(Observable.java:136)
    at rx.Observable.subscribe(Observable.java:7556)
    at rx.internal.operators.BlockingOperatorToFuture.toFuture(BlockingOperatorToFuture.java:57)
    at rx.observables.BlockingObservable.toFuture(BlockingObservable.java:410)
    at com.netflix.hystrix.HystrixCommand.queue(HystrixCommand.java:379)
    at com.netflix.hystrix.HystrixCommand.execute(HystrixCommand.java:335)
    at com.redhat.lightblue.rest.crud.AbstractCrudResource.find(AbstractCrudResource.java:245)
    at com.redhat.lightblue.rest.crud.AbstractCrudResource.find(AbstractCrudResource.java:235)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:168)
    at org.jboss.resteasy.core.ResourceMethod.invokeOnTarget(ResourceMethod.java:269)
    at org.jboss.resteasy.core.ResourceMethod.invoke(ResourceMethod.java:227)
    at org.jboss.resteasy.core.ResourceMethod.invoke(ResourceMethod.java:216)
    at org.jboss.resteasy.core.SynchronousDispatcher.getResponse(SynchronousDispatcher.java:561)
    at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:543)
    at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:128)
    at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:208)
    at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:55)
    at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:50)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:847)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:295)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:214)
    at com.redhat.lightblue.rest.audit.LightblueAuditServletFilter.doFilter(LightblueAuditServletFilter.java:76)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:246)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:214)
    at org.ebaysf.web.cors.CORSFilter.handleNonCORS(CORSFilter.java:438)
    at org.ebaysf.web.cors.CORSFilter.doFilter(CORSFilter.java:173)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:246)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:214)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:231)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149)
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:512)
    at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:150)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97)
    at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:559)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:344)
    at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:854)
    at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:653)
    at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:511)
    at org.jboss.threads.SimpleDirectExecutor.execute(SimpleDirectExecutor.java:33)
    at org.jboss.threads.QueueExecutor.runTask(QueueExecutor.java:808)
    at org.jboss.threads.QueueExecutor.access$100(QueueExecutor.java:45)
    at org.jboss.threads.QueueExecutor$Worker.run(QueueExecutor.java:828)
    at java.lang.Thread.run(Thread.java:745)
    at org.jboss.threads.JBossThread.run(JBossThread.java:122)
2016-04-17 10:00:37,695 [https_executor-threads - 18] ERROR [com.redhat.lightblue.util.Error] {"objectType":"error","context":"rest/FindCommand/myEntity","errorCode":"rest-crud:CantGetMediator"}
2016-04-17 10:00:37,695 [https_executor-threads - 18] ERROR [com.redhat.lightblue.rest.crud.hystrix.FindCommand] find:generic_error failure: {}: {"objectType":"error","context":"rest/FindCommand/myEntity","errorCode":"rest-crud:CantGetMediator"}
    at com.redhat.lightblue.util.Error.get(Error.java:134)
    at com.redhat.lightblue.rest.crud.hystrix.AbstractRestCommand.getMediator(AbstractRestCommand.java:85)
    at com.redhat.lightblue.rest.crud.hystrix.FindCommand.run(FindCommand.java:77)
    at com.redhat.lightblue.rest.crud.hystrix.FindCommand.run(FindCommand.java:36)
    at com.netflix.hystrix.HystrixCommand$1.call(HystrixCommand.java:294)
    at com.netflix.hystrix.HystrixCommand$1.call(HystrixCommand.java:289)
    at rx.Observable$1.call(Observable.java:144)
    at rx.Observable$1.call(Observable.java:136)
    at rx.Observable$1.call(Observable.java:144)
    at rx.Observable$1.call(Observable.java:136)
    at rx.Observable$1.call(Observable.java:144)
    at rx.Observable$1.call(Observable.java:136)
    at rx.Observable$1.call(Observable.java:144)
    at rx.Observable$1.call(Observable.java:136)
    at rx.Observable$1.call(Observable.java:144)
    at rx.Observable$1.call(Observable.java:136)
    at rx.Observable$1.call(Observable.java:144)
    at rx.Observable$1.call(Observable.java:136)
    at rx.Observable$1.call(Observable.java:144)
    at rx.Observable$1.call(Observable.java:136)
    at rx.Observable$1.call(Observable.java:144)
    at rx.Observable$1.call(Observable.java:136)
    at rx.Observable$1.call(Observable.java:144)
    at rx.Observable$1.call(Observable.java:136)
    at rx.Observable$1.call(Observable.java:144)
    at rx.Observable$1.call(Observable.java:136)
    at rx.Observable$1.call(Observable.java:144)
    at rx.Observable$1.call(Observable.java:136)
    at rx.Observable.unsafeSubscribe(Observable.java:7466)
    at com.netflix.hystrix.AbstractCommand$1.call(AbstractCommand.java:398)
    at com.netflix.hystrix.AbstractCommand$1.call(AbstractCommand.java:377)
    at rx.Observable$1.call(Observable.java:144)
    at rx.Observable$1.call(Observable.java:136)
    at rx.Observable$1.call(Observable.java:144)
    at rx.Observable$1.call(Observable.java:136)
    at rx.Observable$1.call(Observable.java:144)
    at rx.Observable$1.call(Observable.java:136)
    at rx.Observable.unsafeSubscribe(Observable.java:7466)
    at com.netflix.hystrix.AbstractCommand$ObservableCommand$1.call(AbstractCommand.java:1135)
    at com.netflix.hystrix.AbstractCommand$ObservableCommand$1.call(AbstractCommand.java:1131)
    at rx.Observable$1.call(Observable.java:144)
    at rx.Observable$1.call(Observable.java:136)
    at rx.Observable.subscribe(Observable.java:7556)
    at rx.internal.operators.BlockingOperatorToFuture.toFuture(BlockingOperatorToFuture.java:57)
    at rx.observables.BlockingObservable.toFuture(BlockingObservable.java:410)
    at com.netflix.hystrix.HystrixCommand.queue(HystrixCommand.java:379)
    at com.netflix.hystrix.HystrixCommand.execute(HystrixCommand.java:335)
    at com.redhat.lightblue.rest.crud.AbstractCrudResource.find(AbstractCrudResource.java:245)
    at com.redhat.lightblue.rest.crud.AbstractCrudResource.find(AbstractCrudResource.java:235)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:168)
    at org.jboss.resteasy.core.ResourceMethod.invokeOnTarget(ResourceMethod.java:269)
    at org.jboss.resteasy.core.ResourceMethod.invoke(ResourceMethod.java:227)
    at org.jboss.resteasy.core.ResourceMethod.invoke(ResourceMethod.java:216)
    at org.jboss.resteasy.core.SynchronousDispatcher.getResponse(SynchronousDispatcher.java:561)
    at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:543)
    at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:128)
    at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:208)
    at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:55)
    at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:50)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:847)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:295)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:214)
    at com.redhat.lightblue.rest.audit.LightblueAuditServletFilter.doFilter(LightblueAuditServletFilter.java:76)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:246)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:214)
    at org.ebaysf.web.cors.CORSFilter.handleNonCORS(CORSFilter.java:438)
    at org.ebaysf.web.cors.CORSFilter.doFilter(CORSFilter.java:173)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:246)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:214)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:231)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149)
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:512)
    at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:150)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97)
    at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:559)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:344)
    at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:854)
    at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:653)
    at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:511)
    at org.jboss.threads.SimpleDirectExecutor.execute(SimpleDirectExecutor.java:33)
    at org.jboss.threads.QueueExecutor.runTask(QueueExecutor.java:808)
    at org.jboss.threads.QueueExecutor.access$100(QueueExecutor.java:45)
    at org.jboss.threads.QueueExecutor$Worker.run(QueueExecutor.java:828)
    at java.lang.Thread.run(Thread.java:745)
    at org.jboss.threads.JBossThread.run(JBossThread.java:122)

Answer 1 · 2016-05-17T19:24:22.000Z

See discussion at http://dev.forum.lightblue.io/Lightblue-Plugin-Design-td425.html

Answer 2 · 2016-05-17T19:24:52.000Z

Possibly also related to lightblue-platform/lightblue-mongo#214

Answer 3 · 2016-05-18T01:28:21.000Z

I think the problem is that core is assuming the plugins are on the current running classpath, and there's not a good way to safely augment the current classpath.

Isolated classloaders are really designed for this class of problem. You probably want to collect URLs to jars to load as plugins, each in their own classloader, and then provide instances of plugins to a LightblueFactory. Each instance's class will be isolated in its own classloader with its own resolution of imports[1].

Either that, or move the plugin resolution inside of core, but I think you'll want to do a similar strategy in any case (load each plugin in its own classloader).

[1]: You can write a bottom-up classloader (default is top-down) and then plugins can even include their own copies of common classes that they'll use instead of those already on the classpath of lightblue-rest.

Answer 4 · 2016-05-18T02:23:10.000Z

+1 to moving the functionality into core. The reason it was not initially done that way was to prevent a chicken/egg problem with the datasources.json which is also loaded in lightblue-rest. Likely not an issue for hooks, but a problem for (as an example) the ldap controller.

Answer 5 · 2016-05-18T02:44:04.000Z

We need something like PluginRegistry in core, with plugin jars registered
to it, and all class lookups should be converted to something like
PluginRegistry.getPluginClass() which should both do an isolated
classloader lookup and system classloader lookup.

On Tue, May 17, 2016 at 8:23 PM, Dennis Crissman notifications@github.com
wrote:

+1 to moving the functionality into core. The reason it was not initially
done that way was to prevent a chicken/egg problem with the
datasources.json which is also loaded in lightblue-rest. Likely not an
issue for hooks, but a problem for (as an example) the ldap controller.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub
#234 (comment)