ClassCastException after plugin gets reloaded

Question

ClassCastException after plugin gets reloaded

guusdk opened this issue 3 years ago · 10 comments

When the plugin gets reloaded (or reinstalled, upgraded), then ClassCastExceptions are very likely to occur (if the plugin has seen usage).

Exceptions relate to the class that's used to store instances of in a cache, as shown below

java.lang.ClassCastException: class org.igniterealtime.openfire.plugins.pushnotification.PushInterceptor$SentNotification cannot be cast to class org.igniterealtime.openfire.plugins.pushnotification.PushInterceptor$SentNotification (org.igniterealtime.openfire.plugins.pushnotification.PushInterceptor$SentNotification is in unnamed module of loader org.jivesoftware.openfire.container.PluginClassLoader @446907d7; org.igniterealtime.openfire.plugins.pushnotification.PushInterceptor$SentNotification is in unnamed module of loader org.jivesoftware.openfire.container.PluginClassLoader @531905e3)
at java.util.Collection.removeIf(Collection.java:576) ~[?:?]
at org.igniterealtime.openfire.plugins.pushnotification.PushInterceptor.wasPushAttemptedFor(PushInterceptor.java:263) ~[?:?]
at org.igniterealtime.openfire.plugins.pushnotification.PushInterceptor.tryPushNotification(PushInterceptor.java:144) ~[?:?]
at org.igniterealtime.openfire.plugins.pushnotification.PushInterceptor.messageStored(PushInterceptor.java:247) ~[?:?]
at org.jivesoftware.openfire.OfflineMessageStrategy.store(OfflineMessageStrategy.java:196) [xmppserver-4.6.1.jar:4.6.1]
at org.jivesoftware.openfire.OfflineMessageStrategy.storeOffline(OfflineMessageStrategy.java:140) [xmppserver-4.6.1.jar:4.6.1]
at org.jivesoftware.openfire.MessageRouter.routingFailed(MessageRouter.java:268) [xmppserver-4.6.1.jar:4.6.1]
at org.jivesoftware.openfire.spi.RoutingTableImpl.routePacket(RoutingTableImpl.java:290) [xmppserver-4.6.1.jar:4.6.1]
at org.jivesoftware.openfire.MessageRouter.route(MessageRouter.java:134) [xmppserver-4.6.1.jar:4.6.1]
at org.jivesoftware.openfire.spi.PacketRouterImpl.route(PacketRouterImpl.java:79) [xmppserver-4.6.1.jar:4.6.1]
at org.jivesoftware.openfire.net.StanzaHandler.processMessage(StanzaHandler.java:422) [xmppserver-4.6.1.jar:4.6.1]
at org.jivesoftware.openfire.net.ClientStanzaHandler.processMessage(ClientStanzaHandler.java:109) [xmppserver-4.6.1.jar:4.6.1]
at org.jivesoftware.openfire.net.StanzaHandler.process(StanzaHandler.java:246) [xmppserver-4.6.1.jar:4.6.1]
at org.jivesoftware.openfire.net.StanzaHandler.process(StanzaHandler.java:209) [xmppserver-4.6.1.jar:4.6.1]
at org.jivesoftware.openfire.nio.ConnectionHandler.messageReceived(ConnectionHandler.java:183) [xmppserver-4.6.1.jar:4.6.1]
at org.apache.mina.core.filterchain.DefaultIoFilterChain$TailFilter.messageReceived(DefaultIoFilterChain.java:1015) [mina-core-2.1.3.jar:?]
at org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:650) [mina-core-2.1.3.jar:?]
at org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1300(DefaultIoFilterChain.java:49) [mina-core-2.1.3.jar:?]
at org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.messageReceived(DefaultIoFilterChain.java:1128) [mina-core-2.1.3.jar:?]
at org.apache.mina.core.filterchain.IoFilterAdapter.messageReceived(IoFilterAdapter.java:122) [mina-core-2.1.3.jar:?]
at org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:650) [mina-core-2.1.3.jar:?]
at org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1300(DefaultIoFilterChain.java:49) [mina-core-2.1.3.jar:?]
at org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.messageReceived(DefaultIoFilterChain.java:1128) [mina-core-2.1.3.jar:?]
at org.apache.mina.filter.codec.ProtocolCodecFilter$ProtocolDecoderOutputImpl.flush(ProtocolCodecFilter.java:413) [mina-core-2.1.3.jar:?]
at org.apache.mina.filter.codec.ProtocolCodecFilter.messageReceived(ProtocolCodecFilter.java:257) [mina-core-2.1.3.jar:?]
at org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:650) [mina-core-2.1.3.jar:?]
at org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1300(DefaultIoFilterChain.java:49) [mina-core-2.1.3.jar:?]
at org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.messageReceived(DefaultIoFilterChain.java:1128) [mina-core-2.1.3.jar:?]
at org.apache.mina.core.filterchain.IoFilterEvent.fire(IoFilterEvent.java:106) [mina-core-2.1.3.jar:?]
at org.apache.mina.core.session.IoEvent.run(IoEvent.java:89) [mina-core-2.1.3.jar:?]
at org.apache.mina.filter.executor.OrderedThreadPoolExecutor$Worker.runTask(OrderedThreadPoolExecutor.java:766) [mina-core-2.1.3.jar:?]
at org.apache.mina.filter.executor.OrderedThreadPoolExecutor$Worker.runTasks(OrderedThreadPoolExecutor.java:758) [mina-core-2.1.3.jar:?]
at org.apache.mina.filter.executor.OrderedThreadPoolExecutor$Worker.run(OrderedThreadPoolExecutor.java:697) [mina-core-2.1.3.jar:?]
at java.lang.Thread.run(Thread.java:832) [?:?]

Answer 1 · 2021-01-29T16:27:41.000Z

The problem is caused by storing instances of a class provided by this plugin in a cache.

Whenever such an instance gets stored in a cache, its class is being references by that cache. When the plugin gets reloaded/updated, the PluginClassLoader that loaded the class gets replaced. When the newly loaded plugin tries to interact with the cache (that still has a reference to 'old' data), instances from the same class, but loaded by both the old as well as new PluginClassLoader, are being used, causing class cast exceptions.

Note that I believe this problem to apply to all Openfire plugins that store instances of classes that they provide in a cache. I don't think that this problem is specific to Hazelcast either: it also applies to caches in a non-clustered setup.

A workaround for this problem is: restart Openfire whenever the plugin gets updated/reloaded.

Answer 2 · 2021-01-29T16:28:41.000Z

The only reliable way to prevent this issue that I can think of is to prevent using classes provided by anything but the Openfire classloader. In other words, do not add instances of classes provided by the plugin into a cache.

Answer 3 · 2021-01-29T16:30:00.000Z

@GregDThomas what's your take on this?

Answer 4 · 2021-01-29T17:14:38.000Z

Whenever such an instance gets stored in a cache, its class is being references by that cache.

That behaviour does seem odd, but certainly explains this problem (and possibly others). I've got around similar problems by caching a String - the JSON representation of the object I wanted to cache, using Jackson (https://www.baeldung.com/jackson-object-mapper-tutorial if you're not already familiar with it). Looking at your fix, you've got two caches, one for each field. That's probably fine, but clearly becomes more of a problem if the number of fields grows.

I'd be very tempted to start with a JSON cache now, tbh.

Answer 5 · 2021-01-29T17:20:23.000Z

(on reflection, I'd be very tempted to write a couple of default methods to the Cache class,

default void putAsJsonString(final K key, final V value) {
    put(objectMapper.writeValueAsString(key), objectMapper.writeValueAsString(value));
}

default V getFromJsonString(final K key, final Class<V> clazz) {
    return get(objectMapper.readValue(get(key), clazz);
}

Answer 6 · 2021-01-29T18:34:09.000Z

Why JSON, and not, for example, XML - which is a lot closer to XMPP, and possibly more type-safe? It might even we worth considering using Java native / object serialization, for complete type safety. All classes for which instances are stored in a cache must be Serializable anyway. We might loose 'human readability' (which might be restored by having a smart utility method, but is limited to the admin console anyway), but it'll be a lot more efficient.

I was also thinking that it might be a good idea to somehow log a warning, when a Cache is instantiated or used with instances of classes that are loaded by a PluginClassLoader. That might help identify potential affected code.

Answer 7 · 2021-01-29T18:38:16.000Z

Why JSON? At the time I was doing it, I was working in JSON a lot (carried over XMPP!) But yes - XML or standard serialization would also do it. Though a textual format may help with the display on the admin screen. And yes, the warning sounds like a good idea.

Answer 8 · 2021-02-03T15:19:33.000Z

Although I think you're right in that we can improve on this significantly, I'm going to merge my fix that simply prevents using any plugin-provided class for now. It's cumbersome and adds code complexity, but works. We should replace this when we've got the infrastructure in place that will 'automagically' does some kind of serialization (always? only for plugins?), but for now, I'd like to release something that no longer has the problem.

Answer 9 · 2021-11-18T15:45:57.000Z

I believe that I have provided a suitable implementation in igniterealtime/Openfire#1926

@GregDThomas, your suggested default implementation (quoted below) didn't work, as the objectmapped keys and values are Strings (obviously). That does not match with the generics definition of the Cache (K and V). Instead, I've opted for a model that uses delegation to do much of the same.

default void putAsJsonString(final K key, final V value) {
put(objectMapper.writeValueAsString(key), objectMapper.writeValueAsString(value));
}

default V getFromJsonString(final K key, final Class clazz) {
return get(objectMapper.readValue(get(key), clazz);
}

Answer 10 · 2021-11-18T16:11:42.000Z

That's fine; the above code was written off the top of my head, the implementation itself looks good.