Error when using `fire-rules` in parallel
mauricioszabo opened this issue · 1 comments
mauricioszabo commented
When trying to run Clara (firing rules) over multiple threads, we get an error
Ofending come (somewhat simplified):
(let [session (-> (mk-session 'events-reports.reports.deferred
'events-reports.reports.normalize-data
:fact-type-fn :fact))
res (jdbc/query db "SELECT * FROM my_table WHERE ...")
location-ids (map :id res)]
(doseq [id location-ids]
(async/go
(println "Emitting for" id)
(jdbc/with-db-connection [db @smart/db]
(let [facts (search-facts db id)
rows (-> session
(insert-all facts)
(fire-rules)
(query detailed-anual-report)
(->> (map make-detailed-anual-row)))]
(println "Saving for" id)
(save-rows rows))))))
Clojure version: Clojure 1.9.0
JVM version: Eclipse OpenJ9 VM 1.8.0_181-b13
Core.Async version: 0.4.490
Exception:
Exception in thread "async-dispatch-4" java.lang.IllegalArgumentException: find not supported on type: clara.rules.engine.Token$reify__26254
at clojure.lang.RT.find(RT.java:863)
at clojure.core$select_keys.invokeStatic(core.clj:1539)
at clojure.core$select_keys.invoke(core.clj:1532)
at clara.rules.engine$propagate_items_to_nodes$fn__26782.invoke(engine.cljc:154)
at clara.rules.platform$group_by_seq$fn__26030.invoke(platform.cljc:39)
at clojure.core.protocols$naive_seq_reduce.invokeStatic(protocols.clj:62)
at clojure.core.protocols$interface_or_naive_reduce.invokeStatic(protocols.clj:72)
at clojure.core.protocols$fn__7852.invokeStatic(protocols.clj:169)
at clojure.core.protocols$fn__7852.invoke(protocols.clj:124)
at clojure.core.protocols$fn__7807$G__7802__7816.invoke(protocols.clj:19)
at clojure.core.protocols$seq_reduce.invokeStatic(protocols.clj:31)
at clojure.core.protocols$fn__7835.invokeStatic(protocols.clj:75)
at clojure.core.protocols$fn__7835.invoke(protocols.clj:75)
at clojure.core.protocols$fn__7781$G__7776__7794.invoke(protocols.clj:13)
at clojure.core$reduce.invokeStatic(core.clj:6748)
at clojure.core$reduce.invoke(core.clj:6730)
at clara.rules.platform$group_by_seq.invokeStatic(platform.cljc:38)
at clara.rules.platform$group_by_seq.invoke(platform.cljc:27)
at clara.rules.engine$propagate_items_to_nodes.invokeStatic(engine.cljc:154)
at clara.rules.engine$propagate_items_to_nodes.invoke(engine.cljc:147)
at clara.rules.engine.LocalTransport.send_elements(engine.cljc:176)
at clara.rules.engine.AlphaNode.alpha_activate(engine.cljc:538)
at clara.rules.engine$flush_updates$flush_all__26940.invoke(engine.cljc:269)
at clara.rules.engine$flush_updates.invokeStatic(engine.cljc:277)
at clara.rules.engine$flush_updates.invoke(engine.cljc:252)
at clara.rules.engine$fire_rules_STAR_.invokeStatic(engine.cljc:1825)
at clara.rules.engine$fire_rules_STAR_.invoke(engine.cljc:1720)
at clara.rules.engine.LocalSession.fire_rules(engine.cljc:1909)
at clara.rules$fire_rules.invokeStatic(rules.cljc:44)
at clara.rules$fire_rules.invoke(rules.cljc:29)
mrrodriguez commented
I discussed this some on #clara in clojurians Slack, but this is an odd looking exception that seems like perhaps could happen from clj's keyword callsite optimizations if the clara.rules.engine.Token record type is perhaps compiled/loaded multiple times in a way where both instances of the class (from separate classloaders) are interacting with one another at.
So perhaps
- the session is compiled with "version 1" of clara.rules.engine.Token,
- the ns is for some reason loaded again
- later a runtime instance of "version 2" of clara.rules.engine.Token comes across this compiled callsite
This is just my suspicion so far, putting it here in case it helps track down anything more.