Faster merge
Opened this issue · 0 comments
joinr commented
I was focused on improving clojure.core/merge performance. Watching Tommi's talk reminded me of the slowness (and I end up using it in a lot of legacy code on hot paths...). I tried just optimizing the persistent map-based variant, which got me into the same ballpark you referenced (~30%). Going with transients and a useful heuristic yields some better fruit though (~50%)...
I did some testing to eliminate sources of slowdown:
- prefer discrete args vs. the default var-args for everything version,
- use transients instead of persistent map-based conj,
- use direct method invocation everywhere,
- accumulate keys from r->l instead of l->r, since we can prune more (in some cases, e.g. 2-arg versions like (merge {:a 2 :b 3 :c 4} {:a 1 :b 1 :c 1}) we can exclude all keys from l since they already appear in R, leading to 0 actual assoc's)
(defn rmerge! [^clojure.lang.IKVReduce l r]
(.kvreduce l
(fn [^clojure.lang.ITransientAssociative acc k v]
(if-not (acc k)
(.assoc acc k v)
acc)) r))
;;~50% faster.
(defn fast-merge
([] {})
([m] m)
([m1 m2] (rmerge m1 m2))
([m1 m2 m3] (->> (transient m3) (rmerge! m2) (rmerge! m1) persistent!))
([m1 m2 m3 m4] (->> (transient m4) (rmerge! m3) (rmerge! m2) (rmerge! m1) persistent!))
([m1 m2 m3 m4 m5] (->> (transient m5) (rmerge! m4) (rmerge! m3) (rmerge! m2) (rmerge! m1) persistent!))
([m1 m2 m3 m4 m5 & ms]
(let [rs (reverse ms)]
(->> (reduce rmerge! (transient (first rs)) (rest rs))
(rmerge! m5)
(rmerge! m4)
(rmerge! m3)
(rmerge! m3)
(rmerge! m1)
persistent!))))