Netflix/PigPen

fold/vec should call clojure.core/vec

minimal opened this issue · 3 comments

minimal commented

Hi,

I'm getting an arity exception on fold/vec when using some fold functions that use it (e.g. fold/first). It looks like vec should be calling (clojure.core/vec (concat l r)) rather than itself.

Caused by: clojure.lang.ArityException: Wrong number of args (1) passed to: fold$vec
        at clojure.lang.AFn.throwArity(AFn.java:437)
        at clojure.lang.AFn.invoke(AFn.java:39)
        at pigpen.fold$vec$fn__1521.invoke(fold.clj:170)
        at clojure.core$comp$fn__4154.invoke(core.clj:2332)
        at pigpen.fold$juxt$fn__1560$fn__1566.invoke(fold.clj:342)
        at clojure.core$map$fn__4214.invoke(core.clj:2498)

(defn vec
  []
  (fold-fn ^:seq (fn
                   ([] [])
                   ([l r] (vec (concat l r))))
           ^:seq (fn [acc val] (conj acc val))))
mbossenbroek commented

Fixed in 0.2.2 - just released, may take an hour or two to propagate.

I noticed that when running locally I was using reducef to consolidate all test values, so I had no test coverage on combinef functions. This has also been rectified - it now splits the test data so that combinef is called at least once.

Thanks for the bug report!

minimal commented

Thanks for the quick turnaround! The update is working great for me.

Interestingly it would only error when run on the cluster with lots of data rather than running pig locally or in the clojure repl.

mbossenbroek commented

Yeah, when run locally it wasn't using the combinef fn because there was so
little data. That's why you didn't hit the problem until the cluster, but
it also meant that no combinef was being called locally, which is bad for
test coverage. This is also fixed now - luckily that was the only one that
had a problem :)

-Matt

On Mar 26, 2014, at 4:43 AM, Chris McDevitt notifications@github.com
wrote:

Thanks for the quick turnaround! The update is working great for me.

Interestingly it would only error when run on the cluster with lots of data
rather than running pig locally or in the clojure repl.

Reply to this email directly or view it on
GitHubhttps://github.com//issues/14#issuecomment-38674047
.