on clojars ?
Closed this issue · 10 comments
Any plans to put this on clojars? or is it already there?
thanks
Would love to! Are you familiar enough with Clojars to help out?
I'm not familiar enough but I have colleagues who are :) Meanwhile I wonder if you could help me understand why I'm not getting expected results for quantiles below. Especially the 10% and 90% quantile look bad. I understand it's only an approximation, but I guess I don't fully understand the scenarios where I can expect reasonable results. Also there seems to be no way to control the "accuracy" of the approximation. Incidentally, I also looked at this Java library https://github.com/mayconbordin/streaminer but the quantile results there (using Frugal2U) seem not so great either.
(ns ...
...
(:require
[skream.core :as sk]))
(def sk
(-> (sk/create-skream)
(sk/track-quantile-ish 0.10)
(sk/track-quantile-ish 0.25)
(sk/track-quantile-ish 0.50)
(sk/track-quantile-ish 0.90)))
(let
[sk (apply (partial sk/add-num sk ) (shuffle (range 1000)))]
(map #(get-in sk [[:quantile %] :quantile] )
[0.10 0.25 0.50 0.90]))
==> (252.27943 252.27943 496.22464 750.6505)
Just deployed and promoted Skream to the clojars system.
You might be able to increase the accuracy of the P2 quantile estimation by adding additional "markers" as described in the original paper "http://www.cs.wustl.edu/~jain/papers/ftp/psqr.pdf", but that is quite involved! That said, I think you 10% and 90% quantiles should be better over the (0...1000) range. Will you please submit a Github issue?
Actually I just re-tried using the Frugal2U I mentioned earlier, and actually their quantile estimates are quite reasonable (I did something wrong in the earlier test).
Also, since you deployed to clojars, it would be worthwhile (as is traditional) to mention in the README, the artifact + version numbers to include as dependencies in the leiningen project.clj
Not sure what you mean. Doesn't the project.clj file in Git list all the project's dependencies?
Yes it lists the dependencies of skream, but I'm talking about what people need to include in their project.clj in order to be able to use skream, i..e
[skream "0.0.2"]
Usually people will find your repo via a web search, and would want to use in in their project, so they would need to know what to put in their project.clj. They may not actually search on clojars.
As an example, please see this repo: https://github.com/clj-time/clj-time
Ahh, that makes sense. Done!
ouch- you linked to clj-time in your README !
Bah, good catch. :)