benchmark/compare against Burkardt's implementation

Question

benchmark/compare against Burkardt's implementation

Opened this issue 9 years ago · 6 comments

It would be interesting to compare against this implementation, both for correctness and performance.

Answer 1 · 2015-09-18T11:50:34.000Z

@Ken-B, if you feel like taking a crack at this, it would be much appreciated.

Answer 2 · 2015-09-18T14:05:28.000Z

That's quite a challenge for me, as I've never compiled c++ before. However, with the help of Google I've compiled his code and the example runs. Now I'm trying to build Cxx.jl. I'll keep you posted if any results. Fingers crossed!

Answer 3 · 2015-09-18T15:10:22.000Z

Oh, if it's a bother for you, I can do it myself at some point; I was just thinking of calling/benchmarking it in C++, not in Julia. But I thought you might enjoy taking a crack at it.

Answer 4 · 2015-09-18T16:08:02.000Z

No bother, it's a welcome challenge. I wanted before to check out Cxx.jl and this seems like a perfect opportunity. Im just saying that it's uncharted territory for me so I don't know if I'll manage.

Answer 5 · 2015-09-25T09:07:02.000Z

Cxx.jl requires llvm-svn or llvm 3.7.0, but I keep getting build errors on my mac. I've tried different versions of xcode. Where could I go to get help on that?

I don't think I would be able to manage this soon.

Answer 6 · 2017-03-29T01:11:12.000Z

I ran a little test using this C++ benchmark program. I benchmarked the Julia code using

function nth(s, n)
          x = Vector{Float64}(ndims(s))
          for i = 1:n
             next!(s, x)
          end
          return x
end

to compute the n-th element of a Sobol sequence.

The good news is that the answers match ours in all the cases I tried, and the performance is within a factor of 2.

The bad news is that we are up to 2x slower, so we are leaving some performance on the table. (Note that the C++ code is returning single-precision vectors, whereas we are returning double precision; this is a slight advantage, but I doubt it explains most of the difference.)