clearlinux/clr-bundles

Build R with Intel® C++ and Fortran Compilers and Intel® MKL on Linux

Closed this issue · 6 comments

jobdiogenes commented

Like many, I am looking for optimization.
Where I work, an Ecology Research Institute/University, R are very used,
and some works are really slow due nature of R, some scripts take weeks to run.
So everything that could accelerate is welcome.
So, then I found an Intel post about Build R with Intel® C++ and Fortran Compilers and Intel® MKL on Linux.

First I try to see if has a Cran like repo already compiled with ICC/MKL/.... Anaconda has something to python, but I did not found about R.

Then I found clearlinux, and see R bundle. My goal is to create a new bundle that adjust to ICC, MKL, Blas, etc.

I'm new to clear Linux, and ICC, could latest R source (4.X) be build using R bundle?
What steps I need to follow?
There's some implication in compatibility?

Thanks for any Help.

fenrus75 commented
lebensterben commented

If you read the article, specifically the part on benchmark, you'd find this is not a legit one.

On one hand, they have a C function calculating powers with vectorization. On the other, they call repeat to calculate z[i] <- x[i] ^ y[i].

This is a joke.

lebensterben commented

To be clear, R primitives are written in C and Fortran. And many packages also use C++.

Compiling everything with ICC could possibly make a difference. But the majority of improvement usually comes from calling C/C++ directly.

R does many safety checks in the background and that seriously slow down your script. If you seriously need good performance, use Rcpp library and port your code to C++. Without ICC or MKL, that would result in 30x improvement in performance in my experience.

  • 👍1
jobdiogenes commented

@lebensterben, @fenrus75 , thanks very much for your comments,

The big question is that R its a Lego house, some packages that we use call many R pieces, the effort to migrate all to Rcpp, C++ are too much.

At the moment, I'm optimizing the top level package, to use GPU and parallelism, that could give some improvements without so much effort.

I'm also looking to similar packages in Python. But in long term, I think that Julia, will be train that people at math/stat will put their code wagons. And even Rust, for base libraries, due memory safety, without slow down.

fenrus75 commented
jobdiogenes commented

you started with an assumption that our R was slower than .... ... and it's not clear really that that is true; our R and the packages we ship with it are pretty optimized.....
…
On Fri, Jul 17, 2020 at 9:02 AM Job Diógenes Ribeiro Borges < @.***> wrote: @lebensterben https://github.com/lebensterben, @fenrus75 https://github.com/fenrus75 , thanks very much your comments, The big question is that R its a Lego house, some packages that we use call many R pieces, the effort to migrate all to Rcpp, C++ are too much. At the moment, I'm optimizing the top level package, to use GPU and parallelism, that could give some improvements without so much effort. I'm also looking to similar packages in Python. But in long term, I think that Julia, will be train that people at math/stat will put their code wagons. And even Rust, for base libraries, due memory safety, without slow down. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#201 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAJ54FMCXPTEIC4PJXRVYVDR4BYX3ANCNFSM4O6I2ULQ .

No, I didn't say that your R are not optimized, I was just following the arguments of Intel ICC post, that args that using ICC, will be more fast. But, now I saw that its not the way.

Thanks