Performance issue when writing large data

Question

Performance issue when writing large data

Closed this issue 7 years ago · 3 comments

For some reason I need to generate a very large excel file, which has 700 cols and 13,000 rows. On my 4 core server it takes nearly 30 minutes to write file.

Because the file is created at the very end of the process (and is fast), I guess the bottle neck is on generating the XML in memory. But I'm not sure if it's an Elixir performance issue or this library's. I also have several steps to transform the data to %Worksheet{} (without stream), and it's very fast.

Answer 1 · 2017-11-29T06:14:34.000Z

Looks pretty similar to #22.

Answer 2 · 2017-11-29T07:36:00.000Z

Is write_to_memory/2 used to generate excel without file writing? I'll try that. BTW can you recommend some ways to do benchmark in Elixir?

Answer 3 · 2017-11-29T11:21:32.000Z

Yep, that it is.
You can profile elixir app the same way as erlang or use something like ExProf.