Change int to long for number of records passed to list...
rajib76 opened this issue · 4 comments
I will work on it and send a PR shortly. Currently the code takes only int for number of records. If we need billions of records created, we will need to change it to long
default MockUnit<List> list(Supplier<List> listSupplier, int size)
I am planning to add another method which will take long as an input.
It will work till 2,147,483,647. For more than this we will need long.
I don't think someone needs to keep in memory a list bigger than Integer max.
Hmm, I agree with you but should we look at any other option on how we can make it work if the need is to create large data let's say 3 billion records. May be some option to create the data in splits and write it in disk.
It's highly improbable someone would want to create a 3 billion records data set by keeping them in memory. Also a file that big is ... very big.
Still if you want to write data in chunks you can do it in a series of way:
For example:
MockNeat m = MockNeat.threadLocal();
int numStep = 10;
int listSize = 1000;
m.csvs()
.column(m.intSeq())
.column(m.names())
.column(m.cities().capitals())
.column(m.days())
.column(m.creditCards().amex())
.list(() -> new ArrayList<>(), listSize)
.consume(numStep, (step, list) -> {
try { Files.write(Paths.get("test.csv"), list, CREATE, WRITE, APPEND); }
catch (IOException e) { e.printStackTrace(); }
});