zrlio/crail-spark-terasort

Partition size >2GB leads to undefined behaviour

PepperJo opened this issue · 0 comments

Partition size >2GB leads to undefined behaviour. Currently we cannot support more than 2GB since we hit the size limit of the ByteBuffer in Java. We should introduce an error message if someone tries to set >2GB.