Dataset can read/load only first n rows of csv file.
sandeepsajan0 opened this issue · 2 comments
If we have a huge csv file and we can test our code only by first n rows then we shouldn't need to load the whole file. It'll be a great feature. I tried to find it but couldn't if it already exists there, can anyone of you guys tell me how can I use that feature?
Or we should implement that feature?
**Edited:
For now, I am just using the list comprehension with del method on loaded data to pass. But it'll be better if I can load only desired data.
What are your ideas?
@sandeepsajan0 the common solution is to read/parse csv file with a generator, so you can iterate over your csv lazily without loading the entire file into memory (and allowing you to peek the first n files).
Closing, as the solution is to iterate lazily using a generator.