allenai/dolma

make_wikipedia in getting_started.md

leeparkuky opened this issue · 3 comments

The code for running make_wikipedia needs to be edited. The one currently in the document

python scripts/make_wikipedia.py
--output wikipedia
--languages simple
--date 20231001
--lang simple
--num_processes 16

should be written as below:

python scripts/make_wikipedia.py
--output wikipedia
--date 20231001
--lang simple
--processes 16

Thank you for the report! Would you have bandwidth for a quick PR?

@soldni I have already submitted that PR which is resolving this issue.

closing since fixed in #117. Thanks again @RohitRathore1!