allenai/dolma

Is there a way to intergratge Dolma toolkit to Spark?

DangoWang opened this issue · 1 comments

My single computer is not powerful enough to run Dolma :(

hi @DangoWang. I haven't investigated spark integration, but you should be able to wrap individual taggers in spark.