EleutherAI/dps

Add Text length Stats for datasets

Closed this issue · 0 comments

Taekyoon commented

Context

  • Output stats for text length in dataset
    • mean
    • median
    • min
    • max
    • 75 percentile
    • std