/Crystydatasetprocess

Some ProcessScript used on my own dataset

Primary LanguagePythonApache License 2.0Apache-2.0

Crystydatasetprocess

Some ProcessScript used on my own dataset 1.Qwen DatasetGeneration script Can process specifc dataset to QwenFintuning Format

2.RWKVProcess used to erase all square and "/n" and replace specific mark"####" onto /n to match dataset format requirment.