Performance in CPM Dataloaders During jieba Prefix Dict Construction, Stuck at "Prefix dict has been built successfully", Taking Around 40s, Possibly Machine-Dependent
clemente0731 opened this issue · 0 comments
clemente0731 commented
While utilizing the data loaders provided by CPM , observed a significant performance issue during the construction of the prefix dictionary in the jieba library.
Specifically, this step gets stuck at the message "Prefix dict has been built successfully," taking approximately 40 seconds to complete.
We suspect that this problem might be related to variations in machine performance.