BUG: FileNotFoundError: [Errno 2] No such file or directory
simplew2011 opened this issue · 2 comments
simplew2011 commented
Describe the bug
A clear and concise description of what the bug is.
To Reproduce
To help us to reproduce this bug, please provide information below:
# download dataset
wget https://atp-modelzoo.oss-cn-hangzhou.aliyuncs.com/release/datasets/WuDaoCorpus2.0_base_sample.tgz
tar zxvf WuDaoCorpus2.0_base_sample.tgz
import xorbits.pandas as pd
import xorbits.datasets as xdatasets
from datasets import load_dataset
import xorbits
xorbits.init()
data = load_dataset("./WuDaoCorpus2.0_base_sample")
print(data)
df = pd.DataFrame(pd.DataFrame(data['train']), chunk_size=1000)
print(df.shape)
print(df.dtypes)
print(df.head())
from xorbits.experimental import dedup
res = dedup(df, col="content")
print(10*"---")
print(res)
- Your Python version:3.8.5
- The version of Xorbits you use:0.7.1
- Versions of crucial packages, such as numpy, scipy and pandas
- Full stack of the error.
- Minimized code to reproduce the error.
Expected behavior
A clear and concise description of what you expected to happen.
Additional context
Add any other context about the problem here.
simplew2011 commented
add a line is ok:
xorbits.shutdown()
aresnow1 commented
So, it was raised when process exit?