xorbitsai/xorbits

BUG: FileNotFoundError: [Errno 2] No such file or directory

simplew2011 opened this issue · 2 comments

Describe the bug

A clear and concise description of what the bug is.
image

To Reproduce

To help us to reproduce this bug, please provide information below:

# download dataset
wget https://atp-modelzoo.oss-cn-hangzhou.aliyuncs.com/release/datasets/WuDaoCorpus2.0_base_sample.tgz
tar zxvf WuDaoCorpus2.0_base_sample.tgz
import xorbits.pandas as pd
import xorbits.datasets as xdatasets
from datasets import load_dataset

import xorbits
xorbits.init()

data = load_dataset("./WuDaoCorpus2.0_base_sample")
print(data)

df = pd.DataFrame(pd.DataFrame(data['train']), chunk_size=1000)
print(df.shape)
print(df.dtypes)
print(df.head())
from xorbits.experimental import dedup

res = dedup(df, col="content")
print(10*"---")
print(res)
  1. Your Python version:3.8.5
  2. The version of Xorbits you use:0.7.1
  3. Versions of crucial packages, such as numpy, scipy and pandas
  4. Full stack of the error.
  5. Minimized code to reproduce the error.

Expected behavior

A clear and concise description of what you expected to happen.

Additional context

Add any other context about the problem here.

add a line is ok:
xorbits.shutdown()

So, it was raised when process exit?