RUCAIBox/TextBox

2.0.0版本运行install.sh报错

Closed this issue · 34 comments

ERROR: Could not find a version that satisfies the requirement rouge-score>=0.1.2 (from versions: 0.0.1, 0.0.2, 0.0.3, 0.0.4)

rouge-score装不上0.1.2版本请问怎么解决

你可以尝试换个pip的源,比如清华源

你可以尝试换个pip的源,比如清华源

感谢回复,但是我又遇到了新的error请问下这个怎么解决

rm: cannot remove ‘/home/cqy/.files2rouge/data/WordNet-2.0-Exceptions/WordNet-2.0.exc.db’: No such file or directory
Can't locate DB_File.pm in @inc (@inc contains: /usr/local/lib64/perl5 /usr/local/share/perl5 /usr/lib64/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5 .) at /home/cqy/.files2rouge/data/WordNet-2.0-Exceptions/buildExeptionDB.pl line 2.
BEGIN failed--compilation aborted at /home/cqy/.files2rouge/data/WordNet-2.0-Exceptions/buildExeptionDB.pl line 2.
chmod: cannot operate on dangling symlink ‘/home/cqy/.files2rouge/data/WordNet-2.0.exc.db’
chmod: cannot access ‘/home/cqy/.files2rouge/data/WordNet-2.0-Exceptions/WordNet-2.0.exc.db’: No such file or directory

请直接运行install.sh进行安装,而不要只运行其中的一部分

请直接运行install.sh进行安装,而不要只运行其中的一部分

我就是直接运行的install.sh,只截取了报错的部分

那请展示从运行之后所有的输出

或者如果files2rouge已经正确安装,则可以忽略上述命令

-> ./install.sh

████████╗███████╗██╗ ██╗████████╗██████╗ ██████╗ ██╗ ██╗
╚══██╔══╝██╔════╝╚██╗██╔╝╚══██╔══╝██╔══██╗██╔═══██╗╚██╗██╔╝
██║ █████╗ ╚███╔╝ ██║ ██████╔╝██║ ██║ ╚███╔╝
██║ ██╔══╝ ██╔██╗ ██║ ██╔══██╗██║ ██║ ██╔██╗
██║ ███████╗██╔╝ ██╗ ██║ ██████╔╝╚██████╔╝██╔╝ ██╗
╚═╝ ╚══════╝╚═╝ ╚═╝ ╚═╝ ╚═════╝ ╚═════╝ ╚═╝ ╚═╝

A modified version of transformers will be installed to python environment. Create a new conda environment (TextBox)? (y/n) y
Creating conda environment named TextBox (python=3.8) ...
Collecting package metadata (current_repodata.json): done
Solving environment: done

==> WARNING: A newer version of conda exists. <==
current version: 4.12.0
latest version: 22.11.1

Please update conda by running

$ conda update -n base -c defaults conda

Package Plan

environment location: /home/cqy/anaconda3/envs/TextBox

added / updated specs:
- python=3.8

The following NEW packages will be INSTALLED:

_libgcc_mutex anaconda/pkgs/main/linux-64::_libgcc_mutex-0.1-main
_openmp_mutex anaconda/pkgs/main/linux-64::_openmp_mutex-5.1-1_gnu
ca-certificates anaconda/pkgs/main/linux-64::ca-certificates-2022.10.11-h06a4308_0
certifi anaconda/pkgs/main/linux-64::certifi-2022.12.7-py38h06a4308_0
ld_impl_linux-64 anaconda/pkgs/main/linux-64::ld_impl_linux-64-2.38-h1181459_1
libffi anaconda/pkgs/main/linux-64::libffi-3.4.2-h6a678d5_6
libgcc-ng anaconda/pkgs/main/linux-64::libgcc-ng-11.2.0-h1234567_1
libgomp anaconda/pkgs/main/linux-64::libgomp-11.2.0-h1234567_1
libstdcxx-ng anaconda/pkgs/main/linux-64::libstdcxx-ng-11.2.0-h1234567_1
ncurses anaconda/pkgs/main/linux-64::ncurses-6.3-h5eee18b_3
openssl anaconda/pkgs/main/linux-64::openssl-1.1.1s-h7f8727e_0
pip anaconda/pkgs/main/linux-64::pip-22.3.1-py38h06a4308_0
python anaconda/pkgs/main/linux-64::python-3.8.15-h7a1cb2a_2
readline anaconda/pkgs/main/linux-64::readline-8.2-h5eee18b_0
setuptools anaconda/pkgs/main/linux-64::setuptools-65.6.3-py38h06a4308_0
sqlite anaconda/pkgs/main/linux-64::sqlite-3.40.1-h5082296_0
tk anaconda/pkgs/main/linux-64::tk-8.6.12-h1ccaba5_0
wheel anaconda/pkgs/main/noarch::wheel-0.37.1-pyhd3eb1b0_0
xz anaconda/pkgs/main/linux-64::xz-5.2.8-h5eee18b_0
zlib anaconda/pkgs/main/linux-64::zlib-1.2.13-h5eee18b_0

Proceed ([y]/n)?

Preparing transaction: done
Verifying transaction: done
Executing transaction: done

To activate this environment, use

$ conda activate TextBox

To deactivate an active environment, use

$ conda deactivate

no change /home/cqy/anaconda3/condabin/conda
no change /home/cqy/anaconda3/bin/conda
no change /home/cqy/anaconda3/bin/conda-env
no change /home/cqy/anaconda3/bin/activate
no change /home/cqy/anaconda3/bin/deactivate
no change /home/cqy/anaconda3/etc/profile.d/conda.sh
no change /home/cqy/anaconda3/etc/fish/conf.d/conda.fish
no change /home/cqy/anaconda3/shell/condabin/Conda.psm1
no change /home/cqy/anaconda3/shell/condabin/conda-hook.ps1
no change /home/cqy/anaconda3/lib/python3.8/site-packages/xontrib/conda.xsh
no change /home/cqy/anaconda3/etc/profile.d/conda.csh
no change /home/cqy/.bashrc
No action taken.
Installation may take a few minutes.
Installing torch ...
Collecting package metadata (current_repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Collecting package metadata (repodata.json): done
Solving environment: done

==> WARNING: A newer version of conda exists. <==
current version: 4.12.0
latest version: 22.11.1

Please update conda by running

$ conda update -n base -c defaults conda

Package Plan

environment location: /home/cqy/anaconda3/envs/TextBox

added / updated specs:
- cudatoolkit=10.2
- pytorch==1.12.1
- torchaudio==0.12.1
- torchvision==0.13.1

The following NEW packages will be INSTALLED:

blas anaconda/pkgs/main/linux-64::blas-1.0-mkl
brotlipy anaconda/pkgs/main/linux-64::brotlipy-0.7.0-py38h27cfd23_1003
bzip2 anaconda/pkgs/main/linux-64::bzip2-1.0.8-h7b6447c_0
cffi anaconda/pkgs/main/linux-64::cffi-1.15.1-py38h5eee18b_3
charset-normalizer anaconda/pkgs/main/noarch::charset-normalizer-2.0.4-pyhd3eb1b0_0
cryptography anaconda/pkgs/main/linux-64::cryptography-38.0.4-py38h9ce1e76_0
cudatoolkit anaconda/pkgs/main/linux-64::cudatoolkit-10.2.89-hfd86e86_1
ffmpeg pytorch/linux-64::ffmpeg-4.3-hf484d3e_0
flit-core anaconda/pkgs/main/noarch::flit-core-3.6.0-pyhd3eb1b0_0
freetype anaconda/pkgs/main/linux-64::freetype-2.12.1-h4a9f257_0
giflib anaconda/pkgs/main/linux-64::giflib-5.2.1-h7b6447c_0
gmp anaconda/pkgs/main/linux-64::gmp-6.2.1-h295c915_3
gnutls anaconda/pkgs/main/linux-64::gnutls-3.6.15-he1e5248_0
idna anaconda/pkgs/main/linux-64::idna-3.4-py38h06a4308_0
intel-openmp anaconda/pkgs/main/linux-64::intel-openmp-2021.4.0-h06a4308_3561
jpeg anaconda/pkgs/main/linux-64::jpeg-9e-h7f8727e_0
lame anaconda/pkgs/main/linux-64::lame-3.100-h7b6447c_0
lcms2 anaconda/pkgs/main/linux-64::lcms2-2.12-h3be6417_0
lerc anaconda/pkgs/main/linux-64::lerc-3.0-h295c915_0
libdeflate anaconda/pkgs/main/linux-64::libdeflate-1.8-h7f8727e_5
libiconv anaconda/pkgs/main/linux-64::libiconv-1.16-h7f8727e_2
libidn2 anaconda/pkgs/main/linux-64::libidn2-2.3.2-h7f8727e_0
libpng anaconda/pkgs/main/linux-64::libpng-1.6.37-hbc83047_0
libtasn1 anaconda/pkgs/main/linux-64::libtasn1-4.16.0-h27cfd23_0
libtiff anaconda/pkgs/main/linux-64::libtiff-4.4.0-hecacb30_2
libunistring anaconda/pkgs/main/linux-64::libunistring-0.9.10-h27cfd23_0
libwebp anaconda/pkgs/main/linux-64::libwebp-1.2.4-h11a3e52_0
libwebp-base anaconda/pkgs/main/linux-64::libwebp-base-1.2.4-h5eee18b_0
lz4-c anaconda/pkgs/main/linux-64::lz4-c-1.9.4-h6a678d5_0
mkl anaconda/pkgs/main/linux-64::mkl-2021.4.0-h06a4308_640
mkl-service anaconda/pkgs/main/linux-64::mkl-service-2.4.0-py38h7f8727e_0
mkl_fft anaconda/pkgs/main/linux-64::mkl_fft-1.3.1-py38hd3c417c_0
mkl_random anaconda/pkgs/main/linux-64::mkl_random-1.2.2-py38h51133e4_0
nettle anaconda/pkgs/main/linux-64::nettle-3.7.3-hbbd107a_1
numpy anaconda/pkgs/main/linux-64::numpy-1.23.5-py38h14f4228_0
numpy-base anaconda/pkgs/main/linux-64::numpy-base-1.23.5-py38h31eccc5_0
openh264 anaconda/pkgs/main/linux-64::openh264-2.1.1-h4ff587b_0
pillow anaconda/pkgs/main/linux-64::pillow-9.3.0-py38hace64e9_1
pycparser anaconda/pkgs/main/noarch::pycparser-2.21-pyhd3eb1b0_0
pyopenssl anaconda/pkgs/main/noarch::pyopenssl-22.0.0-pyhd3eb1b0_0
pysocks anaconda/pkgs/main/linux-64::pysocks-1.7.1-py38h06a4308_0
pytorch pytorch/linux-64::pytorch-1.12.1-py3.8_cuda10.2_cudnn7.6.5_0
pytorch-mutex pytorch/noarch::pytorch-mutex-1.0-cuda
requests anaconda/pkgs/main/linux-64::requests-2.28.1-py38h06a4308_0
six anaconda/pkgs/main/noarch::six-1.16.0-pyhd3eb1b0_1
torchaudio pytorch/linux-64::torchaudio-0.12.1-py38_cu102
torchvision pytorch/linux-64::torchvision-0.13.1-py38_cu102
typing_extensions anaconda/pkgs/main/linux-64::typing_extensions-4.4.0-py38h06a4308_0
urllib3 anaconda/pkgs/main/linux-64::urllib3-1.26.13-py38h06a4308_0
zstd anaconda/pkgs/main/linux-64::zstd-1.5.2-ha4553b6_0

Proceed ([y]/n)?

Preparing transaction: done
Verifying transaction: done
Executing transaction: done
Installing requirements ...
Looking in indexes: https://mirror.baidu.com/pypi/simple
Requirement already satisfied: numpy in /home/cqy/anaconda3/envs/TextBox/lib/python3.8/site-packages (from -r requirements.txt (line 1)) (1.23.5)
Collecting matplotlib
Using cached https://mirror.baidu.com/pypi/packages/c4/49/80590115bcae4bd8ffb30fc52df459d785dd0f1542312c8c996b7206747c/matplotlib-3.6.3-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (9.4 MB)
Collecting tqdm
Using cached https://mirror.baidu.com/pypi/packages/47/bb/849011636c4da2e44f1253cd927cfb20ada4374d8b3a4e425416e84900cc/tqdm-4.64.1-py2.py3-none-any.whl (78 kB)
Collecting nltk
Using cached https://mirror.baidu.com/pypi/packages/a6/0a/0d20d2c0f16be91b9fa32a77b76c60f9baf6eba419e5ef5deca17af9c582/nltk-3.8.1-py3-none-any.whl (1.5 MB)
Collecting pyyaml
Using cached https://mirror.baidu.com/pypi/packages/d7/42/7ad4b6d67a16229496d4f6e74201bdbebcf4bc1e87d5a70c9297d4961bd2/PyYAML-6.0-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (701 kB)
Collecting sentencepiece
Using cached https://mirror.baidu.com/pypi/packages/68/91/ded0f64f90abfc5413c620fc345a0aef1e7ff5addda8704cc6b3bf589c64/sentencepiece-0.1.96-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.2 MB)
Collecting pycocoevalcap
Using cached https://mirror.baidu.com/pypi/packages/08/f9/466f289f1628296b5e368940f89e3cfcfb066d15ddc02ff536dc532b1c93/pycocoevalcap-1.2-py3-none-any.whl (104.3 MB)
ERROR: Could not find a version that satisfies the requirement rouge-score>=0.1.2 (from versions: 0.0.1, 0.0.2, 0.0.3, 0.0.4)
ERROR: No matching distribution found for rouge-score>=0.1.2
Installing requirements (fast-bleu) ...
Installing requirements (fastseq) ...
Running command git clone --filter=blob:none --quiet https://github.com/microsoft/fastseq.git /tmp/pip-req-build-pztflk0s
Installing requirements (rouge) ...
Running command git clone --filter=blob:none --quiet https://github.com/pltrdy/pyrouge.git /tmp/pip-req-build-3kha0188
Cloning into 'files2rouge'...
remote: Enumerating objects: 278, done.
remote: Counting objects: 100% (20/20), done.
remote: Compressing objects: 100% (14/14), done.
remote: Total 278 (delta 9), reused 11 (delta 6), pack-reused 258
Receiving objects: 100% (278/278), 212.16 KiB | 1.02 MiB/s, done.
Resolving deltas: 100% (133/133), done.
/home/cqy/anaconda3/envs/TextBox/lib/python3.8/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
warnings.warn(
/home/cqy/anaconda3/envs/TextBox/lib/python3.8/site-packages/setuptools/command/easy_install.py:144: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools.
warnings.warn(
zip_safe flag not set; analyzing archive contents...
files2rouge.pycache.settings.cpython-38: module references file
rm: cannot remove ‘/home/cqy/.files2rouge/data/WordNet-2.0-Exceptions/WordNet-2.0.exc.db’: No such file or directory
Can't locate DB_File.pm in @inc (@inc contains: /usr/local/lib64/perl5 /usr/local/share/perl5 /usr/lib64/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5 .) at /home/cqy/.files2rouge/data/WordNet-2.0-Exceptions/buildExeptionDB.pl line 2.
BEGIN failed--compilation aborted at /home/cqy/.files2rouge/data/WordNet-2.0-Exceptions/buildExeptionDB.pl line 2.
chmod: cannot operate on dangling symlink ‘/home/cqy/.files2rouge/data/WordNet-2.0.exc.db’
chmod: cannot access ‘/home/cqy/.files2rouge/data/WordNet-2.0-Exceptions/WordNet-2.0.exc.db’: No such file or directory
WARNING: Skipping py-rouge as it is not installed.

我看这报错是不是files2rouge不能用setup.py安装啊?

如果files2rouge已经正确安装,则可以忽略上述命令

如果files2rouge已经正确安装,则可以忽略上述命令

请问下怎么判断files2rouge正确安装了啊,只要有那个文件夹就行了嘛?还有就是~/.files2rouge用不到嘛

命令行中输入files2rouge,显示如下字段

usage: files2rouge [-h] [-v] [-n ROUGE_N] [-nol] [-a ARGS] [-s SAVETO] [-e EOS] [-m] [-nm]
                   [-ir] [-is]
                   reference summary

命令行中输入files2rouge,显示如下字段

usage: files2rouge [-h] [-v] [-n ROUGE_N] [-nol] [-a ARGS] [-s SAVETO] [-e EOS] [-m] [-nm]
                   [-ir] [-is]
                   reference summary
usage: files2rouge [-h] [-v] [-n ROUGE_N] [-nol] [-a ARGS] [-s SAVETO] [-e EOS] [-m] [-nm] [-ir] [-is] reference summary
files2rouge: error: the following arguments are required: reference, summary

error可以忽略嘛

感谢回复,那环境就先这样了,跑跑demo看看会不会出bug

在运行官方训练代码python run_textbox.py --model=BART --dataset=samsum --model_path=facebook/bart-base还是会有error,这个问题有办法解决吗

Can't locate XML/Parser.pm in @INC (@INC contains: /home/cqy/.files2rouge /usr/local/lib64/perl5 /usr/local/share/perl5 /usr/lib64/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5 .) at /home/cqy/.files2rouge/XML/DOM.pm line 41.
BEGIN failed--compilation aborted at /home/cqy/.files2rouge/XML/DOM.pm line 70.
Compilation failed in require at /home/cqy/.files2rouge/ROUGE-1.5.5.pl line 177.
BEGIN failed--compilation aborted at /home/cqy/.files2rouge/ROUGE-1.5.5.pl line 177.
12 Jan 18:00    ERROR Traceback (most recent call last):
  File "/home/cqy/workspace/InterestGraph/video_understanding/TextBox/textbox/utils/dashboard.py", line 312, in new_experiment
    yield True
  File "/home/cqy/workspace/InterestGraph/video_understanding/TextBox/textbox/quick_start/experiment.py", line 136, in run
    self._do_train_and_valid()
  File "/home/cqy/workspace/InterestGraph/video_understanding/TextBox/textbox/quick_start/experiment.py", line 111, in _do_train_and_valid
    self.valid_result = self.trainer.fit(train_data, valid_data)
  File "/home/cqy/workspace/InterestGraph/video_understanding/TextBox/textbox/trainer/trainer.py", line 455, in fit
    self.stopped |= self._valid(valid_data, 'epoch')
  File "/home/cqy/anaconda3/envs/TextBox/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
    return func(*args, **kwargs)
  File "/home/cqy/workspace/InterestGraph/video_understanding/TextBox/textbox/trainer/trainer.py", line 294, in _valid
    valid_results = self.evaluate(valid_data, is_valid=True)
  File "/home/cqy/anaconda3/envs/TextBox/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
    return func(*args, **kwargs)
  File "/home/cqy/workspace/InterestGraph/video_understanding/TextBox/textbox/trainer/trainer.py", line 565, in evaluate
    result = self.evaluator.evaluate(generate_corpus, reference_dataset)
  File "/home/cqy/workspace/InterestGraph/video_understanding/TextBox/textbox/evaluator/base_evaluator.py", line 156, in evaluate
    metric_result = evaluator.evaluate(generate_corpus, reference_corpus, avg=avg)
  File "/home/cqy/workspace/InterestGraph/video_understanding/TextBox/textbox/evaluator/abstract_evaluator.py", line 31, in evaluate
    metric_dict = self._calc_metrics_info(generate_corpus=generate_corpus, reference_corpus=reference_corpus)
  File "/home/cqy/workspace/InterestGraph/video_understanding/TextBox/textbox/evaluator/rouge_evaluator.py", line 57, in _calc_metrics_info
    output = r.convert_and_evaluate(rouge_args=rouge_args_str)
  File "/home/cqy/anaconda3/envs/TextBox/lib/python3.8/site-packages/pyrouge/Rouge155.py", line 368, in convert_and_evaluate
    rouge_output = self.evaluate(system_id, rouge_args)
  File "/home/cqy/anaconda3/envs/TextBox/lib/python3.8/site-packages/pyrouge/Rouge155.py", line 343, in evaluate
    rouge_output = check_output(command, env=env).decode("UTF-8")
  File "/home/cqy/anaconda3/envs/TextBox/lib/python3.8/subprocess.py", line 415, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
  File "/home/cqy/anaconda3/envs/TextBox/lib/python3.8/subprocess.py", line 516, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/home/cqy/.files2rouge/ROUGE-1.5.5.pl', '-e', '/home/cqy/.files2rouge/data', '-c', '95', '-r', '1000', '-n', '2', '-w', '1.2', '-a', '-m', '/tmp/tmpszg0n7fm/rouge_conf.xml']' returned non-zero exit status 2.

安装一下perl

sudo apt-get install libxml-parser-perl

安装一下perl

sudo apt-get install libxml-parser-perl

我这服务器是centos的,找到了类似方案装xml::parser,但是貌似是源的问题总是timeout,心累...

这个似乎没有办法解决,建议自寻可以突破的办法吧

或者如果你不需要进行摘要任务,可以忽略错误

或者如果你不需要进行摘要任务,可以忽略错误

我这边也需要做类似摘要的生成任务,这个error看来必须得解决了

或者如果你不需要进行摘要任务,可以忽略错误

我从前天晚上弄这个环境,弄到现在凌晨,那个rouge的问题还是没解决,centos还要用perl装XML::Parser,这个XML::Parser又有依赖装不上,真的劝退了,装环境从来没装的这么痛苦过

抱歉,这个是files2rouge的要求

要准确的评价摘要任务需要这个包的支持,建议使用Ubuntu系统,我们确实没有在centos系统上进行过测试

请问你需要进行什么任务的评测?中文还是英文?

请问你需要进行什么任务的评测?中文还是英文?

中文任务,类似摘要的生成任务

中文的话可以不使用files2rouge进行评测,你可以下载https://huggingface.co/datasets/RUCAIBox/Chinese-Generation/blob/main/csl.tgz 这个数据集进行测试

中文的话可以不使用files2rouge进行评测,你可以下载https://huggingface.co/datasets/RUCAIBox/Chinese-Generation/blob/main/csl.tgz 这个数据集进行测试

好的,我明天试下

我跑csl数据集没有再报错了,但大致看了一下源码之后有两个问题希望解答一下:
(1)我在设置了2张卡之后,训练时只用了设置的第一张卡,在init_device里面看了下设置多卡的方式我应该没写错并且print看了下CUDA_VISIBLE_DEVICE都是两张卡

python run_textbox.py \
    --gpu_id=\(2,3\) \
    --model=Chinese-BART \
    --dataset=csl \
    --model_path=fnlp/bart-base-chinese \
    --saved_dir=./saved/DEBUG/ \
    --epochs=3 \
    --train_batch_size=32 \
    --eval_batch_size=64 \
    --max_save=1 \
    --wandb=offline \

(2)好像只保存inner model相关的参数config.json,通过配置文件、cmd传入的参数不会保存是吗

(1)你是要多卡DDP训练吗?
(2)config参数在project.log中

(1)是的,单机多卡;(2)个人感觉这些外部参数可以保存到一个文件里,看起来也方便

(1)单机多卡请参考:https://github.com/RUCAIBox/TextBox/blob/2.0.0/asset/efficient_training.md#distributed-data-parallel (2)之后可能会考虑你的建议

好的,谢谢

请问一下,我用chinese-bart,target里面添加了两个新的token,在配置文件里面也添加了'additional_special_tokens': ['[EMP]', '[DEL]'],但是预测的结果没有这两个token,这个是为什么啊

decode_kwargs = {'skip_special_tokens': True, 'clean_up_tokenization_spaces': False}

按照hugging face,我们在生成的时候会跳过特殊字符(例如[SOS],[PAD]),如果你需要添加普通的字符,请通过tokenizer_add_tokens添加
tokenizer_add_tokens: []

此外,如果安装没有问题,请关闭这个issue,与安装无关的问题麻烦新开issue

好的,谢谢