HTTP Error 400 when pushing model to HuggingFace hub

Question

HTTP Error 400 when pushing model to HuggingFace hub

Lucaterre opened this issue 2 years ago · 24 comments

Hello,

I'm not quite sure if this issue is related to #5.

When I'm trying to push a model on Hugging Face Hub organisation with spaCy CLI:

python -m spacy huggingface-hub push fr_core_ner4archives_v3_default-0.0.0-py3-none-any.whl -o ner4archives -V

This raises an HTTP 400 error:

Pushing repository to the hub...
Traceback (most recent call last):
  File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/lterriel/Documents/dev/almanach-projects/N4A_project/Training_pipelines/venv/lib/python3.8/site-packages/spacy/__main__.py", line 4, in <module>
    setup_cli()
  File "/home/lterriel/Documents/dev/almanach-projects/N4A_project/Training_pipelines/venv/lib/python3.8/site-packages/spacy/cli/_util.py", line 71, in setup_cli
    command(prog_name=COMMAND)
  File "/home/lterriel/Documents/dev/almanach-projects/N4A_project/Training_pipelines/venv/lib/python3.8/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/home/lterriel/Documents/dev/almanach-projects/N4A_project/Training_pipelines/venv/lib/python3.8/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/home/lterriel/Documents/dev/almanach-projects/N4A_project/Training_pipelines/venv/lib/python3.8/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/lterriel/Documents/dev/almanach-projects/N4A_project/Training_pipelines/venv/lib/python3.8/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/lterriel/Documents/dev/almanach-projects/N4A_project/Training_pipelines/venv/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/lterriel/Documents/dev/almanach-projects/N4A_project/Training_pipelines/venv/lib/python3.8/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/home/lterriel/Documents/dev/almanach-projects/N4A_project/Training_pipelines/venv/lib/python3.8/site-packages/typer/main.py", line 497, in wrapper
    return callback(**use_params)  # type: ignore
  File "/home/lterriel/Documents/dev/almanach-projects/N4A_project/Training_pipelines/venv/lib/python3.8/site-packages/spacy_huggingface_hub/push.py", line 53, in huggingface_hub_push_cli
    push(whl_path, organization, commit_msg, verbose=verbose)
  File "/home/lterriel/Documents/dev/almanach-projects/N4A_project/Training_pipelines/venv/lib/python3.8/site-packages/spacy_huggingface_hub/push.py", line 130, in push
    url = upload_folder(
  File "/home/lterriel/Documents/dev/almanach-projects/N4A_project/Training_pipelines/venv/lib/python3.8/site-packages/huggingface_hub/hf_api.py", line 2109, in upload_folder
    pr_url = self.create_commit(
  File "/home/lterriel/Documents/dev/almanach-projects/N4A_project/Training_pipelines/venv/lib/python3.8/site-packages/huggingface_hub/hf_api.py", line 1844, in create_commit
    _raise_for_status(commit_resp)
  File "/home/lterriel/Documents/dev/almanach-projects/N4A_project/Training_pipelines/venv/lib/python3.8/site-packages/huggingface_hub/utils/_errors.py", line 84, in _raise_for_status
    _raise_with_request_id(request)
  File "/home/lterriel/Documents/dev/almanach-projects/N4A_project/Training_pipelines/venv/lib/python3.8/site-packages/huggingface_hub/utils/_errors.py", line 95, in _raise_with_request_id
    raise e
  File "/home/lterriel/Documents/dev/almanach-projects/N4A_project/Training_pipelines/venv/lib/python3.8/site-packages/huggingface_hub/utils/_errors.py", line 90, in _raise_with_request_id
    request.raise_for_status()
  File "/home/lterriel/Documents/dev/almanach-projects/N4A_project/Training_pipelines/venv/lib/python3.8/site-packages/requests/models.py", line 1021, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: https://huggingface.co/api/models/lterriel/fr_core_ner4archives_v3_default/commit/main (Request ID: QTOTcNAiXjAsn45fwTKLO)

However, the new model repository is well created in Hugging Face Hub organisation but without the model (files) and its card.

packages installed:

spaCy==3.3.1
spacy-huggingface-hub==0.0.7
huggingface-hub==0.8.1

Answer 1 · 2022-09-21T17:46:01.000Z

Hey there! I just tried with the same package versions and was able to successfully upload at model, which you can check here https://huggingface.co/osanseviero/da_core_news_sm

Would you be able to try again to see if you can reproduce the error please?

Answer 2 · 2022-09-22T06:43:03.000Z

Hi @Lucaterre 👋 To help us diagnose this could try to update huggingface_hub to version 0.9.1 ? It will most probably not solve the issue but error messages have been refactored in v0.9 to provide more information.

Answer 3 · 2022-09-22T10:28:45.000Z

Sorry for delay, thanks for your response.

@osanseviero

I installed the packages again with same version, I restarted the process and I get the same error:

(spacy_venv) lterriel@pl767-pro:~/Documents/dev/almanach-projects/N4A_project/Training_pipelines/spacy/NER4Archives_workflows$ spacy project run push_to_hub

================================ push_to_hub ================================
Running command: /home/lterriel/Documents/dev/almanach-projects/N4A_project/Training_pipelines/spacy/NER4Archives_workflows/spacy_venv/bin/python3.8 -m spacy huggingface-hub push ./packages/fr_core_ner4archives_v3_default-0.0.0/dist/fr_core_ner4archives_v3_default-0.0.0-py3-none-any.whl -o ner4archives
ℹ Publishing to repository
'ner4archives/fr_core_ner4archives_v3_default'
result /tmp/tmpsgyz5z9z/fr_core_ner4archives_v3_default/meta.json fr_core_ner4archives_v3_default/meta.json
result /tmp/tmpsgyz5z9z/fr_core_ner4archives_v3_default/fr_core_ner4archives_v3_default-0.0.0/README.md fr_core_ner4archives_v3_default/fr_core_ner4archives_v3_default-0.0.0/README.md
result /tmp/tmpsgyz5z9z/fr_core_ner4archives_v3_default/fr_core_ner4archives_v3_default-0.0.0/config.cfg fr_core_ner4archives_v3_default/fr_core_ner4archives_v3_default-0.0.0/config.cfg
result /tmp/tmpsgyz5z9z/fr_core_ner4archives_v3_default/fr_core_ner4archives_v3_default-0.0.0/meta.json fr_core_ner4archives_v3_default/fr_core_ner4archives_v3_default-0.0.0/meta.json
result /tmp/tmpsgyz5z9z/fr_core_ner4archives_v3_default/fr_core_ner4archives_v3_default-0.0.0/tokenizer fr_core_ner4archives_v3_default/fr_core_ner4archives_v3_default-0.0.0/tokenizer
result /tmp/tmpsgyz5z9z/fr_core_ner4archives_v3_default/fr_core_ner4archives_v3_default-0.0.0/ner/cfg fr_core_ner4archives_v3_default/fr_core_ner4archives_v3_default-0.0.0/ner/cfg
result /tmp/tmpsgyz5z9z/fr_core_ner4archives_v3_default/fr_core_ner4archives_v3_default-0.0.0/ner/model fr_core_ner4archives_v3_default/fr_core_ner4archives_v3_default-0.0.0/ner/model
result /tmp/tmpsgyz5z9z/fr_core_ner4archives_v3_default/fr_core_ner4archives_v3_default-0.0.0/ner/moves fr_core_ner4archives_v3_default/fr_core_ner4archives_v3_default-0.0.0/ner/moves
result /tmp/tmpsgyz5z9z/fr_core_ner4archives_v3_default/fr_core_ner4archives_v3_default-0.0.0/tok2vec/cfg fr_core_ner4archives_v3_default/fr_core_ner4archives_v3_default-0.0.0/tok2vec/cfg
result /tmp/tmpsgyz5z9z/fr_core_ner4archives_v3_default/fr_core_ner4archives_v3_default-0.0.0/tok2vec/model fr_core_ner4archives_v3_default/fr_core_ner4archives_v3_default-0.0.0/tok2vec/model
result /tmp/tmpsgyz5z9z/fr_core_ner4archives_v3_default/fr_core_ner4archives_v3_default-0.0.0/vocab/key2row fr_core_ner4archives_v3_default/fr_core_ner4archives_v3_default-0.0.0/vocab/key2row
result /tmp/tmpsgyz5z9z/fr_core_ner4archives_v3_default/fr_core_ner4archives_v3_default-0.0.0/vocab/lookups.bin fr_core_ner4archives_v3_default/fr_core_ner4archives_v3_default-0.0.0/vocab/lookups.bin
result /tmp/tmpsgyz5z9z/fr_core_ner4archives_v3_default/fr_core_ner4archives_v3_default-0.0.0/vocab/strings.json fr_core_ner4archives_v3_default/fr_core_ner4archives_v3_default-0.0.0/vocab/strings.json
result /tmp/tmpsgyz5z9z/fr_core_ner4archives_v3_default/fr_core_ner4archives_v3_default-0.0.0/vocab/vectors fr_core_ner4archives_v3_default/fr_core_ner4archives_v3_default-0.0.0/vocab/vectors
result /tmp/tmpsgyz5z9z/fr_core_ner4archives_v3_default/fr_core_ner4archives_v3_default-0.0.0/vocab/vectors.cfg fr_core_ner4archives_v3_default/fr_core_ner4archives_v3_default-0.0.0/vocab/vectors.cfg
result /tmp/tmpsgyz5z9z/fr_core_ner4archives_v3_default-0.0.0.dist-info/METADATA fr_core_ner4archives_v3_default-0.0.0.dist-info/METADATA
result /tmp/tmpsgyz5z9z/fr_core_ner4archives_v3_default-0.0.0.dist-info/WHEEL fr_core_ner4archives_v3_default-0.0.0.dist-info/WHEEL
result /tmp/tmpsgyz5z9z/fr_core_ner4archives_v3_default-0.0.0.dist-info/entry_points.txt fr_core_ner4archives_v3_default-0.0.0.dist-info/entry_points.txt
result /tmp/tmpsgyz5z9z/fr_core_ner4archives_v3_default-0.0.0.dist-info/top_level.txt fr_core_ner4archives_v3_default-0.0.0.dist-info/top_level.txt
result /tmp/tmpsgyz5z9z/fr_core_ner4archives_v3_default-0.0.0.dist-info/RECORD fr_core_ner4archives_v3_default-0.0.0.dist-info/RECORD
✔ Extracted information from .whl file
✔ Created model card
fr_core_ner4archives_v3_default (v0.0.0)
Pushing repository to the hub...
Traceback (most recent call last):
  File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/lterriel/Documents/dev/almanach-projects/N4A_project/Training_pipelines/spacy/NER4Archives_workflows/spacy_venv/lib/python3.8/site-packages/spacy/__main__.py", line 4, in <module>
    setup_cli()
  File "/home/lterriel/Documents/dev/almanach-projects/N4A_project/Training_pipelines/spacy/NER4Archives_workflows/spacy_venv/lib/python3.8/site-packages/spacy/cli/_util.py", line 71, in setup_cli
    command(prog_name=COMMAND)
  File "/home/lterriel/Documents/dev/almanach-projects/N4A_project/Training_pipelines/spacy/NER4Archives_workflows/spacy_venv/lib/python3.8/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/home/lterriel/Documents/dev/almanach-projects/N4A_project/Training_pipelines/spacy/NER4Archives_workflows/spacy_venv/lib/python3.8/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/home/lterriel/Documents/dev/almanach-projects/N4A_project/Training_pipelines/spacy/NER4Archives_workflows/spacy_venv/lib/python3.8/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/lterriel/Documents/dev/almanach-projects/N4A_project/Training_pipelines/spacy/NER4Archives_workflows/spacy_venv/lib/python3.8/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/lterriel/Documents/dev/almanach-projects/N4A_project/Training_pipelines/spacy/NER4Archives_workflows/spacy_venv/lib/python3.8/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/lterriel/Documents/dev/almanach-projects/N4A_project/Training_pipelines/spacy/NER4Archives_workflows/spacy_venv/lib/python3.8/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/home/lterriel/Documents/dev/almanach-projects/N4A_project/Training_pipelines/spacy/NER4Archives_workflows/spacy_venv/lib/python3.8/site-packages/typer/main.py", line 532, in wrapper
    return callback(**use_params)  # type: ignore
  File "/home/lterriel/Documents/dev/almanach-projects/N4A_project/Training_pipelines/spacy/NER4Archives_workflows/spacy_venv/lib/python3.8/site-packages/spacy_huggingface_hub/push.py", line 53, in huggingface_hub_push_cli
    push(whl_path, organization, commit_msg, verbose=verbose)
  File "/home/lterriel/Documents/dev/almanach-projects/N4A_project/Training_pipelines/spacy/NER4Archives_workflows/spacy_venv/lib/python3.8/site-packages/spacy_huggingface_hub/push.py", line 130, in push
    url = upload_folder(
  File "/home/lterriel/Documents/dev/almanach-projects/N4A_project/Training_pipelines/spacy/NER4Archives_workflows/spacy_venv/lib/python3.8/site-packages/huggingface_hub/hf_api.py", line 2109, in upload_folder
    pr_url = self.create_commit(
  File "/home/lterriel/Documents/dev/almanach-projects/N4A_project/Training_pipelines/spacy/NER4Archives_workflows/spacy_venv/lib/python3.8/site-packages/huggingface_hub/hf_api.py", line 1844, in create_commit
    _raise_for_status(commit_resp)
  File "/home/lterriel/Documents/dev/almanach-projects/N4A_project/Training_pipelines/spacy/NER4Archives_workflows/spacy_venv/lib/python3.8/site-packages/huggingface_hub/utils/_errors.py", line 84, in _raise_for_status
    _raise_with_request_id(request)
  File "/home/lterriel/Documents/dev/almanach-projects/N4A_project/Training_pipelines/spacy/NER4Archives_workflows/spacy_venv/lib/python3.8/site-packages/huggingface_hub/utils/_errors.py", line 95, in _raise_with_request_id
    raise e
  File "/home/lterriel/Documents/dev/almanach-projects/N4A_project/Training_pipelines/spacy/NER4Archives_workflows/spacy_venv/lib/python3.8/site-packages/huggingface_hub/utils/_errors.py", line 90, in _raise_with_request_id
    request.raise_for_status()
  File "/home/lterriel/Documents/dev/almanach-projects/N4A_project/Training_pipelines/spacy/NER4Archives_workflows/spacy_venv/lib/python3.8/site-packages/requests/models.py", line 1021, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: https://huggingface.co/api/models/ner4archives/fr_core_ner4archives_v3_default/commit/main (Request ID: PU-dAw4rlv2Sx_FiJo3-Q)

You can check, new repo is created but is still empty: https://huggingface.co/ner4archives/fr_core_ner4archives_v3_default

complete requirements in details:

altair==4.2.0
attrs==22.1.0
backports.zoneinfo==0.2.1
blinker==1.5
blis==0.7.8
cachetools==5.2.0
catalogue==2.0.8
certifi==2022.9.14
charset-normalizer==2.1.1
click==8.1.3
commonmark==0.9.1
cymem==2.0.6
decorator==5.1.1
entrypoints==0.4
filelock==3.8.0
fr-core-news-lg @ https://github.com/explosion/spacy-models/releases/download/fr_core_news_lg-3.3.0/fr_core_news_lg-3.3.0-py3-none-any.whl
fr-core-news-sm @ https://github.com/explosion/spacy-models/releases/download/fr_core_news_sm-3.3.0/fr_core_news_sm-3.3.0-py3-none-any.whl
gitdb==4.0.9
GitPython==3.1.27
huggingface-hub==0.8.1
idna==3.4
importlib-metadata==4.12.0
importlib-resources==5.9.0
Jinja2==3.1.2
jsonschema==4.16.0
langcodes==3.3.0
MarkupSafe==2.1.1
murmurhash==1.0.8
numpy==1.23.3
packaging==21.3
pandas==1.5.0
pathy==0.6.2
Pillow==9.2.0
pkg_resources==0.0.0
pkgutil_resolve_name==1.3.10
preshed==3.0.7
protobuf==3.20.2
pyarrow==9.0.0
pydantic==1.8.2
pydeck==0.8.0b3
Pygments==2.13.0
Pympler==1.0.1
pyparsing==3.0.9
pyrsistent==0.18.1
python-dateutil==2.8.2
pytz==2022.2.1
pytz-deprecation-shim==0.1.0.post0
PyYAML==6.0
requests==2.28.1
rich==12.5.1
semver==2.13.0
six==1.16.0
smart-open==5.2.1
smmap==5.0.0
spacy==3.3.1
spacy-huggingface-hub==0.0.7
spacy-legacy==3.0.10
spacy-loggers==1.0.3
spacy-streamlit==1.0.4
srsly==2.4.4
streamlit==1.12.2
thinc==8.0.17
toml==0.10.2
toolz==0.12.0
tornado==6.2
tqdm==4.64.1
typer==0.4.2
typing_extensions==4.3.0
tzdata==2022.2
tzlocal==4.2
urllib3==1.26.12
validators==0.20.0
wasabi==0.10.1
watchdog==2.1.9
zipp==3.8.1

@Wauplin

I update huggingface_hub 0.8.1 to 0.9.1 to test (this create a conflict with spacy-huggingface-hub 0.0.7)

I raise two exceptions (400 HTTP error and type error):

(spacy_venv) lterriel@pl767-pro:~/Documents/dev/almanach-projects/N4A_project/Training_pipelines/spacy/NER4Archives_workflows$ spacy project run push_to_hub

================================ push_to_hub ================================
Running command: /home/lterriel/Documents/dev/almanach-projects/N4A_project/Training_pipelines/spacy/NER4Archives_workflows/spacy_venv/bin/python3.8 -m spacy huggingface-hub push ./packages/fr_core_ner4archives_v3_default-0.0.0/dist/fr_core_ner4archives_v3_default-0.0.0-py3-none-any.whl -o ner4archives
ℹ Publishing to repository
'ner4archives/fr_core_ner4archives_v3_default'
result /tmp/tmpza7j1lxu/fr_core_ner4archives_v3_default/meta.json fr_core_ner4archives_v3_default/meta.json
result /tmp/tmpza7j1lxu/fr_core_ner4archives_v3_default/fr_core_ner4archives_v3_default-0.0.0/README.md fr_core_ner4archives_v3_default/fr_core_ner4archives_v3_default-0.0.0/README.md
result /tmp/tmpza7j1lxu/fr_core_ner4archives_v3_default/fr_core_ner4archives_v3_default-0.0.0/config.cfg fr_core_ner4archives_v3_default/fr_core_ner4archives_v3_default-0.0.0/config.cfg
result /tmp/tmpza7j1lxu/fr_core_ner4archives_v3_default/fr_core_ner4archives_v3_default-0.0.0/meta.json fr_core_ner4archives_v3_default/fr_core_ner4archives_v3_default-0.0.0/meta.json
result /tmp/tmpza7j1lxu/fr_core_ner4archives_v3_default/fr_core_ner4archives_v3_default-0.0.0/tokenizer fr_core_ner4archives_v3_default/fr_core_ner4archives_v3_default-0.0.0/tokenizer
result /tmp/tmpza7j1lxu/fr_core_ner4archives_v3_default/fr_core_ner4archives_v3_default-0.0.0/ner/cfg fr_core_ner4archives_v3_default/fr_core_ner4archives_v3_default-0.0.0/ner/cfg
result /tmp/tmpza7j1lxu/fr_core_ner4archives_v3_default/fr_core_ner4archives_v3_default-0.0.0/ner/model fr_core_ner4archives_v3_default/fr_core_ner4archives_v3_default-0.0.0/ner/model
result /tmp/tmpza7j1lxu/fr_core_ner4archives_v3_default/fr_core_ner4archives_v3_default-0.0.0/ner/moves fr_core_ner4archives_v3_default/fr_core_ner4archives_v3_default-0.0.0/ner/moves
result /tmp/tmpza7j1lxu/fr_core_ner4archives_v3_default/fr_core_ner4archives_v3_default-0.0.0/tok2vec/cfg fr_core_ner4archives_v3_default/fr_core_ner4archives_v3_default-0.0.0/tok2vec/cfg
result /tmp/tmpza7j1lxu/fr_core_ner4archives_v3_default/fr_core_ner4archives_v3_default-0.0.0/tok2vec/model fr_core_ner4archives_v3_default/fr_core_ner4archives_v3_default-0.0.0/tok2vec/model
result /tmp/tmpza7j1lxu/fr_core_ner4archives_v3_default/fr_core_ner4archives_v3_default-0.0.0/vocab/key2row fr_core_ner4archives_v3_default/fr_core_ner4archives_v3_default-0.0.0/vocab/key2row
result /tmp/tmpza7j1lxu/fr_core_ner4archives_v3_default/fr_core_ner4archives_v3_default-0.0.0/vocab/lookups.bin fr_core_ner4archives_v3_default/fr_core_ner4archives_v3_default-0.0.0/vocab/lookups.bin
result /tmp/tmpza7j1lxu/fr_core_ner4archives_v3_default/fr_core_ner4archives_v3_default-0.0.0/vocab/strings.json fr_core_ner4archives_v3_default/fr_core_ner4archives_v3_default-0.0.0/vocab/strings.json
result /tmp/tmpza7j1lxu/fr_core_ner4archives_v3_default/fr_core_ner4archives_v3_default-0.0.0/vocab/vectors fr_core_ner4archives_v3_default/fr_core_ner4archives_v3_default-0.0.0/vocab/vectors
result /tmp/tmpza7j1lxu/fr_core_ner4archives_v3_default/fr_core_ner4archives_v3_default-0.0.0/vocab/vectors.cfg fr_core_ner4archives_v3_default/fr_core_ner4archives_v3_default-0.0.0/vocab/vectors.cfg
result /tmp/tmpza7j1lxu/fr_core_ner4archives_v3_default-0.0.0.dist-info/METADATA fr_core_ner4archives_v3_default-0.0.0.dist-info/METADATA
result /tmp/tmpza7j1lxu/fr_core_ner4archives_v3_default-0.0.0.dist-info/WHEEL fr_core_ner4archives_v3_default-0.0.0.dist-info/WHEEL
result /tmp/tmpza7j1lxu/fr_core_ner4archives_v3_default-0.0.0.dist-info/entry_points.txt fr_core_ner4archives_v3_default-0.0.0.dist-info/entry_points.txt
result /tmp/tmpza7j1lxu/fr_core_ner4archives_v3_default-0.0.0.dist-info/top_level.txt fr_core_ner4archives_v3_default-0.0.0.dist-info/top_level.txt
result /tmp/tmpza7j1lxu/fr_core_ner4archives_v3_default-0.0.0.dist-info/RECORD fr_core_ner4archives_v3_default-0.0.0.dist-info/RECORD
✔ Extracted information from .whl file
✔ Created model card
fr_core_ner4archives_v3_default (v0.0.0)
Pushing repository to the hub...
Traceback (most recent call last):
  File "/home/lterriel/Documents/dev/almanach-projects/N4A_project/Training_pipelines/spacy/NER4Archives_workflows/spacy_venv/lib/python3.8/site-packages/huggingface_hub/utils/_errors.py", line 187, in _raise_convert_bad_request
    _raise_for_status(resp)
  File "/home/lterriel/Documents/dev/almanach-projects/N4A_project/Training_pipelines/spacy/NER4Archives_workflows/spacy_venv/lib/python3.8/site-packages/huggingface_hub/utils/_errors.py", line 169, in _raise_for_status
    raise e
  File "/home/lterriel/Documents/dev/almanach-projects/N4A_project/Training_pipelines/spacy/NER4Archives_workflows/spacy_venv/lib/python3.8/site-packages/huggingface_hub/utils/_errors.py", line 131, in _raise_for_status
    response.raise_for_status()
  File "/home/lterriel/Documents/dev/almanach-projects/N4A_project/Training_pipelines/spacy/NER4Archives_workflows/spacy_venv/lib/python3.8/site-packages/requests/models.py", line 1021, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: https://huggingface.co/api/models/ner4archives/fr_core_ner4archives_v3_default/commit/main (Request ID: 2IMxrr_feReh1YFkzVFQw)

-------------------------------------------------------------------------
Your push was rejected because it contains binary files.
Please use https://git-lfs.github.com/ to store binary files.
See also: https://hf.co/docs/hub/repositories-getting-started#terminal
-------------------------------------------------------------------------
Offending files:
 - tokenizer (ref: refs/heads/main)


During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/lterriel/Documents/dev/almanach-projects/N4A_project/Training_pipelines/spacy/NER4Archives_workflows/spacy_venv/lib/python3.8/site-packages/spacy/__main__.py", line 4, in <module>
    setup_cli()
  File "/home/lterriel/Documents/dev/almanach-projects/N4A_project/Training_pipelines/spacy/NER4Archives_workflows/spacy_venv/lib/python3.8/site-packages/spacy/cli/_util.py", line 71, in setup_cli
    command(prog_name=COMMAND)
  File "/home/lterriel/Documents/dev/almanach-projects/N4A_project/Training_pipelines/spacy/NER4Archives_workflows/spacy_venv/lib/python3.8/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/home/lterriel/Documents/dev/almanach-projects/N4A_project/Training_pipelines/spacy/NER4Archives_workflows/spacy_venv/lib/python3.8/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/home/lterriel/Documents/dev/almanach-projects/N4A_project/Training_pipelines/spacy/NER4Archives_workflows/spacy_venv/lib/python3.8/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/lterriel/Documents/dev/almanach-projects/N4A_project/Training_pipelines/spacy/NER4Archives_workflows/spacy_venv/lib/python3.8/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/lterriel/Documents/dev/almanach-projects/N4A_project/Training_pipelines/spacy/NER4Archives_workflows/spacy_venv/lib/python3.8/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/lterriel/Documents/dev/almanach-projects/N4A_project/Training_pipelines/spacy/NER4Archives_workflows/spacy_venv/lib/python3.8/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/home/lterriel/Documents/dev/almanach-projects/N4A_project/Training_pipelines/spacy/NER4Archives_workflows/spacy_venv/lib/python3.8/site-packages/typer/main.py", line 532, in wrapper
    return callback(**use_params)  # type: ignore
  File "/home/lterriel/Documents/dev/almanach-projects/N4A_project/Training_pipelines/spacy/NER4Archives_workflows/spacy_venv/lib/python3.8/site-packages/spacy_huggingface_hub/push.py", line 53, in huggingface_hub_push_cli
    push(whl_path, organization, commit_msg, verbose=verbose)
  File "/home/lterriel/Documents/dev/almanach-projects/N4A_project/Training_pipelines/spacy/NER4Archives_workflows/spacy_venv/lib/python3.8/site-packages/spacy_huggingface_hub/push.py", line 130, in push
    url = upload_folder(
  File "/home/lterriel/Documents/dev/almanach-projects/N4A_project/Training_pipelines/spacy/NER4Archives_workflows/spacy_venv/lib/python3.8/site-packages/huggingface_hub/hf_api.py", line 2334, in upload_folder
    pr_url = self.create_commit(
  File "/home/lterriel/Documents/dev/almanach-projects/N4A_project/Training_pipelines/spacy/NER4Archives_workflows/spacy_venv/lib/python3.8/site-packages/huggingface_hub/hf_api.py", line 2051, in create_commit
    _raise_convert_bad_request(commit_resp, endpoint_name="commit")
  File "/home/lterriel/Documents/dev/almanach-projects/N4A_project/Training_pipelines/spacy/NER4Archives_workflows/spacy_venv/lib/python3.8/site-packages/huggingface_hub/utils/_errors.py", line 195, in _raise_convert_bad_request
    raise BadRequestError(
  File "/home/lterriel/Documents/dev/almanach-projects/N4A_project/Training_pipelines/spacy/NER4Archives_workflows/spacy_venv/lib/python3.8/site-packages/huggingface_hub/utils/_errors.py", line 96, in __init__
    super().__init__(message, response=response)
TypeError: BadRequestError() takes no keyword arguments

For more details this is my spacy project workflows that I use (requirements, project.yml, cfg etc.) here:
https://github.com/NER4Archives-project/Training_pipelines/tree/main/spacy/NER4Archives_workflows

Answer 4 · 2022-09-22T13:00:03.000Z

Hi @Lucaterre thanks for testing it and for the traceback.

A few things here:

I am very sorry but I realized the new handling of HTTP errors is not yet released. Could you retry with installing directly from main branch and show again the traceback ? Thank you in advance and sorry for my mistake

pip uninstall huggingface_hub
pip install git+https://github.com/huggingface/huggingface_hub.git@main

I also tried it myself and it worked fine as well (see https://huggingface.co/Wauplin/da_core_news_sm). Could you provide the exact wheel file that you try to push to be sure we are having the same setup ?
Are you sure you are using a write access token (see https://huggingface.co/settings/tokens) ? I would say yes since your repo is created but let's be sure.
The error message seems relate to git-LFS. Could you install it and retry ? This should solve your issue.

Answer 5 · 2022-09-22T14:03:08.000Z

Hi again @Wauplin,

I'm trying again with huggingface_hub update (from main branch), that returns me a new error log:

(spacy_venv) lterriel@pl767-pro:~/Documents/dev/almanach-projects/N4A_project/Training_pipelines/spacy/NER4Archives_workflows$ spacy project run push_to_hub

================================ push_to_hub ================================
Running command: huggingface-cli login

        _|    _|  _|    _|    _|_|_|    _|_|_|  _|_|_|  _|      _|    _|_|_|      _|_|_|_|    _|_|      _|_|_|  _|_|_|_|
        _|    _|  _|    _|  _|        _|          _|    _|_|    _|  _|            _|        _|    _|  _|        _|
        _|_|_|_|  _|    _|  _|  _|_|  _|  _|_|    _|    _|  _|  _|  _|  _|_|      _|_|_|    _|_|_|_|  _|        _|_|_|
        _|    _|  _|    _|  _|    _|  _|    _|    _|    _|    _|_|  _|    _|      _|        _|    _|  _|        _|
        _|    _|    _|_|      _|_|_|    _|_|_|  _|_|_|  _|      _|    _|_|_|      _|        _|    _|    _|_|_|  _|_|_|_|

        To login, `huggingface_hub` now requires a token generated from https://huggingface.co/settings/tokens .
        
Token: 
Login successful
Your token has been saved to /home/lterriel/.huggingface/token
Running command: git lfs install
Updated Git hooks.
Git LFS initialized.
Running command: /home/lterriel/Documents/dev/almanach-projects/N4A_project/Training_pipelines/spacy/NER4Archives_workflows/spacy_venv/bin/python3.7 -m spacy huggingface-hub push ./packages/fr_core_ner4archives_v3_default-0.0.0/dist/fr_core_ner4archives_v3_default-0.0.0-py3-none-any.whl -o ner4archives -V
ℹ Publishing to repository
'ner4archives/fr_core_ner4archives_v3_default'
result /tmp/tmpq2q1x83p/fr_core_ner4archives_v3_default/meta.json fr_core_ner4archives_v3_default/meta.json
result /tmp/tmpq2q1x83p/fr_core_ner4archives_v3_default/fr_core_ner4archives_v3_default-0.0.0/README.md fr_core_ner4archives_v3_default/fr_core_ner4archives_v3_default-0.0.0/README.md
result /tmp/tmpq2q1x83p/fr_core_ner4archives_v3_default/fr_core_ner4archives_v3_default-0.0.0/config.cfg fr_core_ner4archives_v3_default/fr_core_ner4archives_v3_default-0.0.0/config.cfg
result /tmp/tmpq2q1x83p/fr_core_ner4archives_v3_default/fr_core_ner4archives_v3_default-0.0.0/meta.json fr_core_ner4archives_v3_default/fr_core_ner4archives_v3_default-0.0.0/meta.json
result /tmp/tmpq2q1x83p/fr_core_ner4archives_v3_default/fr_core_ner4archives_v3_default-0.0.0/tokenizer fr_core_ner4archives_v3_default/fr_core_ner4archives_v3_default-0.0.0/tokenizer
result /tmp/tmpq2q1x83p/fr_core_ner4archives_v3_default/fr_core_ner4archives_v3_default-0.0.0/ner/cfg fr_core_ner4archives_v3_default/fr_core_ner4archives_v3_default-0.0.0/ner/cfg
result /tmp/tmpq2q1x83p/fr_core_ner4archives_v3_default/fr_core_ner4archives_v3_default-0.0.0/ner/model fr_core_ner4archives_v3_default/fr_core_ner4archives_v3_default-0.0.0/ner/model
result /tmp/tmpq2q1x83p/fr_core_ner4archives_v3_default/fr_core_ner4archives_v3_default-0.0.0/ner/moves fr_core_ner4archives_v3_default/fr_core_ner4archives_v3_default-0.0.0/ner/moves
result /tmp/tmpq2q1x83p/fr_core_ner4archives_v3_default/fr_core_ner4archives_v3_default-0.0.0/tok2vec/cfg fr_core_ner4archives_v3_default/fr_core_ner4archives_v3_default-0.0.0/tok2vec/cfg
result /tmp/tmpq2q1x83p/fr_core_ner4archives_v3_default/fr_core_ner4archives_v3_default-0.0.0/tok2vec/model fr_core_ner4archives_v3_default/fr_core_ner4archives_v3_default-0.0.0/tok2vec/model
result /tmp/tmpq2q1x83p/fr_core_ner4archives_v3_default/fr_core_ner4archives_v3_default-0.0.0/vocab/key2row fr_core_ner4archives_v3_default/fr_core_ner4archives_v3_default-0.0.0/vocab/key2row
result /tmp/tmpq2q1x83p/fr_core_ner4archives_v3_default/fr_core_ner4archives_v3_default-0.0.0/vocab/lookups.bin fr_core_ner4archives_v3_default/fr_core_ner4archives_v3_default-0.0.0/vocab/lookups.bin
result /tmp/tmpq2q1x83p/fr_core_ner4archives_v3_default/fr_core_ner4archives_v3_default-0.0.0/vocab/strings.json fr_core_ner4archives_v3_default/fr_core_ner4archives_v3_default-0.0.0/vocab/strings.json
result /tmp/tmpq2q1x83p/fr_core_ner4archives_v3_default/fr_core_ner4archives_v3_default-0.0.0/vocab/vectors fr_core_ner4archives_v3_default/fr_core_ner4archives_v3_default-0.0.0/vocab/vectors
result /tmp/tmpq2q1x83p/fr_core_ner4archives_v3_default/fr_core_ner4archives_v3_default-0.0.0/vocab/vectors.cfg fr_core_ner4archives_v3_default/fr_core_ner4archives_v3_default-0.0.0/vocab/vectors.cfg
result /tmp/tmpq2q1x83p/fr_core_ner4archives_v3_default-0.0.0.dist-info/METADATA fr_core_ner4archives_v3_default-0.0.0.dist-info/METADATA
result /tmp/tmpq2q1x83p/fr_core_ner4archives_v3_default-0.0.0.dist-info/WHEEL fr_core_ner4archives_v3_default-0.0.0.dist-info/WHEEL
result /tmp/tmpq2q1x83p/fr_core_ner4archives_v3_default-0.0.0.dist-info/entry_points.txt fr_core_ner4archives_v3_default-0.0.0.dist-info/entry_points.txt
result /tmp/tmpq2q1x83p/fr_core_ner4archives_v3_default-0.0.0.dist-info/top_level.txt fr_core_ner4archives_v3_default-0.0.0.dist-info/top_level.txt
result /tmp/tmpq2q1x83p/fr_core_ner4archives_v3_default-0.0.0.dist-info/RECORD fr_core_ner4archives_v3_default-0.0.0.dist-info/RECORD
✔ Extracted information from .whl file
✔ Created model card
fr_core_ner4archives_v3_default (v0.0.0)
tags:
- spacy
- token-classification
language:
- fr
model-index:
- name: fr_core_ner4archives_v3_default
  results:
  - task:
      name: NER
      type: token-classification
    metrics:
    - name: NER Precision
      type: precision
      value: 0.7873493976
    - name: NER Recall
      type: recall
      value: 0.7612114153
    - name: NER F Score
      type: f_score
      value: 0.7740598164

Pushing repository to the hub...
Traceback (most recent call last):
  File "/home/lterriel/Documents/dev/almanach-projects/N4A_project/Training_pipelines/spacy/NER4Archives_workflows/spacy_venv/lib/python3.7/site-packages/huggingface_hub/utils/_errors.py", line 213, in hf_raise_for_status
    response.raise_for_status()
  File "/home/lterriel/Documents/dev/almanach-projects/N4A_project/Training_pipelines/spacy/NER4Archives_workflows/spacy_venv/lib/python3.7/site-packages/requests/models.py", line 1021, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://huggingface.co/ner4archives/fr_core_ner4archives_v3_default.git/info/lfs/objects/batch

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/lterriel/Documents/dev/almanach-projects/N4A_project/Training_pipelines/spacy/NER4Archives_workflows/spacy_venv/lib/python3.7/site-packages/spacy/__main__.py", line 4, in <module>
    setup_cli()
  File "/home/lterriel/Documents/dev/almanach-projects/N4A_project/Training_pipelines/spacy/NER4Archives_workflows/spacy_venv/lib/python3.7/site-packages/spacy/cli/_util.py", line 71, in setup_cli
    command(prog_name=COMMAND)
  File "/home/lterriel/Documents/dev/almanach-projects/N4A_project/Training_pipelines/spacy/NER4Archives_workflows/spacy_venv/lib/python3.7/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/home/lterriel/Documents/dev/almanach-projects/N4A_project/Training_pipelines/spacy/NER4Archives_workflows/spacy_venv/lib/python3.7/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/home/lterriel/Documents/dev/almanach-projects/N4A_project/Training_pipelines/spacy/NER4Archives_workflows/spacy_venv/lib/python3.7/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/lterriel/Documents/dev/almanach-projects/N4A_project/Training_pipelines/spacy/NER4Archives_workflows/spacy_venv/lib/python3.7/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/lterriel/Documents/dev/almanach-projects/N4A_project/Training_pipelines/spacy/NER4Archives_workflows/spacy_venv/lib/python3.7/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/lterriel/Documents/dev/almanach-projects/N4A_project/Training_pipelines/spacy/NER4Archives_workflows/spacy_venv/lib/python3.7/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/home/lterriel/Documents/dev/almanach-projects/N4A_project/Training_pipelines/spacy/NER4Archives_workflows/spacy_venv/lib/python3.7/site-packages/typer/main.py", line 532, in wrapper
    return callback(**use_params)  # type: ignore
  File "/home/lterriel/Documents/dev/almanach-projects/N4A_project/Training_pipelines/spacy/NER4Archives_workflows/spacy_venv/lib/python3.7/site-packages/spacy_huggingface_hub/push.py", line 53, in huggingface_hub_push_cli
    push(whl_path, organization, commit_msg, verbose=verbose)
  File "/home/lterriel/Documents/dev/almanach-projects/N4A_project/Training_pipelines/spacy/NER4Archives_workflows/spacy_venv/lib/python3.7/site-packages/spacy_huggingface_hub/push.py", line 134, in push
    commit_message=commit_msg,
  File "/home/lterriel/Documents/dev/almanach-projects/N4A_project/Training_pipelines/spacy/NER4Archives_workflows/spacy_venv/lib/python3.7/site-packages/huggingface_hub/utils/_validators.py", line 90, in _inner_fn
    return fn(*args, **kwargs)
  File "/home/lterriel/Documents/dev/almanach-projects/N4A_project/Training_pipelines/spacy/NER4Archives_workflows/spacy_venv/lib/python3.7/site-packages/huggingface_hub/hf_api.py", line 2329, in upload_folder
    parent_commit=parent_commit,
  File "/home/lterriel/Documents/dev/almanach-projects/N4A_project/Training_pipelines/spacy/NER4Archives_workflows/spacy_venv/lib/python3.7/site-packages/huggingface_hub/utils/_validators.py", line 90, in _inner_fn
    return fn(*args, **kwargs)
  File "/home/lterriel/Documents/dev/almanach-projects/N4A_project/Training_pipelines/spacy/NER4Archives_workflows/spacy_venv/lib/python3.7/site-packages/huggingface_hub/hf_api.py", line 1999, in create_commit
    num_threads=num_threads,
  File "/home/lterriel/Documents/dev/almanach-projects/N4A_project/Training_pipelines/spacy/NER4Archives_workflows/spacy_venv/lib/python3.7/site-packages/huggingface_hub/utils/_validators.py", line 90, in _inner_fn
    return fn(*args, **kwargs)
  File "/home/lterriel/Documents/dev/almanach-projects/N4A_project/Training_pipelines/spacy/NER4Archives_workflows/spacy_venv/lib/python3.7/site-packages/huggingface_hub/_commit_api.py", line 207, in upload_lfs_files
    endpoint=endpoint,
  File "/home/lterriel/Documents/dev/almanach-projects/N4A_project/Training_pipelines/spacy/NER4Archives_workflows/spacy_venv/lib/python3.7/site-packages/huggingface_hub/utils/_validators.py", line 90, in _inner_fn
    return fn(*args, **kwargs)
  File "/home/lterriel/Documents/dev/almanach-projects/N4A_project/Training_pipelines/spacy/NER4Archives_workflows/spacy_venv/lib/python3.7/site-packages/huggingface_hub/lfs.py", line 192, in post_lfs_batch_info
    hf_raise_for_status(resp)
  File "/home/lterriel/Documents/dev/almanach-projects/N4A_project/Training_pipelines/spacy/NER4Archives_workflows/spacy_venv/lib/python3.7/site-packages/huggingface_hub/utils/_errors.py", line 242, in hf_raise_for_status
    raise RepositoryNotFoundError(message, response) from e
huggingface_hub.utils._errors.RepositoryNotFoundError: 401 Client Error. (Request ID: dlfGCn2Kg6NiTrjutoM3V)

Repository Not Found for url: https://huggingface.co/ner4archives/fr_core_ner4archives_v3_default.git/info/lfs/objects/batch.
Please make sure you specified the correct `repo_id` and `repo_type`.
If the repo is private, make sure you are authenticated.

It's strange because the organisation repo is not private.

Of course, .whl is here: fr_core_ner4archives_v3_default-0.0.0-py3-none-any.zip
I check again in HF settings, I have "writer" access token
git-LFS is installed, and I run git lfs install before push (Cf. error log in 1.)

Answer 6 · 2022-09-22T15:19:39.000Z

Thanks @Lucaterre what testing that and providing the traceback.
I would need more time to investigate it but I'll get back to you as soon as I have something new.

Answer 7 · 2022-09-22T15:46:19.000Z

The problem here is probably the remote git URL:

-https://huggingface.co/ner4archives/fr_core_ner4archives_v3_default.git/info/lfs/objects/batch
+https://huggingface.co/ner4archives/fr_core_ner4archives_V3_default.git/info/lfs/objects/batch

The URLs are case sensitive, and your repo is https://huggingface.co/ner4archives/fr_core_ner4archives_V3_default, so your command should reflect that and be https://huggingface.co/ner4archives/fr_core_ner4archives_V3_default.git/info/lfs/objects/batch

Can you try updating it by running git remote set-url origin https://huggingface.co/ner4archives/fr_core_ner4archives_V3_default, and try again?

cc @coyotte508 @julien-c for case insensitivity, linking to internal discussions here and here

Answer 8 · 2022-09-23T08:09:26.000Z

is GitHub case insensitive on repo names (including via git commands) BTW?

Answer 9 · 2022-09-23T12:22:49.000Z

@julien-c I did a small test with https://github.com/Wauplin/huggingface_hub and it seems case insensitive:

➜  git clone https://github.com/Wauplin/huggingface_hub.git
Cloning into 'huggingface_hub'...
remote: Enumerating objects: 6036, done.
remote: Total 6036 (delta 0), reused 0 (delta 0), pack-reused 6036
Receiving objects: 100% (6036/6036), 11.89 MiB | 3.68 MiB/s, done.
Resolving deltas: 100% (3631/3631), done.
➜  rm -rf huggingface_hub

➜  git clone https://github.com/wauplin/huggingface_hub.git
Cloning into 'huggingface_hub'...
remote: Enumerating objects: 6036, done.
remote: Total 6036 (delta 0), reused 0 (delta 0), pack-reused 6036
Receiving objects: 100% (6036/6036), 11.89 MiB | 2.83 MiB/s, done.
Resolving deltas: 100% (3631/3631), done.
➜  rm -rf huggingface_hub
                                 
➜  git clone https://github.com/wAUplin/huGGINGface_Hub.git 
Cloning into 'huGGINGface_Hub'...
remote: Enumerating objects: 6036, done.
remote: Total 6036 (delta 0), reused 0 (delta 0), pack-reused 6036
Receiving objects: 100% (6036/6036), 11.89 MiB | 3.04 MiB/s, done.
Resolving deltas: 100% (3631/3631), done.
➜  rm -rf huggingface_hub

Answer 10 · 2022-09-26T11:15:55.000Z

Thank you for your feedback and your solutions.

@Pierrci : I changed the model name to push in lowercase and retry the process to see if it was a case related issue but I raise the same error Cf. #9 (comment). (with this config: spacy-huggingface-hub==0.0.7 and huggingface-hub last version from main branch)
I think this is clearly an issue with git lfs on my side. I succeeded to push my model onto Hugging Face Hub with this temporary process (with this config: spacy-huggingface-hub==0.0.7 and huggingface-hub==0.8.1):

I launch python -m spacy huggingface-hub push ./packages/fr_core_ner4archives_v3_default-0.0.0/dist/fr_core_ner4archives_v3_default-0.0.0-py3-none-any.whl -o ner4archives -V then that raises an error 400 and create an empty repo on Hugging Face hub;
In newly created Hugging Face hub model repo, I add manually this new lines to .gitattibutes file:

*.whl filter=lfs diff=lfs merge=lfs -text
tok2vec/model filter=lfs diff=lfs merge=lfs -text
tokenizer filter=lfs diff=lfs merge=lfs -text

then I commit changes;

Finally, I run again python -m spacy huggingface-hub push ./packages/fr_core_ner4archives_v3_default-0.0.0/dist/fr_core_ner4archives_v3_default-0.0.0-py3-none-any.whl -o ner4archives -V and that's work, the model and associated files (meta etc.) are updated and available on repo: https://huggingface.co/ner4archives/fr_core_ner4archives_v3_default

I cannot understand yet why git lfs and spacy-huggingface-hub cannot initialise and track large spaCy files automatically when I push (git lfs is installed correctly), It must be a bad manipulation with git lfs on my side...

Answer 11 · 2022-09-27T06:55:33.000Z

@Lucaterre sorry for the git lfs issue you are having but glad that you find a workaround with the .gitattributes file. Please let me know if we you think we can do something for you here.

Otherwise we can close the issue. Don't hesitate to post a new comment if you find what went wrong with git lfs, that might be useful for future users.

Answer 12 · 2022-09-27T07:23:16.000Z

@Wauplin, I agree, we can close this topic for now; I keep this if I have something new 👍

Answer 13 · 2022-11-15T11:57:22.000Z

Hi, I am having exactly the same problem with my models. Any progress on making git lfs and huggingface-hub play together to upload spacy models?

Answer 14 · 2022-11-15T12:07:59.000Z

@osanseviero @Wauplin we could open a PR to update the code in this repo to use newer huggingface_hub functions (and maybe unpin the dependency) WDYT? (also cc Explosion 💥 team ofc. Hi! 👋 )

Answer 15 · 2022-11-15T13:52:54.000Z

Will have a look at it. What seems weird to me is that for what I see , we already use the HTTP endpoint to upload the files. Which means it shouldn't be related on git lfs being installed or not.
I would need to make some tests with the newest version of hfh and try to understand what's happening.

@cayorodriguez Just to be sure, you are getting a BadRequest error as well, right ?

Answer 16 · 2022-11-15T15:27:34.000Z

Hi! Let me re-open this for now so it's more visible in case others might still be running into the same issue. We'd prefer to keep an upper pin on the dependency, but updates are obviously very welcome.

If you have any recommendations for how to add to some basic tests to this repo, I'd be all ears?

Answer 17 · 2022-11-15T16:27:20.000Z

Hi @adrianeboyd , good idea to re-open.

First thing I think would be to reproduce the bug. I haven't being able yet which makes it hard to test.
I'll make some more investigation tomorrow but if anyone can help, please chime in :)

Answer 18 · 2022-11-16T07:56:33.000Z

Hi, The error is: requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: https://huggingface.co/api/models/crodri/es_pharmaconer_ner_trf/commit/main (Request ID: zOGDnLidpl-ROlzG9ABkl) Full (rellevant) trace is: python -m spacy huggingface-hub push es_pharmaconer_ner_trf-3.4.0-py3-none-any.whl ℹ Publishing to repository 'crodri/es_pharmaconer_ner_trf' result /tmp/tmpwwbvh_05/es_pharmaconer_ner_trf/meta.json es_pharmaconer_ner_trf/meta.json result /tmp/tmpwwbvh_05/es_pharmaconer_ner_trf/es_pharmaconer_ner_trf-3.4.0/README.md es_pharmaconer_ner_trf/es_pharmaconer_ner_trf-3.4.0/README.md result /tmp/tmpwwbvh_05/es_pharmaconer_ner_trf/es_pharmaconer_ner_trf-3.4.0/config.cfg es_pharmaconer_ner_trf/es_pharmaconer_ner_trf-3.4.0/config.cfg result /tmp/tmpwwbvh_05/es_pharmaconer_ner_trf/es_pharmaconer_ner_trf-3.4.0/meta.json es_pharmaconer_ner_trf/es_pharmaconer_ner_trf-3.4.0/meta.json result /tmp/tmpwwbvh_05/es_pharmaconer_ner_trf/es_pharmaconer_ner_trf-3.4.0/tokenizer es_pharmaconer_ner_trf/es_pharmaconer_ner_trf-3.4.0/tokenizer result /tmp/tmpwwbvh_05/es_pharmaconer_ner_trf/es_pharmaconer_ner_trf-3.4.0/ner/cfg es_pharmaconer_ner_trf/es_pharmaconer_ner_trf-3.4.0/ner/cfg result /tmp/tmpwwbvh_05/es_pharmaconer_ner_trf/es_pharmaconer_ner_trf-3.4.0/ner/model es_pharmaconer_ner_trf/es_pharmaconer_ner_trf-3.4.0/ner/model result /tmp/tmpwwbvh_05/es_pharmaconer_ner_trf/es_pharmaconer_ner_trf-3.4.0/ner/moves es_pharmaconer_ner_trf/es_pharmaconer_ner_trf-3.4.0/ner/moves result /tmp/tmpwwbvh_05/es_pharmaconer_ner_trf/es_pharmaconer_ner_trf-3.4.0/transformer/cfg es_pharmaconer_ner_trf/es_pharmaconer_ner_trf-3.4.0/transformer/cfg result /tmp/tmpwwbvh_05/es_pharmaconer_ner_trf/es_pharmaconer_ner_trf-3.4.0/transformer/model es_pharmaconer_ner_trf/es_pharmaconer_ner_trf-3.4.0/transformer/model result /tmp/tmpwwbvh_05/es_pharmaconer_ner_trf/es_pharmaconer_ner_trf-3.4.0/vocab/key2row es_pharmaconer_ner_trf/es_pharmaconer_ner_trf-3.4.0/vocab/key2row result /tmp/tmpwwbvh_05/es_pharmaconer_ner_trf/es_pharmaconer_ner_trf-3.4.0/vocab/lookups.bin es_pharmaconer_ner_trf/es_pharmaconer_ner_trf-3.4.0/vocab/lookups.bin result /tmp/tmpwwbvh_05/es_pharmaconer_ner_trf/es_pharmaconer_ner_trf-3.4.0/vocab/strings.json es_pharmaconer_ner_trf/es_pharmaconer_ner_trf-3.4.0/vocab/strings.json result /tmp/tmpwwbvh_05/es_pharmaconer_ner_trf/es_pharmaconer_ner_trf-3.4.0/vocab/vectors es_pharmaconer_ner_trf/es_pharmaconer_ner_trf-3.4.0/vocab/vectors result /tmp/tmpwwbvh_05/es_pharmaconer_ner_trf/es_pharmaconer_ner_trf-3.4.0/vocab/vectors.cfg es_pharmaconer_ner_trf/es_pharmaconer_ner_trf-3.4.0/vocab/vectors.cfg result /tmp/tmpwwbvh_05/es_pharmaconer_ner_trf-3.4.0.dist-info/METADATA es_pharmaconer_ner_trf-3.4.0.dist-info/METADATA result /tmp/tmpwwbvh_05/es_pharmaconer_ner_trf-3.4.0.dist-info/WHEEL es_pharmaconer_ner_trf-3.4.0.dist-info/WHEEL result /tmp/tmpwwbvh_05/es_pharmaconer_ner_trf-3.4.0.dist-info/entry_points.txt es_pharmaconer_ner_trf-3.4.0.dist-info/entry_points.txt result /tmp/tmpwwbvh_05/es_pharmaconer_ner_trf-3.4.0.dist-info/top_level.txt es_pharmaconer_ner_trf-3.4.0.dist-info/top_level.txt result /tmp/tmpwwbvh_05/es_pharmaconer_ner_trf-3.4.0.dist-info/RECORD es_pharmaconer_ner_trf-3.4.0.dist-info/RECORD ✔ Extracted information from .whl file ✔ Created model card es_pharmaconer_ner_trf (v3.4.0) Pushing repository to the hub... Traceback (most recent call last): File "/home/crodrig1/anaconda3/envs/spacy/lib/python3.8/runpy.py", line 194, in _run_module_as_main return _run_code(code, main_globals, None, File "/home/crodrig1/anaconda3/envs/spacy/lib/python3.8/runpy.py", line 87, in _run_code exec(code, run_globals) File "/home/crodrig1/anaconda3/envs/spacy/lib/python3.8/site-packages/spacy/__main__.py", line 4, in <module> setup_cli() File "/home/crodrig1/anaconda3/envs/spacy/lib/python3.8/site-packages/spacy/cli/_util.py", line 71, in setup_cli command(prog_name=COMMAND) File "/home/crodrig1/anaconda3/envs/spacy/lib/python3.8/site-packages/click/core.py", line 1130, in __call__ return self.main(*args, **kwargs) File "/home/crodrig1/anaconda3/envs/spacy/lib/python3.8/site-packages/click/core.py", line 1055, in main rv = self.invoke(ctx) File "/home/crodrig1/anaconda3/envs/spacy/lib/python3.8/site-packages/click/core.py", line 1657, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/home/crodrig1/anaconda3/envs/spacy/lib/python3.8/site-packages/click/core.py", line 1657, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/home/crodrig1/anaconda3/envs/spacy/lib/python3.8/site-packages/click/core.py", line 1404, in invoke return ctx.invoke(self.callback, **ctx.params) File "/home/crodrig1/anaconda3/envs/spacy/lib/python3.8/site-packages/click/core.py", line 760, in invoke return __callback(*args, **kwargs) File "/home/crodrig1/anaconda3/envs/spacy/lib/python3.8/site-packages/typer/main.py", line 532, in wrapper return callback(**use_params) # type: ignore File "/home/crodrig1/anaconda3/envs/spacy/lib/python3.8/site-packages/spacy_huggingface_hub/push.py", line 53, in huggingface_hub_push_cli push(whl_path, organization, commit_msg, verbose=verbose) File "/home/crodrig1/anaconda3/envs/spacy/lib/python3.8/site-packages/spacy_huggingface_hub/push.py", line 130, in push url = upload_folder( File "/home/crodrig1/anaconda3/envs/spacy/lib/python3.8/site-packages/huggingface_hub/hf_api.py", line 2109, in upload_folder pr_url = self.create_commit( File "/home/crodrig1/anaconda3/envs/spacy/lib/python3.8/site-packages/huggingface_hub/hf_api.py", line 1844, in create_commit _raise_for_status(commit_resp) File "/home/crodrig1/anaconda3/envs/spacy/lib/python3.8/site-packages/huggingface_hub/utils/_errors.py", line 84, in _raise_for_status _raise_with_request_id(request) File "/home/crodrig1/anaconda3/envs/spacy/lib/python3.8/site-packages/huggingface_hub/utils/_errors.py", line 95, in _raise_with_request_id raise e File "/home/crodrig1/anaconda3/envs/spacy/lib/python3.8/site-packages/huggingface_hub/utils/_errors.py", line 90, in _raise_with_request_id request.raise_for_status() File "/home/crodrig1/anaconda3/envs/spacy/lib/python3.8/site-packages/requests/models.py", line 1021, in raise_for_status raise HTTPError(http_error_msg, response=self) requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: https://huggingface.co/api/models/crodri/es_pharmaconer_ner_trf/commit/main (Request ID: zOGDnLidpl-ROlzG9ABkl) ============== Carlos Rodriguez ==============

…

On Tue, Nov 15, 2022 at 2:53 PM Lucain ***@***.***> wrote: Will have a look at it. What seems weird to me is that for what I see <https://github.com/explosion/spacy-huggingface-hub/blob/af104c887b83ffa1f698af68559cc276e56d81aa/spacy_huggingface_hub/push.py#LL130C14-L135C10>, we already use the HTTP endpoint to upload the files. Which means it shouldn't be related on git lfs being installed or not. I would need to make some tests with the newest version of hfh and try to understand what's happening. @cayorodriguez <https://github.com/cayorodriguez> Just to be sure, you are getting a BadRequest error as well, right ? — Reply to this email directly, view it on GitHub <#9 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA2FKPVY4DFQVWOUO24GDP3WIOIUBANCNFSM6AAAAAAQSE7NXY> . You are receiving this because you were mentioned.Message ID: ***@***.***>

Answer 19 · 2022-11-16T10:31:13.000Z

Thanks @cayorodriguez for the update. Sorry again but I don't see from where the bug can come from and how to reproduce it.

A few things we can try:

Are you able to re-run this google colab ? First try as it is (with hardocded .whl) and then with your .whl example ? If same error happens to you again, can you share the corresponding .whl file ?
Which version of hugginface_hub are you using. In the google colab I use huggingface_hub==0.8.1 to fit with this issue description but it would be worth trying with pip install huggingface_hub==0.11.0.rc1 (latest pre-release).
If you can't reproduce the issue in a google colab, could you try to update to latest version (pip install huggingface_hub==0.11.0.rc1) locally and see what issue you are getting. If you still get an error, could you run huggingface-cli env which will dump some machine information that might help us.

Thank you in advance for your help and sorry for the inconvenience. I really hope we can get this fixed :)

Answer 20 · 2022-11-16T15:12:39.000Z

It worked with your wheel in google colab but not with mine. We ended up uploading it as any HF model and creating the card manually. You can find our wheel here: https://huggingface.co/PlanTL-GOB-ES/es_cantemist_ner_trf It is strange, because some other colleagues ath the Barcelona Supercomputing Center have tried to follow the instructions, and they get the same error message: requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: https://huggingface.co/api/models/crodri/es_cantemist_ner_trf/commit/main Thanks for the help ============== Carlos Rodriguez ==============

…

On Wed, Nov 16, 2022 at 11:31 AM Lucain ***@***.***> wrote: Thanks @cayorodriguez <https://github.com/cayorodriguez> for the update. Sorry again but I don't see from where the bug can come from and how to reproduce it. A few things we can try: 1. Are you able to re-run this google colab <https://colab.research.google.com/drive/1hRtplEFM-BnfYuYOCOxm5JigOKwOdcH3?usp=sharing> ? First try as it is (with hardocded .whl) and then with your .whl example ? If same error happens to you again, can you share the corresponding .whl file ? 2. Which version of hugginface_hub are you using. In the google colab I use huggingface_hub==0.8.1 to fit with this issue description but it would be worth trying with pip install huggingface_hub==0.11.0.rc1 (latest pre-release). 3. If you can't reproduce the issue in a google colab, could you try to update to latest version (pip install huggingface_hub==0.11.0.rc1) locally and see what issue you are getting. If you still get an error, could you run huggingface-cli env which will dump some machine information that might help us. Thank you in advance for your help and sorry for the inconvenience. I really hope we can get this fixed :) — Reply to this email directly, view it on GitHub <#9 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA2FKPXMTV4GUGS2FWX33MTWISZXZANCNFSM6AAAAAAQSE7NXY> . You are receiving this because you were mentioned.Message ID: ***@***.***>

Answer 21 · 2022-11-16T15:54:09.000Z

Ok, issue solved: When using the latest release you suggested (0.11.0.rc1), I was provided with more information about the problem. After the 400 Error trace, I was warned about a licence name issue: huggingface_hub.utils._errors.BadRequestError: (Request ID: 9mrP_4f60HpYeIdfNJiD1) Bad request for commit endpoint: "license" must be one of [openrail, bigscience-openrail-m, creativeml-openrail-m, bigscience-bloom-rail-1.0, afl-3.0, apache-2.0, artistic-2.0, bsl-1.0, bsd, bsd-2-clause, bsd-3-clause, bsd-3-clause-clear, c-uda, cc, cc0-1.0, cc-by-2.0, cc-by-2.5, cc-by-3.0, cc-by-4.0, cc-by-sa-3.0, cc-by-sa-4.0, cc-by-nc-2.0, cc-by-nc-3.0, cc-by-nc-4.0, cc-by-nd-4.0, cc-by-nc-nd-3.0, cc-by-nc-nd-4.0, cc-by-nc-sa-2.0, cc-by-nc-sa-3.0, cc-by-nc-sa-4.0, wtfpl, ecl-2.0, epl-1.0, epl-2.0, eupl-1.1, agpl-3.0, gfdl, gpl, gpl-2.0, gpl-3.0, lgpl, lgpl-2.1, lgpl-3.0, isc, lppl-1.3c, ms-pl, mit, mpl-2.0, odc-by, odbl, openrail++, osl-3.0, postgresql, ofl-1.1, ncsa, unlicense, zlib, pddl, lgpl-lr, unknown, other, array] Once I changed the license to one of the allowed ones, the upload was completed. We did not get that warning before when using a previous release, even with the -V flag. Thanks everyone ============== Carlos Rodriguez ============== On Wed, Nov 16, 2022 at 4:12 PM Carlos Rodriguez ***@***.***> wrote:

…

It worked with your wheel in google colab but not with mine. We ended up uploading it as any HF model and creating the card manually. You can find our wheel here: https://huggingface.co/PlanTL-GOB-ES/es_cantemist_ner_trf It is strange, because some other colleagues ath the Barcelona Supercomputing Center have tried to follow the instructions, and they get the same error message: requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: https://huggingface.co/api/models/crodri/es_cantemist_ner_trf/commit/main Thanks for the help ============== Carlos Rodriguez ============== On Wed, Nov 16, 2022 at 11:31 AM Lucain ***@***.***> wrote: > Thanks @cayorodriguez <https://github.com/cayorodriguez> for the update. > Sorry again but I don't see from where the bug can come from and how to > reproduce it. > > A few things we can try: > > 1. Are you able to re-run this google colab > <https://colab.research.google.com/drive/1hRtplEFM-BnfYuYOCOxm5JigOKwOdcH3?usp=sharing> > ? First try as it is (with hardocded .whl) and then with your .whl example > ? If same error happens to you again, can you share the corresponding .whl > file ? > 2. Which version of hugginface_hub are you using. In the google colab > I use huggingface_hub==0.8.1 to fit with this issue description but > it would be worth trying with pip install huggingface_hub==0.11.0.rc1 > (latest pre-release). > 3. If you can't reproduce the issue in a google colab, could you try > to update to latest version (pip install huggingface_hub==0.11.0.rc1) > locally and see what issue you are getting. If you still get an error, > could you run huggingface-cli env which will dump some machine > information that might help us. > > Thank you in advance for your help and sorry for the inconvenience. I > really hope we can get this fixed :) > > — > Reply to this email directly, view it on GitHub > <#9 (comment)>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/AA2FKPXMTV4GUGS2FWX33MTWISZXZANCNFSM6AAAAAAQSE7NXY> > . > You are receiving this because you were mentioned.Message ID: > ***@***.***> >

Answer 22 · 2022-11-16T16:19:24.000Z

Good news ! Thank you for investigating it more. huggingface_hub==0.11 has indeed a better error handling which is especially useful for this case :)

And other good news is that the release is now official so hopefully users will now see the license error as well. I think we can now close this issue.

EDIT: check out the release notes

Answer 23 · 2022-11-16T17:48:17.000Z

ah yes we started validating license ids on the server side a few months ago... glad your issue is fixed! 🔥

Answer 24 · 2022-11-19T02:00:44.000Z

Thanks for the debugging and sorry for the issue, I'm glad the issue was found 🔥 If there is any new license not covered with our validation let us know!