iterative/dvc

`dvc.api.open`: broken with `no_scm`

dberenbaum opened this issue · 2 comments

Bug Report

Description

dvc.api.open() doesn't work when core.no_scm = true.

Reproduce

Script:

DIR=$(mktemp -d)
cd $DIR
dvc init -q --no-scm
echo foo > foo
dvc add foo
rm foo
python -c "import dvc.api
with dvc.api.open('.', 'foo') as f:
    print(f.read())"

Output:

100% Adding...|█████████████████████████████████████████████████|1/1 [00:00, 46.62file/s]
Traceback (most recent call last):
  File "/Users/dave/micromamba/envs/dvc/lib/python3.11/site-packages/scmrepo/git/backend/dulwich/__init__.py", line 260, in clone
    repo = clone_from()
           ^^^^^^^^^^^^
  File "/Users/dave/micromamba/envs/dvc/lib/python3.11/site-packages/dulwich/porcelain.py", line 546, in clone
    return client.clone(
           ^^^^^^^^^^^^^
  File "/Users/dave/micromamba/envs/dvc/lib/python3.11/site-packages/dulwich/client.py", line 753, in clone
    result = self.fetch(path, target, progress=progress, depth=depth)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dave/micromamba/envs/dvc/lib/python3.11/site-packages/dulwich/client.py", line 1510, in fetch
    with self._open_repo(path) as r:
         ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dave/micromamba/envs/dvc/lib/python3.11/site-packages/dulwich/client.py", line 1432, in _open_repo
    return closing(Repo(path))
                   ^^^^^^^^^^
  File "/Users/dave/micromamba/envs/dvc/lib/python3.11/site-packages/dulwich/repo.py", line 1155, in __init__
    raise NotGitRepository(
dulwich.errors.NotGitRepository: No git repository was found at foo

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/dave/Code/dvc/dvc/scm.py", line 152, in clone
    git = Git.clone(url, to_path, progress=pbar.update_git, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dave/micromamba/envs/dvc/lib/python3.11/site-packages/scmrepo/git/__init__.py", line 154, in clone
    backend.clone(url, to_path, bare=bare, mirror=mirror, **kwargs)
  File "/Users/dave/micromamba/envs/dvc/lib/python3.11/site-packages/scmrepo/git/backend/dulwich/__init__.py", line 268, in clone
    raise CloneError(url, to_path) from exc
scmrepo.exceptions.CloneError: Failed to clone repo 'foo' to '/var/folders/24/99_tf1xj3vx8k1k_jkdmnhq00000gn/T/tmpnorzctwbdvc-clone'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<string>", line 2, in <module>
  File "/Users/dave/micromamba/envs/dvc/lib/python3.11/contextlib.py", line 137, in __enter__
    return next(self.gen)
           ^^^^^^^^^^^^^^
  File "/Users/dave/Code/dvc/dvc/api/data.py", line 240, in _open
    with Repo.open(repo, rev=rev, **repo_kwargs) as _repo:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dave/Code/dvc/dvc/repo/__init__.py", line 295, in open
    return open_repo(url, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dave/Code/dvc/dvc/repo/open_repo.py", line 60, in open_repo
    return _external_repo(url, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dave/micromamba/envs/dvc/lib/python3.11/contextlib.py", line 81, in inner
    return func(*args, **kwds)
           ^^^^^^^^^^^^^^^^^^^
  File "/Users/dave/Code/dvc/dvc/repo/open_repo.py", line 23, in _external_repo
    path = _cached_clone(url, rev)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dave/Code/dvc/dvc/repo/open_repo.py", line 134, in _cached_clone
    clone_path, shallow = _clone_default_branch(url, rev)
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dave/micromamba/envs/dvc/lib/python3.11/site-packages/funcy/decorators.py", line 47, in wrapper
    return deco(call, *dargs, **dkwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dave/micromamba/envs/dvc/lib/python3.11/site-packages/funcy/flow.py", line 246, in wrap_with
    return call()
           ^^^^^^
  File "/Users/dave/micromamba/envs/dvc/lib/python3.11/site-packages/funcy/decorators.py", line 68, in __call__
    return self._func(*self._args, **self._kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dave/Code/dvc/dvc/repo/open_repo.py", line 198, in _clone_default_branch
    git = clone(url, clone_path)
          ^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dave/Code/dvc/dvc/scm.py", line 157, in clone
    raise CloneError("SCM error") from exc
dvc.scm.CloneError: SCM error

Expected

Script should print foo.

Environment information

Output of dvc doctor:

$ dvc doctor
DVC version: 3.47.1.dev2+g3309e1562
-----------------------------------
Platform: Python 3.11.7 on macOS-14.3.1-arm64-arm-64bit
Subprojects:
        dvc_data = 3.13.0
        dvc_objects = 5.0.0
        dvc_render = 1.0.1
        dvc_task = 0.3.0
        scmrepo = 3.1.0
Supports:
        azure (adlfs = 2023.12.0, knack = 0.11.0, azure-identity = 1.15.0),
        gdrive (pydrive2 = 1.19.0),
        gs (gcsfs = 2024.2.0),
        hdfs (fsspec = 2024.2.0, pyarrow = 14.0.2),
        http (aiohttp = 3.9.1, aiohttp-retry = 2.8.3),
        https (aiohttp = 3.9.1, aiohttp-retry = 2.8.3),
        oss (ossfs = 2023.12.0),
        s3 (s3fs = 2024.2.0, boto3 = 1.33.13),
        ssh (sshfs = 2023.10.0),
        webdav (webdav4 = 0.9.8),
        webdavs (webdav4 = 0.9.8),
        webhdfs (fsspec = 2024.2.0)
Config:
        Global: /Users/dave/Library/Application Support/dvc
        System: /Library/Application Support/dvc

@dberenbaum, does this work?

with dvc.api.open('foo', '.') as f:
    print(f.read())

Oh wow 🤦 , sorry for the noise!