sentinel-hub/eo-learn

[BUG] Weird path concatenation on macOS

mlubej opened this issue · 2 comments

Describe the bug

I'm trying to run ImportFromTiffTask by specifying the folder parameter when the task is initialized. The path of the filesystem gets duplicated or something. Like Users/mlubej/work/projects/sh-project/Users/mlubej/work/projects/sh-project/eo-learn/example_data/import-tiff-test1.tiff

To Reproduce

mask_feature = (FeatureType.MASK_TIMELESS, "TEST_TIF")
path = '/Users/mlubej/work/projects/sh-project/eo-learn/example_data/'

task = ImportFromTiffTask(feature=mask_feature, folder=path)
task.execute(None, filename='import-tiff-test1.tiff')

Stack trace or screenshots

---------------------------------------------------------------------------
FileNotFoundError                         Traceback (most recent call last)
File ~/.pyenv/versions/3.8.7/envs/sh/lib/python3.8/site-packages/fs/osfs.py:357, in OSFS.openbin(self, path, mode, buffering, **options)
    356         sys_path = os.open(sys_path, os.O_RDWR | os.O_CREAT | os.O_EXCL)
--> 357     binary_file = io.open(
    358         sys_path, mode=_mode.to_platform_bin(), buffering=buffering, **options
    359     )
    360 return binary_file

FileNotFoundError: [Errno 2] No such file or directory: b'/Users/mlubej/work/projects/sh-project/Users/mlubej/work/projects/sh-project/eo-learn/example_data/import-tiff-test1.tiff'

During handling of the above exception, another exception occurred:

ResourceNotFound                          Traceback (most recent call last)
Input In [6], in <module>
      1 mask_feature = (FeatureType.MASK_TIMELESS, "TEST_TIF")
      3 task = ImportFromTiffTask(feature=mask_feature, 
      4                           folder='/Users/mlubej/work/projects/sh-project/eo-learn/example_data/', 
      5                           no_data_value=0)
----> 6 task.execute(None, filename='import-tiff-test1.tiff')

File ~/work/projects/sh-project/eo-learn/io/eolearn/io/local_io.py:442, in ImportFromTiffTask.execute(self, eopatch, filename)
    440 data = []
    441 for path in filename_paths:
--> 442     with filesystem.openbin(path, "r") as file_handle:
    443         with rasterio.open(file_handle) as src:
    445             if eopatch.bbox is None:

File ~/.pyenv/versions/3.8.7/envs/sh/lib/python3.8/site-packages/fs/osfs.py:357, in OSFS.openbin(self, path, mode, buffering, **options)
    355     if six.PY2 and _mode.exclusive:
    356         sys_path = os.open(sys_path, os.O_RDWR | os.O_CREAT | os.O_EXCL)
--> 357     binary_file = io.open(
    358         sys_path, mode=_mode.to_platform_bin(), buffering=buffering, **options
    359     )
    360 return binary_file

File ~/.pyenv/versions/3.8.7/envs/sh/lib/python3.8/site-packages/fs/error_tools.py:89, in _ConvertOSErrors.__exit__(self, exc_type, exc_value, traceback)
     87     if getattr(exc_value, "args", None) == 32:  # pragma: no cover
     88         fserror = errors.ResourceLocked
---> 89 reraise(fserror, fserror(self._path, exc=exc_value), traceback)

File ~/.pyenv/versions/3.8.7/envs/sh/lib/python3.8/site-packages/six.py:718, in reraise(tp, value, tb)
    716         value = tp()
    717     if value.__traceback__ is not tb:
--> 718         raise value.with_traceback(tb)
    719     raise value
    720 finally:

File ~/.pyenv/versions/3.8.7/envs/sh/lib/python3.8/site-packages/fs/osfs.py:357, in OSFS.openbin(self, path, mode, buffering, **options)
    355     if six.PY2 and _mode.exclusive:
    356         sys_path = os.open(sys_path, os.O_RDWR | os.O_CREAT | os.O_EXCL)
--> 357     binary_file = io.open(
    358         sys_path, mode=_mode.to_platform_bin(), buffering=buffering, **options
    359     )
    360 return binary_file

ResourceNotFound: resource 'Users/mlubej/work/projects/sh-project/Users/mlubej/work/projects/sh-project/eo-learn/example_data/import-tiff-test1.tiff' not found

Desktop (please complete the following information):

  • Mac OS Monterey

Something weird happens in https://github.com/sentinel-hub/eo-learn/blob/feat/update-import-from-tiff/core/eolearn/core/utils/fs.py#L62

os.path.join(*path_parts)
'Users[/mlubej/work/projects/sh-project/eo-learn/io/eolearn/tests/../../../example_data/import-tiff-test1.tiff]()'
os.path.abspath(os.path.join(*path_parts))
'[/Users/mlubej/work/projects/sh-project/eo-learn/Users/mlubej/work/projects/sh-project/eo-learn/example_data/import-tiff-test1.tiff]()'

Some of the issues above might be due to my own fault and changes, perhaps they are even not existing, since they happened due to my own change.

However, there is an issue with an extra forward slash in an s3 path, I've traced the issue back to what was already discovered with @zigaLuksic and I forgot about it. I pushed a fix in d0b962b#diff-f43e8f8beb214d89bb32e94c32f4ddefeb303c97acbbcfd1c4148b31947474c0R52

Closing this issue since it's not related.