Race condition when importing the same description in parallel
Closed this issue · 0 comments
stephane-caron commented
Bug description
Actually, even if it the description is already in cache, we will run into an error similar to #114. Adapted the parallel
command from that issue to run on my machine:
parallel -i python -c 'from robot_descriptions import fr3_mj_description; print({}, fr3_mj_description.MJCF_PATH)' -- 1 2 3 4
Error: Another git process seems to be running in this repository, ...
3 ~/.cache/robot_descriptions/mujoco_menagerie/franka_fr3/fr3.xml
Traceback (most recent call last):
File "~/.micromamba/envs/robot_descriptions/lib/python3.12/site-packages/robot_descriptions/_cache.py", line 90, in clone_to_directory
clone.git.checkout(commit)
File "~/.micromamba/envs/robot_descriptions/lib/python3.12/site-packages/git/cmd.py", line 800, in <lambda>
return lambda *args, **kwargs: self._call_process(name, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "~/.micromamba/envs/robot_descriptions/lib/python3.12/site-packages/git/cmd.py", line 1386, in _call_process
return self.execute(call, **exec_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "~/.micromamba/envs/robot_descriptions/lib/python3.12/site-packages/git/cmd.py", line 1183, in execute
raise GitCommandError(redacted_command, status, stderr_value, stdout_value)
git.exc.GitCommandError: Cmd('git') failed due to: exit code(128)
cmdline: git checkout 3d593a327ff8d5efd10f4926ccc477820e9252c1
stderr: 'fatal: Unable to create '~/.cache/robot_descriptions/mujoco_menagerie/.git/index.lock': File exists.
Another git process seems to be running in this repository, e.g.
an editor opened by 'git commit'. Please make sure all processes
are terminated then try again. If it still fails, a git process
may have crashed in this repository earlier:
remove the file manually to continue.'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "~/.micromamba/envs/robot_descriptions/lib/python3.12/site-packages/robot_descriptions/fr3_mj_description.py", line 14, in <module>
REPOSITORY_PATH: str = _clone_to_cache(
^^^^^^^^^^^^^^^^
File "~/.micromamba/envs/robot_descriptions/lib/python3.12/site-packages/robot_descriptions/_cache.py", line 139, in clone_to_cache
clone = clone_to_directory(
^^^^^^^^^^^^^^^^^^^
File "~/.micromamba/envs/robot_descriptions/lib/python3.12/site-packages/robot_descriptions/_cache.py", line 98, in clone_to_directory
clone.git.checkout(commit)
File "~/.micromamba/envs/robot_descriptions/lib/python3.12/site-packages/git/cmd.py", line 800, in <lambda>
return lambda *args, **kwargs: self._call_process(name, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "~/.micromamba/envs/robot_descriptions/lib/python3.12/site-packages/git/cmd.py", line 1386, in _call_process
return self.execute(call, **exec_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "~/.micromamba/envs/robot_descriptions/lib/python3.12/site-packages/git/cmd.py", line 1183, in execute
raise GitCommandError(redacted_command, status, stderr_value, stdout_value)
git.exc.GitCommandError: Cmd('git') failed due to: exit code(128)
cmdline: git checkout 3d593a327ff8d5efd10f4926ccc477820e9252c1
stderr: 'fatal: Unable to create '~/.cache/robot_descriptions/mujoco_menagerie/.git/index.lock': File exists.
Another git process seems to be running in this repository, e.g.
an editor opened by 'git commit'. Please make sure all processes
are terminated then try again. If it still fails, a git process
may have crashed in this repository earlier:
remove the file manually to continue.'
Commit 3d593a327ff8d5efd10f4926ccc477820e9252c1 not found, let's fetch origin and try again...
Commit 3d593a327ff8d5efd10f4926ccc477820e9252c1 not found, let's fetch origin and try again...
Found commit 3d593a327ff8d5efd10f4926ccc477820e9252c1 successfully!
4 ~/.cache/robot_descriptions/mujoco_menagerie/franka_fr3/fr3.xml
Commit 3d593a327ff8d5efd10f4926ccc477820e9252c1 not found, let's fetch origin and try again...
Found commit 3d593a327ff8d5efd10f4926ccc477820e9252c1 successfully!
1 ~/.cache/robot_descriptions/mujoco_menagerie/franka_fr3/fr3.xml
At first read I had a hunch this error is related to the quick-and-dirty way we pass arguments via environment variables, which won't work well when we run in parallel. But in this instance the exception is rather raised from:
robot_descriptions.py/robot_descriptions/_cache.py
Lines 89 to 95 in 380aa5a
And the (currently uncaught) exception message reads:
Unable to create '~/.cache/robot_descriptions/mujoco_menagerie/.git/index.lock': File exists.