Cache contention?

[piyush0101]

We tried using pip-accel on our CI environment. It failed with strange errors. Notice the error near the bottom of the trace where it says CRC check failed. Multiple jobs run on our jenkins slave simultaneously so am not sure if there is a cache contention since those jobs are sharing the same cache location ~/.pip-accel and ~/.pip

Traceback (most recent call last):
11:02:42   File "/var/lib/jenkins/workspace/build-component-virtualenv-debian7/virtualenv/bin/pip-accel", line 9, in <module>
11:02:42     load_entry_point('pip-accel==0.12', 'console_scripts', 'pip-accel')()
11:02:42   File "/var/lib/jenkins/workspace/build-component-virtualenv-debian7/virtualenv/local/lib/python2.7/site-packages/pip_accel/__init__.py", line 106, in main
11:02:42     install_requirements(requirements)
11:02:42   File "/var/lib/jenkins/workspace/build-component-virtualenv-debian7/virtualenv/local/lib/python2.7/site-packages/pip_accel/__init__.py", line 233, in install_requirements
11:02:42     install_binary_dist(members, prefix=install_prefix, python=python)
11:02:42   File "/var/lib/jenkins/workspace/build-component-virtualenv-debian7/virtualenv/local/lib/python2.7/site-packages/pip_accel/bdist.py", line 206, in install_binary_dist
11:02:42     for member, from_handle in members:
11:02:42   File "/var/lib/jenkins/workspace/build-component-virtualenv-debian7/virtualenv/local/lib/python2.7/site-packages/pip_accel/bdist.py", line 76, in get_binary_dist
11:02:42     for member in archive.getmembers():
11:02:42   File "/usr/lib/python2.7/tarfile.py", line 1805, in getmembers
11:02:42     self._load()        # all members, we first have to
11:02:42   File "/usr/lib/python2.7/tarfile.py", line 2380, in _load
11:02:42     tarinfo = self.next()
11:02:42   File "/usr/lib/python2.7/tarfile.py", line 2315, in next
11:02:42     self.fileobj.seek(self.offset)
11:02:42   File "/usr/lib/python2.7/gzip.py", line 429, in seek
11:02:42     self.read(1024)
11:02:42   File "/usr/lib/python2.7/gzip.py", line 256, in read
11:02:42     self._read(readsize)
11:02:42   File "/usr/lib/python2.7/gzip.py", line 303, in _read
11:02:42     self._read_eof()
11:02:42   File "/usr/lib/python2.7/gzip.py", line 342, in _read_eof
11:02:42     hex(self.crc)))
11:02:42 IOError: CRC check failed 0x6eda328f != 0x74cdee77L
11:02:42 [ERROR] '/var/lib/jenkins/workspace/build-component-virtualenv-debian7/virtualenv/bin/python /var/lib/jenkins/workspace/build-component-virtualenv-debian7/virtualenv/bin/pip-accel install -U -r /var/lib/jenkins/workspace/build-component-virtualenv-debian7/etc/venv-requirements.txt' failed with return code 1
11:02:42 
11:02:42 '/var/lib/jenkins/workspace/build-component-virtualenv-debian7/virtualenv/bin/python /var/lib/jenkins/workspace/build-component-virtual

[xolox]

Thanks for the feedback!

Given the error message I would say this is caused by either cache contention or a broken disk, so given your bug report we'll go with cache contention :-). If I understand the problem correctly the latest version of pip-accel (0.12.1) should resolve the issue by moving cached binary distributions into place atomically. Please try it out.

Disclaimer: The latest release contains changes for Python 3 compatibility, I hope those didn't break anything else (the tests run on 2.6, 2.7 and 3.4 but they are not exhaustive). In other words, if you experience any further breakage please feel free to report it here on GitHub.

[piyush0101]

Thanks for the quick fix :), I will try this out and let you know how it goes.

[piyush0101]

Works like magic! Thanks for taking care of this :)