Got Error when running Unittest tests.test_compute_service.test_instance_methods
niejn opened this issue · 2 comments
Thanks a lot for providing such an excellent lib, I love it very much!
Python Environment: Python 3.6.7
Envs: CB_TEST_PROVIDER=gcp;GCP_DEFAULT_REGION=asia-east1;GCP_ZONE_NAME=asia-east1-b
when I am trying to run tests/test_compute_service.test_instance_methods,
Got Result: Tests Passed 6 of 6 tests, but got exception (Error during cleanup: 'NoneType' object has no attribute '_ip') when cleaning the resource floating ip.
It looks like the test code is waiting the GCPFloatingIP state change to FloatingIpState.UNKNOWN, but GCPFloatingIP.state only returns FloatingIpState.IN_USE or FloatingIpState.AVAILABLE. So the test code fip.wait_for([FloatingIpState.UNKNOWN], terminal_states=[FloatingIpState.ERROR]) is an infinite loop.
when the GCPFloatingIP is freed, the code in cloudbridge/providers/gcp/resources.py Class GCPFloatingIP function refreshfip = self._provider.networking._floating_ips.get(None, self.id)
the get function above returned None which is causing Exception 'NoneType' object has no attribute '_ip'
Detailed Error Stack are as follows
`Error during cleanup: CloudBridgeBaseException: <HttpError 404 when requesting https://www.googleapis.com/compute/v1/projects/possible-enigma-236403/zones/asia-east1-b/disks/cb-blkattch-c46cc2-4536a0?alt=json returned "The resource 'projects/possible-enigma-236403/zones/asia-east1-b/disks/cb-blkattch-c46cc2-4536a0' was not found"> from exception type: <class 'googleapiclient.errors.HttpError'>
Traceback (most recent call last):
File "/home/niejjn/codes/cloudbridge/cloudbridge/base/middleware.py", line 45, in wrap_exception
return next_handler.invoke(event_args, *args, **kwargs)
File "/home/niejjn/cloudbridge-2/cloudbridge-2.0.0/venv/lib/python3.6/site-packages/pyeventsystem/events.py", line 110, in invoke
result = self.callback(*args, **kwargs)
File "/home/niejjn/codes/cloudbridge/cloudbridge/providers/gcp/services.py", line 1225, in delete
disk=volume.name)
File "/home/niejjn/cloudbridge-2/cloudbridge-2.0.0/venv/lib/python3.6/site-packages/google_api_python_client-1.7.8-py3.6.egg/googleapiclient/_helpers.py", line 130, in positional_wrapper
return wrapped(*args, **kwargs)
File "/home/niejjn/cloudbridge-2/cloudbridge-2.0.0/venv/lib/python3.6/site-packages/google_api_python_client-1.7.8-py3.6.egg/googleapiclient/http.py", line 851, in execute
raise HttpError(resp, content, uri=self.uri)
googleapiclient.errors.HttpError: <HttpError 404 when requesting https://www.googleapis.com/compute/v1/projects/possible-enigma-236403/zones/asia-east1-b/disks/cb-blkattch-c46cc2-4536a0?alt=json returned "The resource 'projects/possible-enigma-236403/zones/asia-east1-b/disks/cb-blkattch-c46cc2-4536a0' was not found">
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/niejjn/codes/cloudbridge/cloudbridge/base/helpers.py", line 110, in cleanup_action
cleanup_func()
File "/home/niejjn/codes/cloudbridge/tests/test_compute_service.py", line 245, in
with cb_helpers.cleanup_action(lambda: test_vol.delete()):
File "/home/niejjn/codes/cloudbridge/cloudbridge/base/resources.py", line 469, in delete
return self._provider.storage.volumes.delete(self)
File "/home/niejjn/cloudbridge-2/cloudbridge-2.0.0/venv/lib/python3.6/site-packages/pyeventsystem/middleware.py", line 74, in wrapper
return dispatcher.dispatch(self, event, *args, **kwargs)
File "/home/niejjn/cloudbridge-2/cloudbridge-2.0.0/venv/lib/python3.6/site-packages/pyeventsystem/events.py", line 218, in dispatch
return handlers[0].invoke(event_args, *args, **kwargs)
File "/home/niejjn/cloudbridge-2/cloudbridge-2.0.0/venv/lib/python3.6/site-packages/pyeventsystem/events.py", line 78, in invoke
result = self.callback(event_args, *args, **kwargs)
File "/home/niejjn/codes/cloudbridge/cloudbridge/base/middleware.py", line 55, in wrap_exception
six.raise_from(cb_ex, e)
File "", line 3, in raise_from
cloudbridge.interfaces.exceptions.CloudBridgeBaseException: CloudBridgeBaseException: <HttpError 404 when requesting https://www.googleapis.com/compute/v1/projects/possible-enigma-236403/zones/asia-east1-b/disks/cb-blkattch-c46cc2-4536a0?alt=json returned "The resource 'projects/possible-enigma-236403/zones/asia-east1-b/disks/cb-blkattch-c46cc2-4536a0' was not found"> from exception type: <class 'googleapiclient.errors.HttpError'>
/home/niejjn/codes/cloudbridge/cloudbridge/factory.py:75: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
log.warn("Could not import provider: %s", e)
/home/niejjn/cloudbridge-2/cloudbridge-2.0.0/venv/lib/python3.6/site-packages/google_api_python_client-1.7.8-py3.6.egg/googleapiclient/_helpers.py:130: ResourceWarning: unclosed <ssl.SSLSocket fd=8, family=AddressFamily.AF_INET, type=2049, proto=6, laddr=('127.0.0.1', 55616), raddr=('127.0.0.1', 8123)>
return wrapped(*args, **kwargs)
/home/niejjn/cloudbridge-2/cloudbridge-2.0.0/venv/lib/python3.6/site-packages/google_api_python_client-1.7.8-py3.6.egg/googleapiclient/_helpers.py:130: ResourceWarning: unclosed <ssl.SSLSocket fd=10, family=AddressFamily.AF_INET, type=2049, proto=6, laddr=('127.0.0.1', 55634), raddr=('127.0.0.1', 8123)>
return wrapped(*args, **kwargs)
/home/niejjn/cloudbridge-2/cloudbridge-2.0.0/venv/lib/python3.6/site-packages/six.py:587: ResourceWarning: unclosed <ssl.SSLSocket fd=7, family=AddressFamily.AF_INET, type=2049, proto=6, laddr=('127.0.0.1', 55584), raddr=('127.0.0.1', 8123)>
return iter(d.items(**kw))
/home/niejjn/cloudbridge-2/cloudbridge-2.0.0/venv/lib/python3.6/site-packages/six.py:587: ResourceWarning: unclosed <ssl.SSLSocket fd=6, family=AddressFamily.AF_INET, type=2049, proto=6, laddr=('127.0.0.1', 55578), raddr=('127.0.0.1', 8123)>
return iter(d.items(**kw))
/home/niejjn/cloudbridge-2/cloudbridge-2.0.0/venv/lib/python3.6/site-packages/six.py:587: ResourceWarning: unclosed <ssl.SSLSocket fd=8, family=AddressFamily.AF_INET, type=2049, proto=6, laddr=('127.0.0.1', 55622), raddr=('127.0.0.1', 8123)>
return iter(d.items(**kw))
/home/niejjn/cloudbridge-2/cloudbridge-2.0.0/venv/lib/python3.6/site-packages/six.py:587: ResourceWarning: unclosed <ssl.SSLSocket fd=9, family=AddressFamily.AF_INET, type=2049, proto=6, laddr=('127.0.0.1', 55628), raddr=('127.0.0.1', 8123)>
return iter(d.items(**kw))
/home/niejjn/cloudbridge-2/cloudbridge-2.0.0/venv/lib/python3.6/site-packages/google_api_python_client-1.7.8-py3.6.egg/googleapiclient/_helpers.py:130: ResourceWarning: unclosed <ssl.SSLSocket fd=6, family=AddressFamily.AF_INET, type=2049, proto=6, laddr=('127.0.0.1', 55662), raddr=('127.0.0.1', 8123)>
return wrapped(*args, **kwargs)
Error during cleanup: 'NoneType' object has no attribute '_ip'
Traceback (most recent call last):
File "/home/niejjn/codes/cloudbridge/cloudbridge/base/helpers.py", line 110, in cleanup_action
cleanup_func()
File "/home/niejjn/codes/cloudbridge/tests/test_compute_service.py", line 372, in
lambda: helpers.cleanup_fip(fip)):
File "/home/niejjn/codes/cloudbridge/tests/helpers/init.py", line 164, in cleanup_fip
terminal_states=[FloatingIpState.ERROR])
File "/home/niejjn/codes/cloudbridge/cloudbridge/base/resources.py", line 168, in wait_for
self.refresh()
File "/home/niejjn/codes/cloudbridge/cloudbridge/providers/gcp/resources.py", line 1385, in refresh
self._ip = fip._ip
AttributeError: 'NoneType' object has no attribute '_ip'
/usr/lib/python3.6/json/decoder.py:355: ResourceWarning: unclosed <ssl.SSLSocket fd=11, family=AddressFamily.AF_INET, type=2049, proto=6, laddr=('127.0.0.1', 55646), raddr=('127.0.0.1', 8123)>
obj, end = self.scan_once(s, idx)
/usr/lib/python3.6/json/decoder.py:355: ResourceWarning: unclosed <ssl.SSLSocket fd=10, family=AddressFamily.AF_INET, type=2049, proto=6, laddr=('127.0.0.1', 55640), raddr=('127.0.0.1', 8123)>
obj, end = self.scan_once(s, idx)
/usr/lib/python3.6/json/decoder.py:355: ResourceWarning: unclosed <ssl.SSLSocket fd=7, family=AddressFamily.AF_INET, type=2049, proto=6, laddr=('127.0.0.1', 55674), raddr=('127.0.0.1', 8123)>
obj, end = self.scan_once(s, idx)
/usr/lib/python3.6/json/decoder.py:355: ResourceWarning: unclosed <ssl.SSLSocket fd=6, family=AddressFamily.AF_INET, type=2049, proto=6, laddr=('127.0.0.1', 55668), raddr=('127.0.0.1', 8123)>
obj, end = self.scan_once(s, idx)
/home/niejjn/cloudbridge-2/cloudbridge-2.0.0/venv/lib/python3.6/site-packages/google_api_python_client-1.7.8-py3.6.egg/googleapiclient/_helpers.py:130: ResourceWarning: unclosed <ssl.SSLSocket fd=8, family=AddressFamily.AF_INET, type=2049, proto=6, laddr=('127.0.0.1', 55720), raddr=('127.0.0.1', 8123)>
return wrapped(*args, **kwargs)
Ran 6 tests in 1233.988s
OK`
@niejn Thanks for reporting this issue. I think there is indeed an issue with the floating ip state, it should return FloatingIP.UNKNOWN at some point, especially as we are using fip.wait_for([FloatingIpState.UNKNOWN], terminal_states=[FloatingIpState.ERROR]) or else it'll wait for the entire timeout to elapse before erroring out. This probably slows down tests and any other situation where we are waiting for a floating ip to be deleted. This is something that can all probably be fixed in the refresh function itself.
Thanks a lot!