openvstorage/framework

No Storagerouter found while removing vdisk

Opened this issue · 1 comments

As seen on an environment:

Apr 23 11:56:22 SV1SRV0001 ovs-workers[27826]: 2019-04-23 11:56:22 53700 -0700 - SV1SRV0001 - 27826/140536799938304 - celery/celery.worker.job - 238919 - DEBUG - Task accepted: ovs.vdisk.delete[7c460b91-25cc-435
d-ad57-35d85a37ba7f] pid:27838
Apr 23 11:56:22 SV1SRV0001 ovs-workers[27826]: 2019-04-23 11:56:22 55000 -0700 - SV1SRV0001 - 27838/140536799938304 - dal/hybrid - 337394 - ERROR - Could not load feature information
Apr 23 11:56:22 SV1SRV0001 ovs-workers[27826]: Traceback (most recent call last):
Apr 23 11:56:22 SV1SRV0001 ovs-workers[27826]:   File "/opt/OpenvStorage/ovs/dal/hybrids/storagerouter.py", line 163, in _features
Apr 23 11:56:22 SV1SRV0001 ovs-workers[27826]:     client = SSHClient(self, username='root')
Apr 23 11:56:22 SV1SRV0001 ovs-workers[27826]:   File "/opt/OpenvStorage/ovs/extensions/generic/sshclient.py", line 52, in __init__
Apr 23 11:56:22 SV1SRV0001 ovs-workers[27826]:     storagerouter = StorageRouter(endpoint.guid)
Apr 23 11:56:22 SV1SRV0001 ovs-workers[27826]:   File "/opt/OpenvStorage/ovs/dal/dataobject.py", line 237, in __init__
Apr 23 11:56:22 SV1SRV0001 ovs-workers[27826]:     self.__class__.__name__, self._guid
Apr 23 11:56:22 SV1SRV0001 ovs-workers[27826]: ObjectNotFoundException: StorageRouter with guid '42948a18-721a-44c5-b6c7-9a575b1c56af' could not be found
Apr 23 11:56:22 SV1SRV0001 ovs-workers[27826]: 2019-04-23 11:56:22 58800 -0700 - SV1SRV0001 - 27826/140536799938304 - celery/celery.worker.job - 238920 - ERROR - Task ovs.vdisk.delete[7c460b91-25cc-435d-ad57-35d
85a37ba7f] raised unexpected: TypeError("'NoneType' object has no attribute '__getitem__'",)
Apr 23 11:56:22 SV1SRV0001 ovs-workers[27826]: Traceback (most recent call last):
Apr 23 11:56:22 SV1SRV0001 ovs-workers[27826]:   File "/usr/lib/python2.7/dist-packages/celery/app/trace.py", line 240, in trace_task
Apr 23 11:56:22 SV1SRV0001 ovs-workers[27826]:     R = retval = fun(*args, **kwargs)
Apr 23 11:56:22 SV1SRV0001 ovs-workers[27826]:   File "/usr/lib/python2.7/dist-packages/celery/app/trace.py", line 438, in __protected_call__
Apr 23 11:56:22 SV1SRV0001 ovs-workers[27826]:     return self.run(*args, **kwargs)
Apr 23 11:56:22 SV1SRV0001 ovs-workers[27826]:   File "/opt/OpenvStorage/ovs/lib/vdisk.py", line 198, in delete
Apr 23 11:56:22 SV1SRV0001 ovs-workers[27826]:     if 'directory_unlink' in storagerouter.features['volumedriver']['features']:
Apr 23 11:56:22 SV1SRV0001 ovs-workers[27826]: TypeError: 'NoneType' object has no attribute '__getitem__'
Apr 23 11:57:02 SV1SRV0001 ovs-workers[27826]: 2019-04-23 11:57:02 48500 -0700 - SV1SRV0001 - 27826/140536799938304 - celery/celery.worker.strategy - 238921 - INFO - Received task: ovs.storagerouter.ping[9473334
8-cfdd-447f-ae00-a76b2515824f]

I suspect a second delete that has already removed the link between the volumedriver and the framework and for some reason the vdisk is still known to the Framework.