No resources imported on a remote service
Closed this issue · 12 comments
Expected Behavior
A user should be able to import resources on a remote service.
Actual Behavior
Import Service Resources always displays this message with 0 resources fetched.
Steps to Reproduce the Problem
- Go to https://master.demo.geonode.org.
Data->Remote Services->Add remote service.- The import resources page will always have the above message.
Specifications
- GeoNode version: master
- Installation method (manual, GeoNode Docker, SPCGeoNode Docker):
- Platform:
- Additional details:
@marthamareal which remote resource URl are you testing?
@marthamareal which remote resource URl are you testing?
Am testing this, in particular, https://master.demo.geonode.org/services/969/, however there other existing services with the same status.
Am testing this, in particular, https://master.demo.geonode.org/services/969/, however there other existing services with the same status.
Did you inspect the GetCapabilities response from that WMS service? Resources can only be imported if they are advertized inside the GetCap response.
Did you inspect the GetCapabilities response from that WMS service? Resources can only be imported if they are advertized inside the GetCap response.
@giohappy, From here GetCapabilities is among the supported service requests.
Also in 3.3.x, resources import resources are listed, using this service
Could you share the URL that you set for the remote service?
Could you share the URL that you set for the remote service?
https://www.gebco.net/data_and_products/gebco_web_services/web_map_service/mapserv
Indeed it looks like the underlying harvesters are not obtaining the harvestable resources @afabiani. It happens for other services too.
The tasks that should retrieve harvestable resources are pending since days, so it looks like they're stuck inside the celery queues.
So far I had to remove that on-going session and refresh the others.
Restarting the GEBCO one, worked
Had to click on Rescan also in order to refresh the Harvesting Session
Not sure why that one was blocked... we would need probably to envisage an expiration time for those sessions so that in the case of a server crash or restart they won't block the new ones.
The scheduled harvesting mechanism must be made more robust. It happens that some harvesting sessions remain stuck. The frequent deployments might be the reason, with pending tasks failing to recover.
@ricardogsilva might give us some more insights on the cases that might get stuck in a running session.
We have stopped any scheduled harvester now.
@marthamareal I would like you to make some tests creating / updating the services and see if they are configured with the scheduling enabled, which shouldn't happen. The scheduling should be enabled only by configuration. The remote services should perform one-time harvesting and wait for requests triggered by the user.
@giohappy, I am testing this with master.demo, and these are the steps am taking with results:
- Removed the service URL i was testing with.
- Created the service with url (https://www.gebco.net/data_and_products/gebco_web_services/web_map_service/mapserv)
- On checking
Admin->Harvesting->Harvester, scheduling is enabled.
