Data_proxy not working

Question

Data_proxy not working

Closed this issue 3 years ago · 14 comments

Data proxy seems to require a public ip address or ckan. We can avoid using it by using resource_proxy. Simply add resource_proxy to the plugins list in the ckan config file. This combine to a good usage of datapusher should enable the preview functionnality

Answer 1 · 2020-12-07T19:21:17.000Z

A double check on somebody else's ckan installation will be required to move forward. Once this is done, simply adding resource_proxy to the list of ckans.plugins in the configuration file should be enough

Answer 2 · 2020-12-09T14:40:43.000Z

@J-bytes resource_proxy added to the plugin list on development.ini on my VM. Unable to visualize a csv from @ data. I tried the link with both http and https. Am I missing something?

Answer 3 · 2020-12-09T14:54:24.000Z

@nfee006 when you created the dataset , was datapusher running ? If not , simply running it and changing the url ( when a new url is detected , datapusher re-fetch the data so changing it to a fake url and changing it back will do the trick) should do the trick . If however datapusher was running, I would need to dig deeper to find the root of the issue

Answer 4 · 2020-12-09T16:08:20.000Z

Reimported all the data with datapusher running and getting the same error as Natasha.

Answer 5 · 2020-12-09T16:13:55.000Z

@J-bytes Datapusher was running but I restarted CKAN and Datapusher & recreated the dataset just in case. I did realize that I was running an outdated version of ckanext-asc-csa so I I updated that. Still getting this. Maybe it's a firewall issue? I guess it's working on Citrix for you?

What file did you edit to add that warning by the way? It doesn't seem to translate.

Answer 6 · 2020-12-09T16:23:54.000Z

Translation are not handled in a file by file basis. See https://docs.ckan.org/en/2.8/extensions/translating-extensions.html . Translation are added to the .po file. I've suggested to wait until we do new releases to push to the server to review new translations. If needed I can do the translation today ( "@" me if necessary)

Answer 7 · 2020-12-09T16:25:04.000Z

Also, are urls to the data server publicly accessible or only internally through the vpn? If so, I would expect the links only to work on the server and not on our local VM

Answer 8 · 2020-12-09T16:34:34.000Z

@ data is only accessible via the VPN right now (my understanding) but my VM has access to the CSA network. I'm able to view those files if I go to the link directly.

Answer 9 · 2020-12-09T16:41:28.000Z

You are then officially further then I've ever been on the development of the visualisation as my VM can't access the VPN. Did you tried creating a dud datasets with some public facing csv file from another source? I had tried it and it seemed to work with another source of data. Try replicating this so we can pinpoint the error.

Answer 10 · 2020-12-09T16:44:37.000Z

Works fine with any public csv. It always has though on my side.

Answer 11 · 2020-12-09T16:45:47.000Z

when you loaded the data did the datapusher's console report any error?

Answer 12 · 2020-12-09T16:53:57.000Z

I have a sneaking suspicion that there may be something wrong with CKAN on my VM. I'll email you once I figure this out. Thanks!

Answer 13 · 2020-12-11T20:31:18.000Z

I think there might be something wrong with how data is stored and/or retrieved on/from the datastore. If you compare the output from https://demo.ckan.org/api/3/action/datastore_search?resource_id=_table_metadata to the same call on @ data, there is clearly something off. I get the same output as @DaTa on my VM CKAN as well.

Resource_proxy has been added to the plugin list on @ data.

Answer 14 · 2021-01-15T14:12:13.000Z

Resolved on @ data. The solution was to avoid using either - we fixed SSL certificate issues in datapusher, allowing us to use the datastore instead of a proxy.