Run against some portals on staging

Question

Run against some portals on staging

Closed this issue 3 years ago · 9 comments

Floppy commented 8 years ago

Add some portals here to test against.

Answer 1 · 2016-07-09T01:43:42.000Z

If you're looking for CKAN portals to test, here's my top 3:

Or my full list. Note: they're not all CKAN portals.

Answer 2 · 2016-07-29T10:38:32.000Z

Canada: http://open.canada.ca

Answer 3 · 2016-07-29T12:21:20.000Z

To test an organisation within a portal try http://data.sa.gov.au/data/organization/department-of-planning-transport-and-infrastructure or https://data.qld.gov.au/dataset?organization=transport-and-main-roads

Answer 4 · 2016-08-16T13:03:59.000Z

https://data.qld.gov.au works
http://data.sa.gov.au mostly works but broke with datasets that were links to other portals
https://data.brisbane.qld.gov.au/ works (tested with 'recreation' subset)
http://open.canada.ca/data/en/api works (but had a lot of unanswered questions)

Answer 5 · 2016-08-16T20:34:24.000Z

Thanks @digitalWestie for testing the Australian portals.

I assume this was tested locally as I don't see the results on http://staging.certificates.theodi.org/en/datasets (which by the way isn't rendering fully for me).

What will be the approach for portal owners that harvest from other portals and encounter the data.sa.gov.au issue above? Should they run a collection of organisation based campaigns or just ignore the errors from a full run?

Answer 6 · 2016-08-16T21:16:55.000Z

The data.sa.gov.au issue is in #1436, and is on the list to resolve - hopefully we can make it follow the proper links.

Answer 7 · 2016-08-18T16:12:53.000Z

See #1442

But just as I pushed this up I realised, do we really want to include harvested datasets in these campaigns? Do we not want to run campaigns against a specific portal? Not just any that it comes across?

Answer 8 · 2016-08-18T19:14:09.000Z

@digitalWestie No you don't want to include them. Just elegantly ignore them (that's what I meant in my last comment above).

To scan my whole portal and not get any errors due to harvested metadata, I would prefer that I just specify the portal (e.g. data.sa.gov.au) and the campaign intelligently ignores harvested metadata.

I'm not sure how you are determining what is harvested from elsewhere. My guess was by organisation, hence my question "Should they run a collection of organisation based campaigns [to avoid getting errors when running the campaign]?".

Hope that's clear - sorry.

So perhaps no need for #1436?

Answer 9 · 2017-05-05T02:42:24.000Z

Can this be closed given #1442 ?