Tasked to automate the submission of Datasets and Resources from various Organizations throughout the municipality of the City of Denton to the Open Data CKAN website.
-
CKAN provides a front-end interface for users to upload datasets and resources, making data accessible to find, share, and use.
-
the CKAN API provides two means for organizations to upload data:
- as a data blob via the FileStore API
- as a searchable ElasticSearch persistent query layer via the DataStore API
-
SQL Database Export -> DataStore API
- Data to upload to CKAN will be provided in CSV or JSON as a direct export from the current systems, and upcoming Denton DataWarehouse
- Each Organization will determine what DataSets to upload, each DataSet containing specific Resources (files, external API endpoints, external URL's to display)
- Each DataSet will be a SQL query, which will be exported and passed to the DataStore API for creation (of new files), or modification/syncing (of existing files)
-
CSV Export -> DataStore API
- less optimal/automated, but still viable
- CSV exported data passed to system administrator for adding to the CKAN website via the DataStore API
Resources
CKAN Terminology
FileStore: allows users to upload data files to CKAN resources, and to upload logo images for groups and organizations
DataStore: provides an ad hoc database for storage of structured data from CKAN resources. Data can be pulled out of resource files and stored in the DataStore.
DataStore API](https://docs.ckan.org/en/latest/maintaining/datastore.html#the-datastore-api): search, filter and update the data, without having to download and upload the entire data file
Original Contributors
Emily Aquin: https://github.com/eaquin1
Andrew Bement: https://github.com/bemental
Brent Grim: https://github.com/brentgrim
Victor Faner: https://gitlab.com/y_hat
Ryan Paglinawan: https://github.com/rymotion