ckan-custom-forms: A JavaScript repository from luisfsantos22

ckanext-ext_v1

Ext_v1 is the first version of an objective, efficient and usable CKAN extension, whose primary goal is to allow users to create and fill one or more questionnaires. Its ultimate purpose is to evaluate information and collect more concise and precise results about all the stakeholders' experience and end-users’ data. Due to CKAN's default structure, the platform easily integrates with the extension.

For instance, a registered user creates a specific organisation and then adds several datasets to this organisation. With this extension, each dataset created has a particular purpose (e.g. processing a survey template for questionnaire rendering, storing questionnaire responses or storing users’ background information), and tasks for the correct providing and collection of information in a secure way.

The questionnaires have to be uploaded into a specific dataset of the platform, as we will explain further. Their data came from JSON files that were generated on an external website that handles standard survey generation for the web (https://surveyjs.io/Examples/Survey-Creator). Our extension is responsible for reading and taking all that data as well as rendering structured and organised questionnaires.

In short, using extras pair of key/values in the creation of new datasets, it is possible to define each dataset for a specific purpose. For now, we have three types of keys:

is_data_store: has the purpose of defining a dataset as a datastore to the questionnaires’ responses;
is_patient_store: has the purpose of defining a dataset as a datastore to the uploaded patients files;
is_templating:has the purpose of defining a dataset as a survey datastore.

Note: when creating new datasets, add the key into 'Custom Field' input. You can only use one for each dataset.

is_data_store

Datasets with this extra field will store data gathered from filling the questionnaires that were provided. Each questionnaire response is associated with a record and can be visualized by entering the resource and searching by id, username, questions or answers. All the resources can be automatically created, depending on the name of the questionnaire that it is associated with. In the following image, it is possible to see an example of a record and all the data recorded.

Note: all questions and consequent responses are stored

is_patient_store

With this key, a dataset is defined as a store to upload files with information from patients. For now, we only test one example (SKBA example file) and, although the data visualization is incorrect (rows and columns do not match), the information is well recognized.

It is important to refer that this dataset can only create one resource per patient.

is_templating

In order to allow CKAN to list questionnaires, a new dataset must be created with this extra field. Consequently, the resources within this type of datasets are associated with the questionnaires’ rendering.

For instance, if a user wants to add a new questionnaire for a certain community, it is mandatory to create a new resource in a dataset for 'templating'. Only then, a JSON file with the structure of the questionnaire can be uploaded to that resource (to be processed by the extension).

The follow image presents an example of a structure associated with a resource of this type of dataset.

Note: You can choose which datasets are private or public to the organization or all platform.

Create Questionnaire

The data from the JSON files that are uploaded to CKAN for questionnaire rendering is generated in an external website. To facilitate the perception and understanding of this process, we will explain, step by step, what must be done.

SurveyJS is an online visual survey creator and form builder that offers exactly what we want.

This is a critical part of the process. he structure used in the website is what the extension will try to understand to be able to render it in an efficient and organized way. So, we define a default structure, where some pages and corresponding properties are mandatory to avoid malformed questionnaires.

Go to https://surveyjs.io/Examples/Survey-Creator#content-result and, using the survey design, you can start creating the questionnaire.
Change the page properties to create and introduction page (title, name, description).
Create the right components to the introduction page (Component objects).
Add new pages to fill them with questions.
All the questions must be inside a Panel object. After insert one or several Panels there is two types of questions that our extension accepts:
- radiogroup: To add it into the questionnaire, simply choose the tool ‘RadioGroup’ and click or drag into the panel. It is possible to change the order of the questions by dragging them up and down. Having the object in the survey design and inside a panel, click on it and go to ‘Properties’. There you can define the default fields of a question (here the ‘Description’ field is ignored) and then you can define if it is required or not. By activating the field ‘Is required’ our extension will assume the obligation and the user will have to answer it. Having the question text, we need to configure the possible answers. For that, we need to go to ‘Choices’,a dropdown button in the ‘Properties’ area. It contains the default key/values generated by the website and they are the fields that we must change. It is possible to erase and add options, and change their values. For a correct definition of each option, the following steps must be followed:
  - change the ‘Text’ input to the value that will appear in the question as possible options;
  - change the ‘Value’ input with a snake case style (p.e not_at_all ). Basically, it is the same as the ‘Text’ input but converted into a snake case.
- single input: It is the classic type of questions where the user needs to write his own answer. To add it, choose the tool ‘Single input’ (it is also possible to order it). The rules are the same as for the radiogroup questions but in this case, there are no choices and it is possible to write a placeholder.
Finishing the questionnaire pages and having prepared the introduction page as well, the questionnaire is complete. Now we need to be able to access the raw data and then export it. For that, SurveyJS provides a JSON Editor. It is a tab that enables the visualization of questionnaire raw data. It provides the information in JSON format. To be able to export this data, this tab has several buttons with different actions. By clicking on the ‘Copy’ button, all the data is copied.
Once all the necessary data has been copied from the JSON Editor, the next step is to save it in a local file. To do it, use a text editor. Open a new file, paste all the data and then save it as a JSON file. It is mandatory that the file is saved in JSON format (p.e patients.json).
It is just necessary to create a new resource in a templating dataset and upload the JSON file to CKAN.
In CKAN, we provide an example JSON file that contains SurveyJS generated data and that can be imported into the website and then changed to the creators’ requirements.

Submit Questionnaire

Having questionnaires already in the platform, ‘ext_v1’ has the permissions to list all of them in the main page. Since each organization can have several templating datasets, each one is labelled with the name of the organization followed by the title of the dataset. In the image below, we can see part of the questionnaires as well as the overall information associated with them.

Important rules

The following points are the rules and features that need to be followed for our extension to work efficiently: * To create an organisation, you need to be a registered user;

We use Keycloak as an external server for the authentication process, so every user has to be registered there and the login is done in it as well;
Only admin and editors’ users can create new datasets and resources;
If a templating dataset is public, users from all organisations and non-registered users can visualise it and answer it as well;
If a templating dataset is private, only members from that organisation can have access to the questionnaires in the dataset. It is possible to add specific members as well by writing the username on the ‘Add Member’ page;
Datasets that store questionnaires’ responses (is_data_store) are automatically created;
Questionnaires that do not respect the default rules will return error messages to the final user. In that case, export the JSON data to SurveyJS and review it;
If a user does not answer all the required questions, an alert will be displayed;
Try to give simple identifiers/names to the questionnaires’ files to get cleaner and more objective words in the questionnaires list.

Requirements

This extension was tested and can be executed for version 2.8 of CKAN. Any other version wasn't tested so its not completly safe to use it in others versions.

Installation

To install ckanext-ext_v1 in a set of Docker images and configuration files to run a CKAN site (https://github.com/okfn/docker-ckan):
1. Go to Dockerfile in ckan folder /ckan/Dockerfile and add:
```
RUN pip install -e git+https://gitlab.ubiwhere.com/smart-cities-h2020/tenderhealth/ckan-custom-forms.git@master#egg=ckanext-ext_v1
```
2. Add the plugin ext_v1 to the ckan.plugins setting in your CKAN config file
3. Run or Restart CKAN container:
```
docker container start/restart <name_of_ckan_container>
```
To install ckanext-ext_v1 on local CKAN project:
1. Activate your CKAN virtual environment, for example:
```
. /usr/lib/ckan/default/bin/activate
```
2. Install the ckanext-ext_v1 Python package into your virtual environment:
```
pip install ckanext-ext_v1
```
3. Add ext_v1 to the ckan.plugins setting in your CKAN config file (by default the config file is located at /etc/ckan/default/production.ini).
4. Restart CKAN. For example if you've deployed CKAN with Apache on Ubuntu:
```
sudo service apache2 reload
```

Development Installation

To install ckanext-ext_v1 for development, activate your CKAN virtualenv and do:

git clone https://github.com/lsantos/ckanext-ext_v1.git
cd ckanext-ext_v1
python setup.py develop
pip install -r dev-requirements.txt