Utilities for Processing the FRAMES Corpus available here. Frames is meant to encourage research towards conversational agents which can support decision-making in complex settings, in this case – booking a vacation including flights and a hotel. The utilities process the original transcripts into plain text or json formats.
frames_to_json.py script processes the dialogues from the original format into .json files using the format outlined below. Each dialogue set (train and test) is output as a separate .json file. This format is intended to facilitate annotation of the dialogue using the Conversation Analysis Modelling Schema.
frames_to_text.py processes the dialogues from the .json format into plain text files, with one line per-utterance, using the format outlined below. Setting the utterance_only flag to true will remove the speaker label from the output text files.
frames_utilities.py script contains various helper functions for loading/saving and processing the data.
The original slots and dialogue scenario has been preserved to maintain compatibility with the original dataset. Where no dialogue act is present it is replaces with 'null'. By default utterances are written one per line in the format Speaker | Utterance Text | Dialogue Act Tag. This can be changed to only output the utterance text by setting the utterance_only_flag = True.
USR|I'd like to book a trip to Atlantis from Caprica on Saturday, August 13, 2016 for 8 adults.|inform
USR|I have a tight budget of 1700.|inform
SYS|Hi...I checked a few options for you, and unfortunately, we do not currently have any trips that meet this criteria.|sorry
SYS|Would you like to book an alternate travel option?|sorry
The following is an example of the JSON format for the FRAMES corpus.
{
"dataset": "dataset_name",
"num_dialogues": 1,
"dialogues": [
{
"dialogue_id": "dataset_name_1",
"num_utterances": 2,
"utterances": [
{
"speaker": "A",
"text": "Utterance 1 text.",
"ap_label": "AP-Label",
"da_label": "DA-Label"
},
{
"speaker": "B",
"text": "Utterance 2 text.",
"ap_label": "AP-Label",
"da_label": "DA-Label",
"slots": { //Optional
"slot_name": "slot_value"
}
}
],
"scenario": { //Optional
"db_id": "1",
"db_type": "i.e booking",
"task": "i.e book",
"items": []
}
}
]
}The original paper for the FRAMES corpus: Asri, L. El, Schulz, H., Sharma, S., et al., (2017) Frames: A Corpus for Adding Memory to Goal-Oriented Dialogue Systems.
The code within this repository is distributed under the GNU General Public License.