Implementing visio-acoustic associative memory model for fast mapping of word categories in small children
This repository contains the data, model code, and processing tools for the term paper. Paper online: https://bjarke.me/2019-cogsys-paper
After cloning this repository, install the dependencies:
pipenv install
Run the four pre-experiments:
pipenv run python tools/3-experiment/1_pre_experiment.py
pipenv run python tools/3-experiment/2_pre_experiment.py
pipenv run python tools/3-experiment/3_pre_experiment.py
pipenv run python tools/3-experiment/4_pre_experiment.py
Analyze the results:
pipenv run python tools/3-experiment/5_main_experiment.py
The output can be found in the data/3-experiment/
folder.
This repository is organized according to the following file structure:
data/
— all data related to the research1-visual/
— visual stimuli datagenerated/
graph/
— some statistics for the generated datapattern/
— visualization of the generated data
prototypes.csv
— the prototypes, not included in training setstimuli.csv
— the visual input stimuli
2-acoustic/
— acoustic stimuli dataoriginal/
1-wordlist/
— complete and manually augmented word list2-audio/
— audio recordings of narrated words
processed/
audio/
— automatically cut audio samples after running toolsgraph/
— some statistics for the cut audio samples after running toolswordlist/
— selected word list with 100 word senses (categories)
stimuli.csv
— the acoustic input stimuli
3-experiment/
graph/
— analyzed results and statistics for the experimentslog/
— log protocols after running the experiments
lib/
— implementation code in Pythondata/
— code related to data processing and generationmodel/
— model implementation
paper/
— code for the paper (LaTeX)img/
— images used in the paper
tools/
— tools to automatically process data and run experiments1-visual/
— tools related to visual stimuligenerate_patterns.py
— re-generate the visual dot patterns
2-acoustic/
— tools related to acoustic stimuli1-wordlist/
— tools related to word listsprocess_wordlist.py
— re-generate the selected word list
2-audio/
— tools related to audio recordingsprocess_full_recordings.py
— (currently defunct) cut full recordingsprocess_audio.py
— process audio files to generate stimuli data
3-experiment/
— tools related to running experiments1_pre_experiment.py
— run 1st pre-experiment2_pre_experiment.py
— run 2nd pre-experiment3_pre_experiment.py
— run 3rd pre-experiment4_pre_experiment.py
— run 4th pre-experiment5_main_experiment.py
— (currently defunct) run main experiment6_analyze.py
— analyze the results of all experiments7_prepare_images.sh
— copy images used in the paper
Pipfile
— track dependencies to installtestall.py
— run all unit tests