/mturk2gform

Primary LanguagePythonMIT LicenseMIT

!-- PROJECT LOGO -->

Mturk2Gform

Logo

The flexibility of Google forms on Amazon MTurk
Explore the docs »

View Demo · Report Bug · Request Feature

Table of Contents
  1. About The Project
  2. Usage
  3. Getting Started
  4. Contributing
  5. License
  6. Contact

About the Project

Do you need it?

✔️ You need to gather a big amount of human-labeled data.

✔️ This data should be answered on distinct forms following all a similar template, but with different questions each (image description for instance)

✔️You want to limit the number of workers answering a given form in order to guarantee diversity amongst your data source

If you still are still reading, chances are that Mturk2gform will help you saving some work/time!

What is it?

Mturk2Gform lets you easily link a set of generated Google forms to Amazon Mechanical Turk HITs, taking advantage of both frameworks benefits:

  • The generative capability of Google Appscript to automate personalized Google forms creation.
  • The robust features of Amazon MTurk to control task scheduling and workers payment.

What is it not?

Mturk2Gform does not aim at replacing one or the other frameworks, but rather to provide the user with a quick, scalable and reliable solution to automate human-labeled data gathering from google forms.

It consists of a wrapper class around the boto3 library and various functionalities from Google python client

Disclaimer: Although MTurk already provides the user with forms to gather data, its features are somehow limited in comparison with what can be done in Google Forms, in particular when it comes to generate a big number of these forms. (image integration, easy linkage with other drive documents etc.)

Alt Text

Usage

Mturk2Gform provides you with 3 levels of control on the Mturk HITs dependingon your needs:

  • High-level interface: a control panel based on Ipywidgets which lets you create, list, stop delete and monitor hits conveniently Alt Text

  • Mid-level interface: The Object-Oriented implementation of Mturk2Gform presents some extra methods in addition to the control panel

  • Low-level interface: every Mturk2Gform instance gives you full access to the underlying Mturk boto3 client as an instance attribute: if some functionalities are missing, you can always complete them on the fly.

In addition to these three levels of control, Mturk2Gform provides you with some extra features not present in Mturk or Google Forms:

  • Monitoring of HITs: you can limit the number of forms one worker can answer among your Google forms set

  • Template Appscript: to automatize the generation of ilimited number of personalized Google Forms

  • Confirmation Code: Although this feature is not native to google forms, Mturk2Gform implements a confirmation code features ensuring that workers indeed completed your forms

For other examples and features, please refer to the Documentation

Getting Started

1. Prerequisites

python >= 3.6

2. Installation

Using pip

pip install mt2gf
jupyter labextension install @jupyter-widgets/jupyterlab-manager

Cloning the repo

git clone https://github.com/ymentha14/mturk2gform.git

3. Credentials Settings

Amazon Credentials

It is assumed that you already possess an AWS account: if this is not the case you can set it up here

Once this is done, you need to download your public and secret key into a .csv file as described here. These keys will allow mt2gf to connect to your AWS account and help you managing the HITs.

Google Credentials

You need to set up the Google People API associated with your Google Drive account to establish the link with Mturk2Gform: once you are done, you can save the credentials.json file.

4. Forms Generation

Google Appscript

Mt2gf assumes you generated a serie of google forms (for ex. using Appscript ) present on a Google Drive folder along with a gform_mapping.txt file whose rows respect the following structure:

<form index>,<url to form>,<url to form spreadsheet>

Tipp: An Appscript template is provided to help you creating the form and linking the forms to their respective spreadsheets.

MTurk HIT Layout

The MTurk layout will have to describe what your task consists in: it is on this layout that the url to the google form wil appear. To make this hitlayout compatible with Mturk2gform, enter ${url} where you want the link to appear. (Detailed explanations here). Save the changes, selet your task and save the hitlayout_id that shows up when clicking on your task: Alt Text

Once these 4 steps are done, you can start running a template mt2gf notebook with your own data to familiarize with the framework.

Project Structure

├── LICENSE
├── README.md <-- The README you're currently reading
├── Dockerfile <-- Dockerfile
├── requirements.txt <-- python packages required for mt2gf
├── setup.py <-- setuptools script
├── bash_scripts
│   └── jupy_ext.sh <-- Script to install IPywidgets extensions
├── docs <-- Sphinx documentation ofthe project
├── Makefile
├── mt2gf
│   ├── appscript
│   │   ├── appscript_template.js
│   │   └── mouse_clicker_appscript.sh
│   ├── auto_drive.py
│   ├── auto_mturk.py
│   ├── constants.py
│   ├── fraudulous.py
│   ├── preprocess.py
│   ├── turk_widgets.py
│   ├── utils.py
│   └── watcher.py
├── notebooks
│   └── HIT_generator.ipynb
├── test_environment.py
└── tox.ini

Contributing

Contributions are what make the open source community such an amazing place to be learn, inspire, and create. Any contributions you make are greatly appreciated.

  1. Fork the Project
  2. Create your Feature Branch (git checkout -b feature/AmazingFeature)
  3. Commit your Changes (git commit -m 'Add some AmazingFeature')
  4. Push to the Branch (git push origin feature/AmazingFeature)
  5. Open a Pull Request

License

Distributed under the MIT License. See LICENSE for more information.

Contact

Your Name - yann.mentha@gmail.com

Project Link: https://github.com/ymentha14/mturk2gform