/collect

ODK Collect is an Android app for filling out forms. It's been used to collect billions of data points in challenging environments around the world. Contribute and make the world a better place! ✨📋✨

Primary LanguageJavaOtherNOASSERTION

ODK Collect

Platform License Build status codecov.io Slack status

ODK Collect is an Android app for filling out forms. It is designed to be used in resource-constrained environments with challenges such as unreliable connectivity or power infrastructure. ODK Collect is part of Open Data Kit (ODK), a free and open-source set of tools which help organizations author, field, and manage mobile data collection solutions. Learn more about the Open Data Kit project and its history here and read about example ODK deployments here.

ODK Collect renders forms that are compliant with the ODK XForms standard, a subset of the XForms 1.1 standard with some extensions. The form parsing is done by the JavaRosa library which Collect includes as a dependency.

Table of Contents

Learn more about ODK Collect

Release cycle

New versions of ODK Collect are generally released on the last Sunday of a month. We freeze commits to the master branch on the preceding Wednesday (except for bug fixes). Releases can be requested by any community member and generally happen every 2 months. @yanokwa pushes the releases to the Play Store.

Suggesting new features

We try to make sure that all issues in the issue tracker are as close to fully specified as possible so that they can be closed by a pull request. Feature suggestions should be described in the forum Features category and discussed by the broader user community. Once there is a clear way forward, issues should be filed on the relevant repositories. More controversial features will be discussed as part of the Technical Steering Committee's roadmapping process.

Setting up your development environment

  1. Download and install Git and add it to your PATH

  2. Download and install Android Studio

  3. Fork the collect project (why and how to fork)

  4. Clone your fork of the project locally. At the command line:

     git clone https://github.com/YOUR-GITHUB-USERNAME/collect

    If you prefer not to use the command line, you can use Android Studio to create a new project from version control using https://github.com/YOUR-GITHUB-USERNAME/collect.

  5. Open the project in the folder of your clone from Android Studio. To run the project, click on the green arrow at the top of the screen.

  6. Make sure you can run unit tests by running everything under collect_app/src/test/java in Android Studio or on the command line:

    ./gradlew testDebug

  7. Make sure you can run instrumented tests by running everything under collect_app/src/androidTest/java in Android Studio or on the command line:

    ./gradlew connectedAndroidTest

    Note: You can see the emulator setup used on CI in .circleci/config.yml.

Testing a form without a server

When you first run Collect, it is set to download forms from https://opendatakit.appspot.com/, the demo server. You can sometimes verify your changes with those forms but it can also be helpful to put a specific test form on your device. Here are some options for that:

  1. The All Widgets form from the default Aggregate server is here. You can also try example forms and test forms or make your own.

  2. Convert the XLSForm (xlsx) to XForm (xml). Use the ODK website or XLSForm Offline or pyxform.

  3. Once you have the XForm, use adb to push the form to your device (after enabling USB debugging) or emulator.

    adb push my_form.xml /sdcard/odk/forms/

  4. Launch ODK Collect and tap Fill Blank Form. The new form will be there.

Using APIs for local development

To run functionality that makes API calls from your debug-signed builds, you may need to get an API key or otherwise authorize your app. The API keys included in the source code are the ones used for releases and they only work with release builds signed with the release keys.

Google Drive and Sheets APIs - Follow the instructions in the "Generate the signing certificate fingerprint and register your application" section from here. Enable the Google Drive API here. Enable the Google Sheets API here.

Google Maps API - Getting a Google Maps API key now requires providing a credit card number. As of October 2018, there is some free API usage provided and the card will not be charged without explicit user approval. You should carefully read the terms before providing a credit card number. Once you have created a billing account, follow the instructions here and paste your key in the AndroidManifest as the value for com.google.android.geo.API_KEY. Please be sure not to commit your personal API key to a branch that you will submit a pull request for.

Debugging JavaRosa

JavaRosa is the form engine that powers Collect. If you want to debug or change that code while running Collect, you have two options. You can include the source tree as a module in Android Studio or include a custom jar file you build.

Source tree

  1. Get the code from the JavaRosa repo
  2. In Android Studio, select File -> New -> New Module -> Import Gradle Project and choose the project
  3. In Collect's build.gradle file, find the JavaRosa section: 
    implementation("org.opendatakit:opendatakit-javarosa:x.y.z") {
    ...
}
  4. Replace the first line like this, using javarosa or whatever name you specified when importing: 
    implementation (project(path: ':javarosa')) {
    ...
}

Jar file

  1. In JavaRosa, change the version in build.gradle and build the jar

    jar {
    baseName = 'opendatakit-javarosa'
    version = 'x.y.z-SNAPSHOT'

  2. In Collect, add the path to the jar to the dependencies in build.gradle

    compile files('/path/to/javarosa/build/libs/opendatakit-javarosa-x.y.z-SNAPSHOT.jar')

Contributing code

Any and all contributions to the project are welcome. ODK Collect is used across the world primarily by organizations with a social purpose so you can have real impact!

Issues tagged as good first issue should be a good place to start. There are also currently many issues tagged as needs reproduction which need someone to try to reproduce them with the current version of ODK Collect and comment on the issue with their findings.

If you're ready to contribute code, see the contribution guide.

Contributing translations

If you know a language other than English, consider contributing translations through Transifex.

Translations are updated right before the first beta for a release and before the release itself. To update translations, download the zip from https://www.transifex.com/opendatakit/collect/strings/. The contents of each folder then need to be moved to the Android project folders. A quick script like the one in this gist can help. We currently copy everything from Transifex to minimize manual intervention. Sometimes translation files will only get comment changes. When new languages are updated in Transifex, they need to be added to the script above. Additionally, ApplicationConstants.TRANSLATIONS_AVAILABLE needs to be updated. This array provides the choices for the language preference in general settings. Ideally the list could be dynamically generated.

Contributing testing

All pull requests are verified on the following devices (ordered by Android version):

Our regular code contributors use these devices (ordered by Android version):

The best way to help us test is to build from source! If you aren't a developer and want to help us test release candidates, join the beta program!

Testing checklists can be found on the Collect testing plan.

If you have finished testing a pull request, please use a template from Testing result templates to report your insights.

Downloading builds

Per-commit debug builds can be found on CircleCI. Login with your GitHub account, click the build you'd like, then find the APK in the Artifacts tab.

Current and previous production builds can be found on the ODK website.

Creating signed releases for Google Play Store

Project maintainers have the keys to upload signed releases to the Play Store.

Maintainers have a secrets.properties file in the collect_app folder with the following:

// collect_app/secrets.properties
RELEASE_STORE_FILE=/path/to/collect.keystore
RELEASE_STORE_PASSWORD=secure-store-password
RELEASE_KEY_ALIAS=key-alias
RELEASE_KEY_PASSWORD=secure-alias-password

Maintainers also have a google-services.json file in the collect_app/src/odkCollectRelease folder. The contents of the file are similar to the contents of collect_app/src/google-services.json.

To generate official signed releases, you'll need the keystore file, the keystore passwords, a configured collect_app/secrets.properties file, and a configured collect_app/src/odkCollectRelease/google-services.json file. Then run ./gradlew assembleOdkCollectRelease. If successful, a signed release will be at collect_app/build/outputs/apk.

Troubleshooting

Error when running Robolectric tests from Android Studio on macOS: build/intermediates/bundles/debug/AndroidManifest.xml (No such file or directory)

Configure the default JUnit test runner configuration in order to work around a bug where IntelliJ / Android Studio does not set the working directory to the module being tested. This can be accomplished by editing the run configurations, Defaults -> JUnit and changing the working directory value to $MODULE_DIR$.

Source: Robolectric Wiki.

Android Studio Error: SDK location not found. Define location with sdk.dir in the local.properties file or with an ANDROID_HOME environment variable.

When cloning the project from Android Studio, click "No" when prompted to open the build.gradle file and then open project.

Execution failed for task ':collect_app:transformClassesWithInstantRunForDebug'.

We are unsure why this problem occurs, but it seems to happen with IntelliJ IDEA and not with Android Studio. Try turning off Instant Run and see if that helps.

Moving to the main view if user minimizes the app

If you build the app on your own using Android Studio (Build -> Build APK) and then install it (from an .apk file), you might notice this strange behaviour thoroughly described: #1280 and #1142.

This problem occurs building other apps as well.

gradlew Failure: FAILURE: Build failed with an exception.

If you encounter an error similar to this when running gradlew:

FAILURE: Build failed with an exception

What went wrong:
A problem occurred configuring project ':collect_app'.
> Failed to notify project evaluation listener.
   > Could not initialize class com.android.sdklib.repository.AndroidSdkHandler

You may have a mismatch between the embedded Android SDK Java and the JDK installed on your machine. You may wish to set your JAVA_HOME environment variable to that SDK. For example, on macOS:

export JAVA_HOME="/Applications/Android Studio.app/Contents/jre/jdk/Contents/Home"

Note that this change might cause problems with other Java-based applications (e.g., if you uninstall Android Studio).

gradlew Failure: java.lang.NullPointerException (no error message).

If you encounter the java.lang.NullPointerException (no error message). when running gradlew, please make sure your Java version for this project is Java 8.

This can be configured under File > Project Structure in Android Studio, or by editing $USER_HOME/.gradle/gradle.properties to set org.gradle.java.home=(path to JDK home) for command-line use.