FitBit Python Library
FitBit Group Data Downloader A Python console application for FitBit data gathering. Includes the fitbit.py library with fitbit class module, which contains the necessary methods for completing the FitBit OAuth authentication and for querying the FitBit API. Also contains the fitbit_authenticate.py script which walks a user through the authentication and data retrieval process as outlined below. The application has been developed to meet our needs for a specific research study, but we are slowly making the code configurable.
Heritage
Based on the fitbit.py library originally found at https://github.com/jplattel/FitBit.py.git and modified for Oauth2 use at https://github.com/jflasher/FitBit.py/blob/master/fitbit.py. This new fork of https://github.com/rhavasy/FitBit-Group-Data-Downloader was modified to add new API calls and output data to a SQL database.
Design
This script was designed to allow the retrieval of FitBit data for a list of users, outputting each user to a SQL database. This works really well for a group of many users with the option to enter each user into the shell, or to read user information from a CSV file. FitBit currently enforces an API rate limit of 150 calls on a per user basis. This application is intended to run weekly to add new users to the database for authentication and obtain their last sync date and monthly to pull down all report data including food logs, activity and sleep data. As food logs do not have a time series option, they require 1 API call per day of data. As a result of these API limitations, this application does not work well if more than a few months of data need to be pulled for a given user.
Use
The following additional libraries will need to be installed via pip install or easy_install
- oauth
- mechanize
- pyodbc
- passlib
- beautifulsoup4
Note: If you have trouble installing beautiful soup or are receiving a module not found error, see Found at http://stackoverflow.com/questions/12228102/how-to-install-beautiful-soup-4-with-python-2-7-on-windows. See also the beginning of http://spartanthe.blogspot.com/2009/02/installing-beautifulsoup-for-python-in.html. You may need to run python setup.py install from the directory that you have unzipped BeautifulSoup in (assuming that you have added Python to your system PATH - if you haven't and you don't want to you can run C:\Path\To\Python27\python "C:\Path\To\BeautifulSoup\setup.py" install).
This library contains the following files:
The INI file is where you enter your Fitbit consumer key information, enter database connection information and make selections for the menu items that are configurable.
Contains the scripts needed to create the database on a SQL server.
Builds the database connection string based on information supplied in the INI file. This module also contains code to encrypt the Fitbit user password in the database.
Builds the final menu in the user interface with available actions based on information supplied in the INI file.
Parse variables from the INI file.
List of users, Fitbit usernames and passwords that need to be authenticated via Oauth and saved to the database.
The application contains some error trapping and writes the results to this file. Current available errors are incorrect password, duplicate user, and no Fitbit device mapped.
The fitbit.py library provides the necessary Oauth functions to complete the "Oauth Dance" as well as make a series of API calls and parse the XML in these calls to the SQL database. This includes methods for retrieving request and access tokens.
Allows a user to pick from a menu of actions.
- Add a new user: adds users one at a time to the User table in the database by prompting for required information in the shell and then makes the series of calls to authenticate the user and save the access token.
- Manage user from spreadsheet: iterates through all users in the enrollments spreadsheet instead of prompting for manual input for one user at a time. Also allows for batch disenrollments.
- Get Weekly Report: for our purposes, this option was created to pull down the last upload date (last sync date from the Device API call) and output that information to the Device table in the database.
- Get Monthly Reports: makes all of the API calls to obtain activity data, device data, sleep information and food logs. It uses a last refresh date in the user table as the from date (or enrolled date for the first report pull) and pulls down all information from yesterday to that date.
- Get Sleep, Get Activity, Get Foodlog: if these are set to True in the INI file, this allows all the calls in the GetAll option to be made separately.
Note: This library is designed to use FitBit's "desktop" app authentication using a PIN, rather than a callback URL. Therefore, the user will be required to enter the username and password for each user on the list during the GetAccessToken sequence. This repository was enhanced to ask for this information in the fitbit_authenticate script and scrape the PIN in the background.
- Register an app with FitBit and obtain Consumer Key and Consumer Secret. Note: the app must be registered as a desktop app, not a web app.
- Add the Consumer Key and Consumer secret values to the fitbit_config.ini file. Make interface selections by setting options to True or False.
- Create the SQL database by running the scripts contained in the repository on your existing SQL server (or download SQL Express to your local or remote machine here: http://www.microsoft.com/en-us/download/details.aspx?id=29062 .
- Input database connection information into the INI file. The database.py module will build the connection string. Note: if you are using a remote instance of SQL server, you must make sure it is configured to make remote connections. Here is a useful tutorial: http://blogs.msdn.com/b/walzenbach/archive/2010/04/14/how-to-enable-remote-connections-in-sql-server-2008.aspx .
- Execute the fitbit_authenticate.py script and make appropriate menu selections. Review the errors.txt file upon exiting the program.
- There is light error trapping only
- A random SSLError: [Errno 8] _ssl.c:507 EOF occurred in violation of protocol occurs on Windows (replicated twice) but has not been replicated on Linux.
Modifications made by Rob Havasy, Alyssa Woulfe, and Ahnissa Beaupre at the Center for Connected Health, Boston, MA. For help (best effort only) or information contact: rhavasy@partners.org, awoulfe@partners.org, or abeaupre@partners.org Center for Connected Health 25 New Chardon St. Suite 300 Boston, MA 02114 http://www.connected-health.org/