Benchmarking of a few Fit file parsing libraries
ConnectStats is an iOS application that displays and analyse workout saved in Garmin or Strava. Since it started using the new Garmin Health API, the core input files where in the FIT Format or Flexible and Interoperable Data Transfer.
Both ConnectStats and its backend server need to process large amount of diverse fit files and need to extract the information quite generically. The initial implementation I used derived from the cpp sdk was actually quite slow and the experience on the app was noticeably bad for users while a file was processed.
ConnectStats now uses a swift library that does the core parsing in c using the official c sdk and that was much faster, and the user experience and performance now on the app is very acceptable.
The ConnectStats backend server also need to process fit files and, given the server is implemented in php, it uses phpFITFileAnalysis.php.
As the swift library used by the FitFileParser was moved to a Swift Package, I came accross another swift library, and it also opened up the possibility to use the swift library on linux for the server, should that be faster than the php implementation.
This lead to a quick benchmarking of the different libraries. We'll review swift/cpp/php and python as I used python quite a bit and if fast enough could be used on the server
Fit Parsing Libraries compared
- FitFileParser is a swift/c library based on the official c sdk
- FitDataProtocol is swift library
- phpFITFileAnalysis.php is a php library used by the ConnectStats server
- python-fitparse is a python library
- fitdecode is a python library
- fit-file-parser.js a javascript library (*)
- Official cpp SDK is the official cpp sdk.
- fit is a go implementation (*)
(*) thanks to karaul for the javascript help and tormoder for the go contribution.
Results
The benchmarking was done with two file corresponding to ConnectStats use. A fairly representative fit file of a 1h run (sample.fit). And an extreme case file of a 12h ultra marathon (large.fit). One should easily be able to adapt the test to the kind of files relevant for their use case.
The tests where run on a 2017 iMac. You should be able to run them for yourself by executing make in the root directory of the project or run the code from the xcodeproject included.
The fastest library ended up being the one combining c and swift FitFileParser. Interestingly, and a bit surprisingly to me, the php library phpFITFileAnalysis.php ended up very fast as well.
Given the two fastest library are the one used in practice by ConnectStats and its server, it won't be necessary to make any changes.
Time reported inside the tool
| language | command | file | time | Library |
|---|---|---|---|---|
| go | fit | sample.fit | 0.007 seconds | fit |
| swift/c | fitparser .fast | sample.fit | 0.037 seconds | FitFileParser |
| php | fitanalysis.php | sample.fit | 0.110 seconds | phpFITFileAnalysis.php |
| swift/c | fitparser .generic | sample.fit | 0.214 seconds | FitFileParser |
| swift | fitprotocol | sample.fit | 0.475 seconds | FitDataProtocol |
| cpp | fitsdkcpp | sample.fit | 0.557 seconds | Official cpp SDK |
| objc/cpp | fitsdkobjc | sample.fit | 0.467 seconds | Official objc SDK |
| python | fitfitdecode.py | sample.fit | 0.889 seconds | fitdecode |
| javascript | fit-file-parser.js | sample.fit | 1.025 seconds | fit-file-parser.js |
| python | fitfitparse.py | sample.fit | 1.121 seconds | python-fitparse |
| go | fit | large.fit | 0.052 seconds | fit |
| swift/c | fitparser .fast | large.fit | 0.475 seconds | FitFileParser |
| swift/c | fitparser .generic | large.fit | 1.475 seconds | FitFileParser |
| php | fitanalysis.php | large.fit | 5.979 seconds | phpFITFileAnalysis.php |
| swift | fitprotocol | large.fit | 6.763 seconds | FitDataProtocol |
| objc/cpp | fitsdkobjc | large.fit | 9.412 seconds | Official objc SDK |
| python | fitfitdecode.py | large.fit | 9.414 seconds | fitdecode |
| javascript | fit-file-parser.js | large.fit | 10.505 seconds | fit-file-parser.js |
| cpp | fitsdkcpp | large.fit | 11.147 seconds | Official cpp SDK |
| python | fitfitparse.py | large.fit | 11.420 seconds | python-fitparse |
Time accross architecture
Here is a comparison of running the parsing benchmark across a few architecture. The primary use case for the library being ConnectStats on iPhone, so here the benchmarks is compared accross an intel Macbook pro 16'' 2019, Apple Silicon Macbook Air 2020 and iPhone 12 Pro. It's quite interesting that the iphone and the macbook air both beat the MacBook pro...
| cpu | language | command | file | time |
|---|---|---|---|---|
| macbook pro intel i9 | swift/c | fitparser .fast | sample.fit | 0.034 seconds |
| macbook air M1 | swift/c | fitparser .fast | sample.fit | 0.022 seconds |
| iPhone 12 Pro | swift/c | fitparser .fast | sample.fit | 0.024 seconds |
| macbook pro intel i9 | swift/c | fitparser .generic | sample.fit | 0.174 seconds |
| macbook air M1 | swift/c | fitparser .generic | sample.fit | 0.120 seconds |
| iPhone 12 Pro | swift/c | fitparser .generic | sample.fit | 0.127 seconds |
| macbook pro intel i9 | swift/c | fitparser .fast | large.fit | 0.510 seconds |
| macbook air M1 | swift/c | fitparser .fast | large.fit | 0.335 seconds |
| iPhone 12 Pro | swift/c | fitparser .fast | large.fit | 0.327 seconds |
| macbook pro intel i9 | swift/c | fitparser .generic | large.fit | 1.486 seconds |
| macbook air M1 | swift/c | fitparser .generic | large.fit | 1.042 seconds |
| iPhone 12 Pro | swift/c | fitparser .generic | large.fit | 1.097 seconds |