Feature Request: Add Accelerometer Values
vijaykarla opened this issue · 19 comments
Adding accelerometer values (G-Force in X, Y, Z along with tilt, roll and pitch angles) will make the app usable for multi purposes.
What kind of purposes?
Android device fixed on a vehicle with GPS Logger having GPS sensor combined with accelerometer sensor could be used for security and safety.
Also, the generated data could be useful for research purposes to learn characteristics of specific movements if the recording device is fixed to the moving system in question.
I'd be thankful for some pointers where I'd need to start to add such a feature. I imagine there would be at least
- some UI parts to show the state of the additional sensors
- something to extend the logging fields and to extend the "send to server" macros
- the actual sensor data recording, I have some code that queries the SensorManager from SDK 24 for it
Your starting point would be registering the service as a listener, and then handling the intent where the arr contains the activity's details.
In my example I pass the activityRecognitionEvent via an event bus and then handle it here to get the most probable activity.
This doesn't use the SensorManager, instead only relies on the activity recognition provided by Google Play Services, so I believe a lot of the detailed research-oriented information you're looking for will be missing.
However implementing it would be similar, you'd just have a separate thread, which means a separate object is required to hold the most recent values of the sensor readings somewhere.
So to actually get this information to the various senders and loggers - it needs to be added in here as an extra on the Location object. But you'll see that the actual value is read from a session object, so it'll never be a 100% match, just the closest or latest one. Reading it is just a matter of getExtras() with an example here
As for UI... since the values would be stored in a session object or session list it'll be up to the UI update part to refresh those fields.
So I now have some code that is successfully collecting information on accelerometer, orientation and compass heading based on accelerometer and magnetic field sensor.
It needs clean-up, maybe - if reasonably possible - a few tests, ui polishing, testing on more devices and translations of the relevant strings.
I've made a few test runs on real devices, most of them being Samsung Galaxy devices.
First:
- non-factory fresh battery
- a number of other applications running at the same time,
- enabled data connection
- 2s distance
- 2 sensor data samples per fix (leading to up to 20 onChange-Events for the sensors)
- constantly enabled GPS
- 25m recording, 20% battery usage (79% -> 59%)
Second: - non-factory fresh battery
- a number of other applications running at the same time,
- disabled data connection
- 2s distance
- 2 sensor data samples per fix (leading to up to 20 onChange-Events for the sensors)
- constantly enabled GPS
- 90m recording, 50% battery usage (59% -> 9%)
Third: - non-factory fresh battery
- a number of other applications running at the same time,
- disabled data connection
- 2s distance
- 2 sensor data samples per fix (leading to up to 20 onChange-Events for the sensors)
- constantly enabled GPS
- 3,5h recording, 70% battery usage (90% -> ~20%)
Oops I only just saw this. Hmm those battery drain numbers look really bad! I'm getting worried that you're putting in all this work now and going down a path already. It's good you measured some of the drain right now. I just want to reiterate that the battery performance is the most important aspect for GPSLogger, cannot compromise on this.
I tried doing a quick compare - but there seem to be multiple alarm managers calls, this should be only one and have it kick off the various readers (location, sensor, etc).
GPX10 won't take custom extensions, no point adding to that file format, no intentions of moving to GPX11 either.
Maybe a switch to select between GPX10 and GPX11 would be an option, after all the fields mentioned in #418 could be mapped into extensions upon a switch to GPX11.
The current battery consumption already yielded quite useful data for me, so that's been useful already. However with the inclusion to the mainline I'd surely be interested to see how the consumption could be lowered.
It would likely be useful to try this on a phone with a minimal amount of running software as well.
After a slight clean up of the crufty residues of multiple approaches to the problem some assistance with the alarm manager / scheduling would be very welcome, I struggled with those to some amount.
Can't move to GPX11 - it'll be dropping some important fields for some custom ones. More people use the mentioned fields than the sensor fields. Better options would then be KML since it's extension friendly already. CSV could take it or it goes into a .sensorlog file - seems like there are lots of fields in which case maybe it can be its own file with CSV format, that way not restricted by the formats and their quirks. In a way it's sort of like NMEA.
I can have a look at what's involved in consolidating those. If you do the pull request thing for your latest work, I can try to see how they can be combined. I'll also measure battery drain on my own phones, I have a few with no software installed.
Sorry I never mentioned the file format earlier I think I had just made assumptions - but even minor changes have big impacts on users as a many people have many workflows that's not always obvious.
I see, that's why I would suggest an ability to switch between writing GPX10 and GPX11 with extensions.
The recent work does KML already, so that's there already. I'd have to see into what would be best in regards to parsing on my side.
Sensorlog might be possible, however I prefer the direct coupling of GPS and sensor data. The current solution basically sensor data samples to their GPS point.
Getting more battery data surely is a good thing. Also worth considering might be that the battery discharge curve often is not linear, so that experiments are probably best done starting with a full battery - I did not have the option with the data points I mentioned earlier on.
GPX11 - Even with switches/options, that's too big a change, the impact is going to be too large across other parts of the app. That's why I suggested .sensorlog - the user would have to join up the data by timestamps with the location, and it won't stomp on the existing file formats. Otherwise the user would only get sensor data if they use KML but not GPX; CSV would get bloated way too much. I'm considering the sensor data as an extra rather than an integral part of the app.
I mentioned in another thread - I have to balance features between what users consider to be priorities and what works in general for most users. This means that well intentioned features and requests don't always make it through or in this case meet certain restrictions.
So with these problems in mind, the question is - does it still make sense to have sensor data in GPSLogger? I do like the idea of it, as long as it doesn't affect anything else. Hopefully you can see the reason for my worry now.
My issue around a separated sensorlog would be that joining the data would be more effort due to having to fetch data from a time window (last GPS timestamp until next timestamp) from the sensorlog instead of having it already joined upon creation.
I've just taken a look into http://www.topografix.com/GPX/1/0/gpx.xsd and found that trkpt does indeed have
<!--
you can add your own privately defined trkpt elements at the end of the trkpt
-->
<xsd:any namespace="##other" minOccurs="0" maxOccurs="unbounded"/>
in its schema. This would allow for inclusion of the custom namespace extensions I used in my current code. I would not put them into a separate extensions element thus avoiding the GPX11 structure while stil keeping the association of sensordata to the GPS stamps.
As for CSV I agree with you completely, which is why I have not yet touched it.
For my use case the sensor data as well as their association to GPS positions is pretty central, however for other users it might be an extra.
I suggest to integrate a matching warning as with the "Keep on GPS between fixes" feature to highlight the possible strong battery consumption. This way users are warned and can make informed decisions on what the need and want.
What do you think?
Well I'll be darned - I had no idea that was in there. You can work with something like GPX for years and still be learning new things. Yes that does solve it doesn't it - a custom namespace with some sensor data. That takes care of GPX, KML. I will also validate the output to make sure. Yeah leave CSV alone.
For my use case the sensor data as well as their association to GPS positions is pretty central, however for other users it might be an extra.
This is where the balancing comes in - everyone has a use case but no two people have the same use case. 😉 GPS Logger is first and foremost a GPS Logger, sensors are secondary. I think what's not immediately obvious in github's issue section is the amount of email support I deal with. Most users use email to communicate about this app rather than github - the use cases here are a small minority compared to what is requested (and abused) in my inbox.
I suggest to integrate a matching warning as with the "Keep on GPS between fixes" feature to highlight the possible strong battery consumption. This way users are warned and can make informed decisions on what the need and want.
What do you think?
It'll depend on the battery consumption. If the battery consumption is too high, any warning is not enough; users usually ignore warnings and often have a different interpretation of what high is. The battery consumption increase needs to be reasonable - if you put up your updated PR I can also try to grab and compile the APK then I will also test it. I have a few test phones just for testing battery consumption whenever I add some features.
In retrospect (hindsight is 20:20...) I think we should have taken a reverse approach or I should have suggested a reverse approach. Hardcode everything, do sensor logging, measure battery consumption first. Prepare sample GPX/KML file ensure they still validate and work with certain software. Then start breaking them up, productionizing/options/etc.
Do you already happen to have a sample GPX with custom sensor data?
Also I want to look at the multiple alarm managers, that's going to be a high overhead for me to maintain in the future so want to look at having everything drive off the existing alarm managers.
I've updated the PR as mentioned. Please have a look into it, I'd be happy to hear about your battery consumption numbers.
The most interesting parts are the GeneralEventListener and the GPSLoggingService.
More changes are in the KML and GPX writers, I've modified the GPX writer as discussed above.
Remaining changes are in detailed view fragment and a few supporting classes.
There are a number of parts marked as FIXME that contain planned changes. Also, at parts that may need context I've added a few notices on the context.
I've modified it such that only the one existing AlarmManager is required.
I also do have custom sensor data GPX files available, however, they contain specific location information, which is why I would prefer to send them to you by means other than the open GitHub discussion.
I have the PR code, built + put on my phones. What logging interval did you use? Right now unable to keep it running - the OS warns about high usage then kills it. I've tried 15seconds, 30 then 60s
My settings were as follows:
- 2s GPS point interval, only allowed location source GPS.
- 2 sensor samples between one GPS point (you'll need to adjust that, the default is at 10 which is probably ok with the 60s GPS point default, but way to high with a 2s GPS point interval).
- Between 5 and 10 sensor events to be catched. Currently there's a limit to 9 to prevent people setting it to 99.
I tested with with API level 22 and 23 in the emulator. The real phones in question had 6.0 Samsung images.
You could also try the ticke524-gradle-update branch from my repo. I've done most work there and then merged it into the PR branch.
I grabbed the ticket524-gradle-update branch, started running that by your settings. You are right, the battery usage is very high on all phones but on most it's being killed off after a while. I'm trying to run some debug logs to hopefully get a clue. I also had to delete some LOG.debug as the debug logs were reaching 400MB very quickly. I'll report back when I find something. There must be a reason this is happening.
I had a look through the code, I see it has multiple Alarm Managers, there are in-between sampling points durations for these as well. I hadn't realized there were other options like samples and durations too but that would likely explain a bit. In a way it's like another app embedded inside this app as it's behaving independently.
I don't believe any of this should be necessary right - just have it work off the main existing alarm, when that alarm reports then setup all the listeners and take just one sensor reading, then remove all listeners. No need for intermediate samples, or any duration.
First of all: Thank you for your review!
I'll answer inline on the rest.
I grabbed the ticket524-gradle-update branch, started running that by your settings. You are right, the battery usage is very high on all phones but on most it's being killed off after a while.
What causes the killing? I have not seen any sort of the service of application being killed on any of the phones I tested it with.
I'm trying to run some debug logs to hopefully get a clue. I also had to delete some LOG.debug as the debug logs were reaching 400MB very quickly.
The one time I had debug logs activated on an actual device I ended up with a huge 2,2GB debug log. Most of the debugging output can actually be removed, I've added FIXME notes where this is definitely the case at multiple locations in the code. Just throw them out, I mostly needed them to validate assumptions and increase my understanding of the code and Androids ways.
I had a look through the code, I see it has multiple Alarm Managers, there are in-between sampling points durations for these as well.
On the current release you should only see the nextPointAlarmManager being used for the different required alarms.
From my limited understanding of android this yields just one AlarmManager? What do you refer to when referring to multiple alarm managers?
I hadn't realized there were other options like samples and durations too but that would likely explain a bit. In a way it's like another app embedded inside this app as it's behaving independently.
The sensor data collection should be tightly coupled in terms of frequency to the collection of the GPS points.
I don't believe any of this should be necessary right - just have it work off the main existing alarm, when that alarm reports then setup all the listeners and take just one sensor reading, then remove all listeners. No need for intermediate samples, or any duration.
The problem here is that one sample is not enough - in the current way it will yield either a accelerometer sample or a magnetic field sample in a non-deterministic way. However both types of samples are required to calculate the desired output values
The issue is also described here
Also, depending on the timespan between single GPS points it can be of interest to get multiple output value samples. This leads to creation of additional alarm triggers between the GPS points. However, the actual alarms should run off of the main alarm manager. If the setting to control how many sensor samples are fetched between GPS points is set to 1 this should at maximum generate one additional trigger.
One possible approach to reduce collection of unnecessary samples could be to enable the accelerometer first, collect the sample, deactivate it, enable the magnetic field sensor, collect the sample, deactivate and then combine both samples instead of activating both sensors until all required samples have appeared.
I'll see if I can implement that.
You can also directly comment in the PR if you like, it's a nice and new GitHub feature that is pretty useful.
Closing this as part of v88 cleanup