OxWearables/biobankAccelerometerAnalysis

GT3X expected time zone behaviour

angerhang opened this issue · 2 comments

At the moment, it is a bit unclear in terms of how to deal with the GT3X, time zone, day light saving and what parameters to pass in. Related issues are: wadpac/GGIR#580 and actigraph/GT3X-File-Format#20.

Will update the processing for GT3X time zone in the due course.

Ok getting things right for GT3X is really tricky because we need to accommodate two versions. The below is my first attempt:

For V1:

  • Obtain the start time from info.txt which contains the local_time only.
  • This starting timestamp will be used to calculate the time for every sample given the sampling frequency.
  • A user should supply a time zone to help to convert to UNIX time for the npy format. If no time zone is provided, we will default to UTC.

Test cases for V1:

java AccelerometerParser Tests/Resources/gt3xV1.gt3x npyOutput:True  # reference
java AccelerometerParser Tests/Resources/gt3xV1.gt3x npyOutput:True  rawOutput:True timeZone:Asia/Shanghai # npy doesn't change but epoch file moves forward by 7 hours
java AccelerometerParser Tests/Resources/gt3xV1.gt3x npyOutput:True  rawOutput:True timeZone:Asia/Shanghai offsetHour:08:00:00 # npy moves forward by 7 hours but epoch time is the same at the reference time

So it turns out that GT3X V2 also stores the UNIX time in local time format despite their email correspondence claims that the timestamps are stored in local time. Think this is likely a legacy issue from the C# tick time format. It is wrong and confusing to store timestamps in local time in a UNIX time.

For V2:

  • Each sample comes with its own timestamp in UNIX time. We will use the offsetHour flag to obtain the true UNIX time.
  • A user should supply a time zone to help to convert to UNIX time for the npy format. If no time zone is provided, we will default to UTC.

This leaves both parsing for V1 and V2 the same. Since we both only know the local time, offsetHour will determine how we can obtain the true UNIX time and the timeZone flag will determine how the epoch time is stored in the end. Both flags should be provided for GT3X.

java AccelerometerParser Tests/Resources/gt3xV1.gt3x npyOutput:True  # reference
java AccelerometerParser Tests/Resources/gt3xV1.gt3x npyOutput:True  rawOutput:True timeZone:Asia/Shanghai # npy doesn't change but epoch file moves forward by 7 hours
java AccelerometerParser Tests/Resources/gt3xV1.gt3x npyOutput:True  rawOutput:True timeZone:Asia/Shanghai offsetHour:08:00:00 # npy moves forward by 7 hours but epoch time is the same at the reference time

@angerhang I think the fix for this would be similar to your recent fix to GENEA devices? Or are they different case?