Further BST time bug

Question

Further BST time bug

Closed this issue a month ago · 27 comments

Describe the bug

Target rate sensor is an hour out. Last night I had two slots between 03:00 & 04:00 but the sensor turned on battery charging between 04:00 & 05:00. Probably related to #815. I also find that the target rates are not generated as per #814 and only got slots last night after restarting Home Assistant at midnight

Reproduction steps

Configure target rate sensor to give two slots

Expected behaviour

Expect target sensor to switch on during the time slots allocated.

Tariff Code

E-1R-AGILE-23-12-06-E

Integration Version

10.1.4

Home Assistant Version

2024.2.1

Fresh Install?

Not specified

Home Assistant Logs

NA

Confirmation

I confirm that I cannot find my solution within the documentation
I confirm that I cannot find my solution within the FAQ

Answer 1 · 2024-04-03T08:55:59.000Z

Update, it's the way slots are displayed that is wrong, the front end showed

value_inc_vat: 0.095445
start: '2024-04-03T03:00:00+00:00'
end: '2024-04-03T03:30:00+00:00'
tariff_code: E-1R-AGILE-23-12-06-E
is_capped: false
value_inc_vat: 0.08715
start: '2024-04-03T03:30:00+00:00'
end: '2024-04-03T04:00:00+00:00'
tariff_code: E-1R-AGILE-23-12

These are the correct two lowest cost slots, but the wrong time is shown, the prices above were actually available between between 04:00 & 05:00 which was when the sensor turned on.

Answer 2 · 2024-04-03T18:46:41.000Z

I have the sane issue. Target Boolean sensors are late by one hour.

Answer 3 · 2024-04-04T21:09:27.000Z

same issue here, target rates are supposed to be be evaluated at 21:30 but they done get updated until 22:30

Once defined, the display is in GMT and so the switch takes place at the right time, thankfully :-) but only after a restart

Answer 4 · 2024-04-04T21:13:09.000Z

Update, it's the way slots are displayed that is wrong, the front end showed

value_inc_vat: 0.095445
start: '2024-04-03T03:00:00+00:00'
end: '2024-04-03T03:30:00+00:00'
tariff_code: E-1R-AGILE-23-12-06-E
is_capped: false

value_inc_vat: 0.08715
start: '2024-04-03T03:30:00+00:00'
end: '2024-04-03T04:00:00+00:00'
tariff_code: E-1R-AGILE-23-12

These are the correct two lowest cost slots, but the wrong time is shown, the prices above were actually available between between 04:00 & 05:00 which was when the sensor turned on.

if you notice the time shown is in GMT, what you see the sensor turning on at 04:00 and 05:00 is in BST, thats why it looks off by 1 hr but is actually correct as per the full time, "4:00:00+00:00" => this is 05:00 BST

Answer 5 · 2024-04-05T06:37:20.000Z

same issue here, target rates are supposed to be be evaluated at 21:30 but they done get updated until 22:30

Once defined, the display is in GMT and so the switch takes place at the right time, thankfully :-) but only after a restart

Not quite sure I follow. I have a 30 minute target, which I use to trigger a battery discharge to grid office have sufficient capacity left to get me through to turn next charge. This occurs at 18:30 to get the last applicable 30 minutes of the peak time on Flux export.

You're right in that the target times are in GMT but now we're in BST this means it's now triggering at 19:30 instead of 18:30.

Answer 6 · 2024-04-05T07:58:25.000Z

the triggering is at the right time, if you note below, rates are lowest at 2:30 bst, which corresponds to 1:30gmt, that’s what it’s shown in the target rate sensor, and it triggers at the right time.

In my example: slot 1 was at 1:30 gmt, triggered at 2:30 bst (start importing msg)

Answer 7 · 2024-04-05T10:06:39.000Z

I'm finding some variation going on. On my import target sensors, looking at my history the timings are performing perfectly, however my export target isn't working as expected, it's always one hour late.

I set up two test targets (one for import meter, one on export) and they are both one hour outside of the configured time. I configured a minimum start time of 08:30, and maximum stop of 10:00 and to get the last rate. Meaning I'd expect it to go true at 09:30. However, the sensors went true at 10:30.

On my normal export sensor in the images in my previous post, I have set a maximum stop time of 1900, but the sensor reacts at 18:30+00:00, making it 19:30 BST. Before BST kicked in, the sensor correctly reacted at 18:30+00:00 which matched the GMT we were in at the time.

Answer 8 · 2024-04-05T10:22:53.000Z

Looks like @BottlecapDave is going to become the proud owner of a migraine next time he logs in to GitHub 😬

Answer 9 · 2024-04-05T18:19:12.000Z

I didn't hit the issue with tracker rates not firing but then I did reset my instance to fix another DST bug.

I use target rate trackers dynamically using the service call to update them. It looks like the input start/end times are processed as GMT times rather than HA local times.

Answer 10 · 2024-04-06T19:52:01.000Z

Hello and sorry for the late response.

All times reported by the integration and by the OE APIs are in UTC format, and have been since the beginning. This matches GMT during the winter, but as noticed is an hour out in the summer. HA helpfully converts all date based attributes to local time when displayed in the entity dialogs, unless those dates are part of an object or array, which is why some dates are local time and some have the timezone displayed (helpful). If you look at the attributes in the developer tools or use them in automations, all datetimes are in UTC/GMT format. All internal calculations are done in UTC, so any picked times should come on at the time specified (adjusted for timezones).

I don't currently have any plan to change this behaviour at the present time as this would result in a breaking change as it's more than just visual and could be used in automation. This could result in some people's automations potientially breaking if they are relying on them in UTC format.

There is an issue as reported in #815, which I'll try and fix within the next couple of days. This is only an issue if target times fall on the 1 hour boundaries of the application time periods as it will be looking at the specified timeframe in UTC rather than local/BST (e.g. if you want a time from 16:00 to 21:00, it'll actually be looking at times between 17:00 to 22:00 local/BST). This will also cause issues if the cheapest rates fall on the boundaries as they might not be picked. This is made more apparent if the target time frames are small. This also has a knock on effect to target rate times being updated, as this only happens once all previous selected times are in the past.

@Meatballs1 Once the above is fixed, the service should also behave as expected.

Answer 11 · 2024-04-06T20:52:07.000Z

Yes, that’s what I thought was happening as well, I changed my settings to 8:30 gmt which is 9;30 bst , what I had previously. Can we have a time zone as well when we specify the schedule for target rates? Maybe that will clarify the confusion

Answer 12 · 2024-04-07T05:14:09.000Z

The from/to should always be in local time, which is the bug. Adding a timezone will just frustrate people twice a year when they have to update their target rate sensors and cause more bugs to be raised :)

Answer 13 · 2024-04-07T06:16:03.000Z

The underlying issue should now be fixed in v10.2.1 which should result in the sensor turning on/off during the requested times.

Answer 14 · 2024-04-12T15:41:55.000Z

I believe this isn't fixed. My current electricity rates are off by 1 hour still and i'm on 10.2.1

Answer 15 · 2024-04-12T15:55:26.000Z

I believe this isn't fixed. My current electricity rates are off by 1 hour still and i'm on 10.2.1

Hi, the rates were never a bug. The target time frame is in GMT and that’s how it’s always been. It will still trigger correctly as the rates you see in octopus are in local time which matches the gmt time in target rates.

Answer 16 · 2024-04-12T15:59:06.000Z

I believe this isn't fixed. My current electricity rates are off by 1 hour still and i'm on 10.2.1

Hi, the rates were never a bug. The target time frame is in GMT and that’s how it’s always been. It will still trigger correctly as the rates you see in octopus are in local time which matches the gmt time in target rates.

So forgive my stupidity...how do I make the current rate entity work for me? Do I need to set my timezone to always be GMT with no daylight savings?

Answer 17 · 2024-04-12T16:13:53.000Z

To clarify @deepanshu-sh

My current time is 1712 as of writing this mesage
My timezone in HA is set to GMT 00 (London)
HA thinks my time is 1712
The sensor.octopus_energy_electricity_xx_current_rate looks like this:

state_class: total
mpan: "xx"
serial_number: xx
is_export: false
is_smart_meter: true
tariff: E-1R-AGILE-23-12-06-G
start: "2024-04-12T16:00:00+00:00"
end: "2024-04-12T16:30:00+00:00"
is_capped: false
is_intelligent_adjusted: false
current_day_min_rate: -0.015645
current_day_max_rate: 0.2583
current_day_average_rate: 0.075453
data_last_retrieved: "2024-04-12T16:55:17.181984+01:00"
last_evaluated: "2024-04-12T16:09:18.182081+00:00"
unit_of_measurement: GBP/kWh
device_class: monetary
icon: mdi:currency-gbp
friendly_name: Electricity 22L3444132 1610012177566 Current Rate

As you can see it's offset incorrectly by 1 hour

Answer 18 · 2024-04-12T16:15:42.000Z

for this to work, you don't need to worry about current rates. The integration would install an entity something like, binary_sensor.octopus_energy_target_octopus_agile_import (this will be your own name you give when setting up the integration) and this entity will come on and off at the right time. then you use it in the automation like below:

Answer 19 · 2024-04-12T16:22:23.000Z

To clarify @deepanshu-sh

My current time is 1712 as of writing this mesage

My timezone in HA is set to GMT 00 (London)

HA thinks my time is 1712

The sensor.octopus_energy_electricity_xx_current_rate looks like this:
state_class: total
mpan: "xx"
serial_number: xx
is_export: false
is_smart_meter: true
tariff: E-1R-AGILE-23-12-06-G
start: "2024-04-12T16:00:00+00:00"
end: "2024-04-12T16:30:00+00:00"
is_capped: false
is_intelligent_adjusted: false
current_day_min_rate: -0.015645
current_day_max_rate: 0.2583
current_day_average_rate: 0.075453
data_last_retrieved: "2024-04-12T16:55:17.181984+01:00"
last_evaluated: "2024-04-12T16:09:18.182081+00:00"
unit_of_measurement: GBP/kWh
device_class: monetary
icon: mdi:currency-gbp
friendly_name: Electricity 22L3444132 1610012177566 Current Rate
As you can see it's offset incorrectly by 1 hour

your local time of HA is not GMT, its BST (British Summer Time, which is GMT +1) the clocks changed on 31 March.

Answer 20 · 2024-04-12T16:28:16.000Z

To clarify @deepanshu-sh

My current time is 1712 as of writing this mesage

My timezone in HA is set to GMT 00 (London)

HA thinks my time is 1712

The sensor.octopus_energy_electricity_xx_current_rate looks like this:
state_class: total
mpan: "xx"
serial_number: xx
is_export: false
is_smart_meter: true
tariff: E-1R-AGILE-23-12-06-G
start: "2024-04-12T16:00:00+00:00"
end: "2024-04-12T16:30:00+00:00"
is_capped: false
is_intelligent_adjusted: false
current_day_min_rate: -0.015645
current_day_max_rate: 0.2583
current_day_average_rate: 0.075453
data_last_retrieved: "2024-04-12T16:55:17.181984+01:00"
last_evaluated: "2024-04-12T16:09:18.182081+00:00"
unit_of_measurement: GBP/kWh
device_class: monetary
icon: mdi:currency-gbp
friendly_name: Electricity 22L3444132 1610012177566 Current Rate
As you can see it's offset incorrectly by 1 hour
your local time of HA is not GMT, its BST (British Summer Time, which is GMT +1) the clocks changed on 31 March.

No I don't think this is the problem.

When you set your system time you have 2 choices:

(GMT+00:00) London
(GMT+00:00) GMT (no daylight saving)

I chose option one because this is the option that takes daylight savings into account and is therefore showing the real current time (aka 1727 now).

I would expect this to mean that my current rate shows the window from 1700-1730, not 1600-1630

Answer 21 · 2024-04-12T16:30:49.000Z

"2024-04-12T16:00:00+00:00"
"2024-04-12T16:30:00+00:00"

is same as 5PM to 5:30 PM local time... which is BST

Answer 22 · 2024-04-12T16:32:46.000Z

Yes, and London is now in BST (+01:00). HA will still misleadingly show London GMT in the timezone selection, it's aware that London is now in BST and is showing you that time in Logs etc

Answer 23 · 2024-04-12T16:36:42.000Z

Yeah everything you guys are saying makes sense, I've realised that showing the from and to times in UTC didn't illustrate my point at all and made me look very stupid.

The issue I'm having is that my state value for the current rate matches the real value from 1 hour ago and is not the actual current value.

e.g. at 1630 it was saying the rate was still at the plunge rate from 1530

Answer 24 · 2024-04-13T09:15:56.000Z

@covertbert if you are facing a different BST bug, please raise a separate issue so it's easier for me to keep track with issues. Please raise it with a screenshot of the current rate sensors history during the time in question, along with your tariff.

Answer 25 · 2024-04-13T09:22:53.000Z

The inconsistency issues with how HA displays the time I believe is causing confusion over the behavior being incorrect in the majority of cases - e.g. top level items are displayed in local time, but sub items (e.g. rates in arrays) or viewing the data via developer tools is displayed in it's native value (UTC).

If different BST issues are found, it would be best to raise separate issues so that I can keep track easily of the areas that need investigation rather than a generic BST time bug issue.

I intend to try and have a look at better ways of displaying the information in local time. Unfortunately, the data that is used to display the information is also used to power automations etc so if changed it will be a breaking change. This will also take time due to the vast number of entities provided.

Answer 26 · 2024-04-28T07:32:08.000Z

There is a beta release of the next major release, including updating all attributes for sensors having timestamps converted to local timezone.

Answer 27 · 2024-05-12T14:47:34.000Z

All attributes should now be displayed in local time in v11.0.0. The majority of issues reported in this ticket were due to confusion in how the data was reported rather than it being wrong, so closing this ticket. If you still have an issue, please open a new ticket.