bug: UTC (Zulu) events seem to not be converted to local time (X-WR-TIMEZONE)
Closed this issue · 6 comments
Describe the bug
I am processing a Google Calendar ICS file where some events include full timezone and then the local time, while others, for reasons I am not really sure about, only list the time in Zulu format. Instead of converting these zulu-events to local time, they just get an offset of 0 and so in my case seem to occur two hours prior to the actual time.
To Reproduce
I have added a couple of events below, where one has full timezone and the others do not, including the start that defines my timezone. I commented out writing to a file and only did debug to screen in this case.
I expect the magic ought to happen in component.get('dtstart').dt and component.get('dtend').dt
It might be an icalendar thing more than yours. If so, I apologize.
This is the function I call to parse the events:
def open_cal():
if os.path.isfile(filename):
if file_extension == 'ics':
print("Extracting events from file:", filename, "\n")
f = open(sys.argv[1], 'rb')
gcal = Calendar.from_ical(f.read())
revents = recurring_ical_events.of(gcal).between(istart,istop)
# for component in gcal.walk():
for component in revents:
event = CalendarEvent("event")
v=(dir(component).count('get')) # Only proces data if object is a valid event
if (v != 0):
if component.get('TRANSP') == 'TRANSPARENT': continue #skip all day events and the like
if component.get('SUMMARY') == None: continue #skip blank items
event.summary = component.get('SUMMARY')
event.uid = component.get('UID')
if component.get('DESCRIPTION') == None: continue #skip blank items
event.description = component.get('DESCRIPTION')
event.location = component.get('LOCATION')
if hasattr(component.get('dtstart'), 'dt'):
event.start = component.get('dtstart').dt
if hasattr(component.get('dtend'), 'dt'):
event.end = component.get('dtend').dt
event.url = component.get('URL')
events.append(event)
f.close()
else:
print("You entered ", filename, ". ")
print(file_extension.upper(), " is not a valid file format. Looking for an ICS file.")
exit(0)
else:
print("I can't find the file ", filename, ".")
print("Please enter an ics file located in the same folder as this script.")
exit(0)
Full script is at: https://github.com/martinm76/ical2csv/blob/sortedevents/ical2txt.py
ICS file
BEGIN:VCALENDAR
PRODID:-//Google Inc//Google Calendar 70.9054//EN
VERSION:2.0
CALSCALE:GREGORIAN
METHOD:PUBLISH
X-WR-CALNAME:martin.moller@pwc.com
X-WR-TIMEZONE:Europe/Copenhagen
BEGIN:VTIMEZONE
TZID:Europe/Copenhagen
X-LIC-LOCATION:Europe/Copenhagen
BEGIN:DAYLIGHT
TZOFFSETFROM:+0100
TZOFFSETTO:+0200
TZNAME:CEST
DTSTART:19700329T020000
RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=-1SU
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:+0200
TZOFFSETTO:+0100
TZNAME:CET
DTSTART:19701025T030000
RRULE:FREQ=YEARLY;BYMONTH=10;BYDAY=-1SU
END:STANDARD
END:VTIMEZONE
BEGIN:VTIMEZONE
TZID:Europe/Amsterdam
X-LIC-LOCATION:Europe/Amsterdam
BEGIN:DAYLIGHT
TZOFFSETFROM:+0100
TZOFFSETTO:+0200
TZNAME:CEST
DTSTART:19700329T020000
RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=-1SU
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:+0200
TZOFFSETTO:+0100
TZNAME:CET
DTSTART:19701025T030000
RRULE:FREQ=YEARLY;BYMONTH=10;BYDAY=-1SU
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTART:20210107T120000Z
DTEND:20210107T123000Z
DTSTAMP:20210419T094655Z
UID:46d2pth49elq9stq4fun6f8nel@google.com
CREATED:20210107T121525Z
DESCRIPTION:
LAST-MODIFIED:20210107T121525Z
LOCATION:
SEQUENCE:0
STATUS:CONFIRMED
SUMMARY:RSYD - Rapport-afklaringer med Lars
TRANSP:OPAQUE
END:VEVENT
BEGIN:VEVENT
DTSTART:20210107T150000Z
DTEND:20210107T153000Z
DTSTAMP:20210419T094655Z
UID:0rjth8ik4nar2kmkvfd8dqu417@google.com
CREATED:20210107T152038Z
DESCRIPTION:
LAST-MODIFIED:20210107T152038Z
LOCATION:
SEQUENCE:0
STATUS:CONFIRMED
SUMMARY:DGH - Hvordan får man en ethernet-only enhed til at komme på WiFi?
TRANSP:OPAQUE
END:VEVENT
BEGIN:VEVENT
DTSTART:20210107T143000Z
DTEND:20210107T150000Z
DTSTAMP:20210419T094655Z
UID:05n9bm0gu5nhb9iotrhil5edrk@google.com
CREATED:20210107T152052Z
DESCRIPTION:
LAST-MODIFIED:20210107T152052Z
LOCATION:
SEQUENCE:0
STATUS:CONFIRMED
SUMMARY:Zabbix
TRANSP:OPAQUE
END:VEVENT
BEGIN:VEVENT
DTSTART;TZID=Europe/Copenhagen:20200629T090000
DTEND;TZID=Europe/Copenhagen:20200629T093000
DTSTAMP:20210419T094655Z
UID:k1gfio4b0coichhg609kevd5g4_R20200403T070000@google.com
RECURRENCE-ID;TZID=Europe/Copenhagen:20200629T090000
CREATED:20190913T124856Z
DESCRIPTION:-::~:~::~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~
:~:~:~:~:~:~:~:~::~:~::-\nPlease do not edit this section of the descriptio
n.\n\nThis event has a video call.\nJoin: https://meet.google.com/iam-sdfz-
ang\n(DK) +45 22 22 22 22 PIN: 222222#\nView more joining options: https://
tel.meet/XXX-YYYY-ZZZ?pin=xxxxxxxxxxxxx&hs=7\n-::~:~::~:~:~:~:~:~:~:~:~:~:~
:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~::~:~::-
LAST-MODIFIED:20210111T154744Z
LOCATION:
SEQUENCE:2
STATUS:CONFIRMED
SUMMARY:Standup
TRANSP:OPAQUE
BEGIN:VALARM
ACTION:DISPLAY
DESCRIPTION:This is an event reminder
TRIGGER:-P0DT0H15M0S
END:VALARM
END:VEVENT
BEGIN:VEVENT
DTSTART:20210407T070000Z
DTEND:20210407T073000Z
DTSTAMP:20210419T094655Z
UID:1vl7105mu73rkvjbj6ngt5pddo@google.com
CREATED:20210407T131456Z
DESCRIPTION:
LAST-MODIFIED:20210407T131456Z
LOCATION:
SEQUENCE:0
STATUS:CONFIRMED
SUMMARY:Post og opdateringer + chat
TRANSP:OPAQUE
END:VEVENT
BEGIN:VEVENT
DTSTART:20210407T073000Z
DTEND:20210407T080000Z
DTSTAMP:20210419T094655Z
UID:44hrnlmiibllp7e3rvb0ifv9q6@google.com
CREATED:20210407T131502Z
DESCRIPTION:
LAST-MODIFIED:20210407T131502Z
LOCATION:
SEQUENCE:0
STATUS:CONFIRMED
SUMMARY:ServiceDesk
TRANSP:OPAQUE
END:VEVENT
END:VCALENDAR
Expected behavior
I had expected the events to get either +01:00 or +02:00 as timezone information rather than +00:00.
Yes, they are specified in UTC time, but the calendar is in Europe/Copenhagen timezone, so in my view, the events should be, too. They show correctly in Google Calendard, where I exported them from.
Console output
./ical2txt.py mm-test.ics
Extracting events from file: mm-test.ics
Contents of event :
Standup
k1gfio4b0coichhg609kevd5g4_R20200403T070000@google.com
-::~:~::~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~::~:~::-
Please do not edit this section of the description.
This event has a video call.
Join: https://meet.google.com/iam-sdfz-ang
(DK) +45 22 22 22 22 PIN: 222222#
View more joining options: https://tel.meet/XXX-YYYY-ZZZ?pin=xxxxxxxxxxxxx&hs=7
-::~:~::~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~:~::~:~::-
2020-06-29 09:00:00+02:00
2020-06-29 09:30:00+02:00
None
Contents of event :
RSYD - Rapport-afklaringer med Lars
46d2pth49elq9stq4fun6f8nel@google.com
2021-01-07 12:00:00+00:00
2021-01-07 12:30:00+00:00
None
Contents of event :
Zabbix
05n9bm0gu5nhb9iotrhil5edrk@google.com
2021-01-07 14:30:00+00:00
2021-01-07 15:00:00+00:00
None
Contents of event :
DGH - Hvordan får man en ethernet-only enhed til at komme på WiFi?
0rjth8ik4nar2kmkvfd8dqu417@google.com
2021-01-07 15:00:00+00:00
2021-01-07 15:30:00+00:00
None
Contents of event :
Post og opdateringer + chat
1vl7105mu73rkvjbj6ngt5pddo@google.com
2021-04-07 07:00:00+00:00
2021-04-07 07:30:00+00:00
None
Contents of event :
ServiceDesk
44hrnlmiibllp7e3rvb0ifv9q6@google.com
2021-04-07 07:30:00+00:00
2021-04-07 08:00:00+00:00
None
Version:
Additional context
pip3 list
Package Version
--------------------- ---------
beautifulsoup4 4.9.3
certifi 2020.6.20
chardet 4.0.0
dblatex 0.3.12
icalendar 4.0.7
idna 2.10
olefile 0.46
Pillow 8.1.2
pip 21.0
Pygments 2.7.3
python-dateutil 2.8.1
pytz 2021.1
PyYAML 5.3.1
recurring-ical-events 0.1.21b0
requests 2.24.0
setuptools 51.1.1
six 1.15.0
soupsieve 2.2.1
toggl-cli 3
urllib3 1.25.10
Suggested implementation
I would like the events to automatically get local timezone added if they are in UTC format, at least as an option.
I might be able to manually add the 7200 seconds in the current code, but would be nice to have a global fix (unless of course you believe it's working as intended and Google is doing it wrong).
During the parsing of the calendar, there is timezone information at the top, which I would expect to be applied as default, if no timezone is explicitly given.
Update: Not sure how this fixes it, but apparently, if I cast to astimezone() after I get my start and end dates, I get the correct date and time in my calendar, seemingly no matter how the event was formatted in the ics file!
for component in revents:
event = CalendarEvent("event")
v=(dir(component).count('get')) # Only proces data if object is a valid event
if (v != 0):
if component.get('TRANSP') == 'TRANSPARENT': continue #skip all day events and the like
if component.get('SUMMARY') == None: continue #skip blank items
event.summary = component.get('SUMMARY')
event.uid = component.get('UID')
if component.get('DESCRIPTION') == None: continue #skip blank items
event.description = component.get('DESCRIPTION')
event.location = component.get('LOCATION')
if hasattr(component.get('dtstart'), 'dt'):
event.start = component.get('dtstart').dt
if hasattr(component.get('dtend'), 'dt'):
event.end = component.get('dtend').dt
event.start=event.start.astimezone()
event.end=event.end.astimezone()
event.url = component.get('URL')
events.append(event)
f.close()
I guess that solves the issue for me. If it is a more systemic issue that you have any power over, I hope this helps a little.
It looks like the X-WR-TIMEZONE attribute is interpreted differently. Some ignore it, some do not. Source
I guess it would be ok to go through the calendar before parsing it or after when the events are there and assign time zones to the events.
This is related: #63
What we could do, though, is that we add something to this module which allows the parsing/using of X-WR-TIMEZONE.
What do you think?
Update: Not sure how this fixes it, but apparently, if I cast to astimezone() after I get my start and end dates, I get the correct date and time in my calendar, seemingly no matter how the event was formatted in the ics file!
for component in revents: event = CalendarEvent("event") v=(dir(component).count('get')) # Only proces data if object is a valid event if (v != 0): if component.get('TRANSP') == 'TRANSPARENT': continue #skip all day events and the like if component.get('SUMMARY') == None: continue #skip blank items event.summary = component.get('SUMMARY') event.uid = component.get('UID') if component.get('DESCRIPTION') == None: continue #skip blank items event.description = component.get('DESCRIPTION') event.location = component.get('LOCATION') if hasattr(component.get('dtstart'), 'dt'): event.start = component.get('dtstart').dt if hasattr(component.get('dtend'), 'dt'): event.end = component.get('dtend').dt event.start=event.start.astimezone() event.end=event.end.astimezone() event.url = component.get('URL') events.append(event) f.close()I guess that solves the issue for me. If it is a more systemic issue that you have any power over, I hope this helps a little.
Won't astimezone use the local timezone of the machine you are running on, so If the ics is processed in a different timezone, wouldn't it be inaccurate? It does seem for events added to the calendar without a TZ that the default X-WR-TIMEZONE would make sense (if it's in the standard).
A list of other discussions:
- ical4j/ical4j#27 raising the issue
- mangstadt/biweekly#90
discussing it - https://github.com/mangstadt/biweekly/wiki/Timezones#x-wr-timezone
This says this might be good to help displaying the calendar.
For me, it seems that if you find that property, it does not make a difference to unfolding the events. - jemproject/JEM-Project#585
- Example: https://github.com/jazzband/icalevents/tree/c368ad8ec4981f58754faba3ef2668cd176d1350/icalevents/icalparser.py#L320
It is true that the X-WR-TIMEZONE property is not understood by this library at this point in time. Ideally, the calendar is modified for support of X-WR-TIMEZONE before it is used by this library - if you really need that.
My proposal for a fix is:
- Find if there is a X-WR-TIMEZONE property present.
- Use
between()anat()as follows, depending on their arguments:- If the arguments are
datetimewith time zone: First go through all the events in the calendar, modify DTSTART, DTEND, RRULE, RDATE, RECURRENCE-ID, EXDATE if you need precise querying (events move several hours because of time zones and might be included or excluded on the edges of the time span specified in the arguments). - If the arguments are
dateordatetimewithout time zone: When you get the resulting events and use them, put the time zone from the X-WR-TIMEZONE property into thedatetimeobjects which need it. Be sure to notice that you have events of several different time zones in the resulting list. If you display the events in a different time zone than that of X-WR-TIMEZONE, add a day in front and at the end of the arguments to make sure you include all required events.
- If the arguments are
Feel free to open a new issue or discuss this one further. According to my research, I will close this issue now. It can be reopened it necessary or a new discussion started.
My suggested implementation would be a module which handles these use-cases.
As soon as this library supports the VTIMEZONE definitions, understanding X-WR-TIMEZONE might be under discussion again.
I opened #71 to discuss it because you are not the only one facing this.