API for Prague public transport (CZ)

Question

API for Prague public transport (CZ)

Opened this issue 2 years ago · 10 comments

Hi,
Can you help me? My data source gives me data, see attachment. I am interested in the L175 route and the U738Z1P stop. According to the log, the integration normally loads the data from the API (it does not display any errors...) and the log contains a record of the L175. But the integration doesn't print any time. I am attaching the LOG sample below... We deleted the lines because they keep repeating themselves...

Does my API data contain the necessary information? Am I doing something wrong in configuration.yaml?

My settings:

sensor:
   - platform: gtfs_rt
     trip_update_url: 'https://api.golemio.cz/v2/vehiclepositions/gtfsrt/trip_updates.pb'
     vehicle_position_url: 'https://api.golemio.cz/v2/vehiclepositions/gtfsrt/vehicle_positions.pb'
     x_api_key: !secret PID_golemio_key
     departures:
     - name: 175 Bus
       route: L175
       stop id: U738Z1P
       icon: mdi:bus

sample LOG:

2022-12-15 13:57:06.942 INFO (SyncWorker_7) [custom_components.gtfs_rt.sensor] trip_update_url: https://api.golemio.cz/v2/vehiclepositions/gtfsrt/trip_updates.pb
2022-12-15 13:57:06.943 INFO (SyncWorker_7) [custom_components.gtfs_rt.sensor] vehicle_position_url: https://api.golemio.cz/v2/vehiclepositions/gtfsrt/vehicle_positions.pb
2022-12-15 13:57:06.943 INFO (SyncWorker_7) [custom_components.gtfs_rt.sensor] route_delimiter: _
2022-12-15 13:57:06.943 INFO (SyncWorker_7) [custom_components.gtfs_rt.sensor] header: {'x-api-key': 'XXXhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJlbWFpbCI6Im5sb0BzZXpuYW0uY3oiLCJpZCI6MTU3OSwibmFtZSI6bnVsbCwic3VybmFtZSI6bnVsbCwiaWF0IjoxNjcxMDI3ODQ0LCJleHAiOjExNjcxMDI3ODQ0LCJpc3MiOiJnb2xlbWlvIiwianRpIjoiMTdhYTQwNDItNGMwYi00Yzg3LWI1ZDAtZDgwZThlYjcwMjY4In0.TjwI1gSAGqnPJCrqVdGZSR6HErGMu1VYCl__BJrGXXX'}
2022-12-15 13:57:08.369 INFO (SyncWorker_7) [custom_components.gtfs_rt.sensor] Successfully updated vehicle positions - 200
2022-12-15 13:57:12.636 DEBUG (SyncWorker_7) [custom_components.gtfs_rt.sensor] ......Adding position for trip id 128_25_220103 position latitude 50.04314041137695 longitude 14.405719757080078
2022-12-15 13:57:12.637 DEBUG (SyncWorker_7) [custom_components.gtfs_rt.sensor] ......Adding position for trip id 128_24_220103 position latitude 50.04314041137695 longitude 14.405719757080078
2022-12-15 13:57:12.637 DEBUG (SyncWorker_7) [custom_components.gtfs_rt.sensor] ......Adding position for trip id 107_1651_220926 position latitude 50.100868225097656 longitude 14.394060134887695
2022-12-15 13:57:12.637 DEBUG (SyncWorker_7) [custom_components.gtfs_rt.sensor] ......Adding position for trip id 107_1654_220926 position latitude 50.100868225097656 longitude 14.394060134887695
2022-12-15 13:57:12.637 DEBUG (SyncWorker_7) [custom_components.gtfs_rt.sensor] ......Adding position for trip id 107_1652_220926 position latitude 50.100868225097656 longitude 14.394060134887695
2022-12-15 13:57:12.638 DEBUG (SyncWorker_7) [custom_components.gtfs_rt.sensor] ......Adding position for trip id 107_1653_220926 position latitude 50.100868225097656 longitude 14.394060134887695
2022-12-15 13:57:12.638 DEBUG (SyncWorker_7) [custom_components.gtfs_rt.sensor] ......Adding position for trip id 128_23_220103 position latitude 50.04314041137695 longitude 14.405719757080078
2022-12-15 13:57:12.638 DEBUG (SyncWorker_7) [custom_components.gtfs_rt.sensor] ......Adding position for trip id 129_92_221026 position latitude 49.95600128173828 longitude 14.3816499710083
2022-12-15 13:57:12.638 DEBUG (SyncWorker_7) [custom_components.gtfs_rt.sensor] ......Adding position for trip id 298_30_221128 position latitude 50.127830505371094 longitude 14.379019737243652
2022-12-15 13:57:12.638 DEBUG (SyncWorker_7) [custom_components.gtfs_rt.sensor] ......Adding position for trip id 340_382_220901 position latitude 50.127830505371094 longitude 14.379019737243652
2022-12-15 13:57:12.638 DEBUG (SyncWorker_7) [custom_components.gtfs_rt.sensor] ......Adding position for trip id 129_91_211101 position latitude 49.95600128173828 longitude 14.3816499710083
2022-12-15 13:57:12.639 DEBUG (SyncWorker_7) [custom_components.gtfs_rt.sensor] ......Adding position for trip id 170_2111_220905 position latitude 50.03649139404297 longitude 14.386899948120117
2022-12-15 13:57:12.639 DEBUG (SyncWorker_7) [custom_components.gtfs_rt.sensor] ......Adding position for trip id 170_2040_220901 position latitude 50.03649139404297 longitude 14.386899948120117
...
2022-12-15 13:57:15.429 INFO (SyncWorker_7) [custom_components.gtfs_rt.sensor] Successfully updated trip data - 200
2022-12-15 13:57:38.534 DEBUG (SyncWorker_7) [custom_components.gtfs_rt.sensor] ...Feed Route Id L128 changed to L128
2022-12-15 13:57:38.561 DEBUG (SyncWorker_7) [custom_components.gtfs_rt.sensor] ...Feed Route Id L128 changed to L128
2022-12-15 13:57:38.561 DEBUG (SyncWorker_7) [custom_components.gtfs_rt.sensor] ...Feed Route Id L107 changed to L107
2022-12-15 13:57:38.562 DEBUG (SyncWorker_7) [custom_components.gtfs_rt.sensor] ...Feed Route Id L107 changed to L107
2022-12-15 13:57:38.563 DEBUG (SyncWorker_7) [custom_components.gtfs_rt.sensor] ...Feed Route Id L107 changed to L107
2022-12-15 13:57:38.563 DEBUG (SyncWorker_7) [custom_components.gtfs_rt.sensor] ...Feed Route Id L107 changed to L107
2022-12-15 13:57:38.565 DEBUG (SyncWorker_7) [custom_components.gtfs_rt.sensor] ...Feed Route Id L128 changed to L128
2022-12-15 13:57:38.565 DEBUG (SyncWorker_7) [custom_components.gtfs_rt.sensor] ...Feed Route Id L129 changed to L129
2022-12-15 13:57:38.566 DEBUG (SyncWorker_7) [custom_components.gtfs_rt.sensor] ...Feed Route Id L298 changed to L298
2022-12-15 13:57:38.606 DEBUG (SyncWorker_7) [custom_components.gtfs_rt.sensor] ...Feed Route Id L340 changed to L340
2022-12-15 13:57:38.607 DEBUG (SyncWorker_7) [custom_components.gtfs_rt.sensor] ...Feed Route Id L129 changed to L129
2022-12-15 13:57:38.608 DEBUG (SyncWorker_7) [custom_components.gtfs_rt.sensor] ...Feed Route Id L170 changed to L170
2022-12-15 13:57:38.608 DEBUG (SyncWorker_7) [custom_components.gtfs_rt.sensor] ...Feed Route Id L170 changed to L170
2022-12-15 13:57:39.605 DEBUG (SyncWorker_7) [custom_components.gtfs_rt.sensor] ...Feed Route Id L152 changed to L152
2022-12-15 13:57:39.606 DEBUG (SyncWorker_7) [custom_components.gtfs_rt.sensor] ...Feed Route Id L125 changed to L125
2022-12-15 13:57:39.607 DEBUG (SyncWorker_7) [custom_components.gtfs_rt.sensor] ...Feed Route Id L125 changed to L125
2022-12-15 13:57:39.621 DEBUG (SyncWorker_7) [custom_components.gtfs_rt.sensor] ...Feed Route Id L197 changed to L197
2022-12-15 13:57:39.624 DEBUG (SyncWorker_7) [custom_components.gtfs_rt.sensor] ...Feed Route Id L197 changed to L197
2022-12-15 13:57:39.629 DEBUG (SyncWorker_7) [custom_components.gtfs_rt.sensor] ...Feed Route Id L152 changed to L152
2022-12-15 13:57:39.630 DEBUG (SyncWorker_7) [custom_components.gtfs_rt.sensor] ...Feed Route Id L128 changed to L128
2022-12-15 13:57:39.654 DEBUG (SyncWorker_7) [custom_components.gtfs_rt.sensor] ...Feed Route Id L152 changed to L152
2022-12-15 13:57:39.655 DEBUG (SyncWorker_7) [custom_components.gtfs_rt.sensor] ...Feed Route Id L108 changed to L108
2022-12-15 13:57:39.668 DEBUG (SyncWorker_7) [custom_components.gtfs_rt.sensor] ...Feed Route Id L174 changed to L174
2022-12-15 13:57:39.669 DEBUG (SyncWorker_7) [custom_components.gtfs_rt.sensor] ...Feed Route Id L193 changed to L193
2022-12-15 13:57:39.703 DEBUG (SyncWorker_7) [custom_components.gtfs_rt.sensor] ...Feed Route Id L193 changed to L193
2022-12-15 13:57:39.704 DEBUG (SyncWorker_7) [custom_components.gtfs_rt.sensor] ...Feed Route Id L108 changed to L108
2022-12-15 13:57:39.705 DEBUG (SyncWorker_7) [custom_components.gtfs_rt.sensor] ...Feed Route Id L129 changed to L129
...
2022-12-15 13:57:54.489 DEBUG (SyncWorker_7) [custom_components.gtfs_rt.sensor] Sensor Update:
2022-12-15 13:57:54.490 DEBUG (SyncWorker_7) [custom_components.gtfs_rt.sensor] ...Name: 175 Bus
2022-12-15 13:57:54.490 DEBUG (SyncWorker_7) [custom_components.gtfs_rt.sensor] ...Route: L175
2022-12-15 13:57:54.490 DEBUG (SyncWorker_7) [custom_components.gtfs_rt.sensor] ...Stop ID: U738Z1P
2022-12-15 13:57:54.490 DEBUG (SyncWorker_7) [custom_components.gtfs_rt.sensor] ...Icon: mdi:bus
2022-12-15 13:57:54.490 DEBUG (SyncWorker_7) [custom_components.gtfs_rt.sensor] ...Service Type: Bus
2022-12-15 13:57:54.491 DEBUG (SyncWorker_7) [custom_components.gtfs_rt.sensor] ...unit_of_measurement: min
2022-12-15 13:57:54.491 DEBUG (SyncWorker_7) [custom_components.gtfs_rt.sensor] ...Due in: -
2022-12-15 13:57:54.491 DEBUG (SyncWorker_7) [custom_components.gtfs_rt.sensor] ...Due at not defined
2022-12-15 13:57:54.491 DEBUG (SyncWorker_7) [custom_components.gtfs_rt.sensor] ...latitude not defined
2022-12-15 13:57:54.491 DEBUG (SyncWorker_7) [custom_components.gtfs_rt.sensor] ...longitude not defined
2022-12-15 13:57:54.492 DEBUG (SyncWorker_7) [custom_components.gtfs_rt.sensor] ...Next Bus not defined

Thanks for any help or solution

API_data.zip

Answer 1 · 2022-12-18T07:31:37.000Z

Hi,
Thanks for reporting your issue. After doing some analysis it looks as if golemio are only publishing vehicle positions and not stop times. Here is an example of what is received for your route L306 bus in Prague.

WARNING:main:...Received Trip Id 306_156_210904 Route Id: L306 Start Time: 08:14:00 Start Date: 20221218
WARNING:main:......Trip Route Id L306 changed to L306
WARNING:main:......Stop: U2115Z2 Stop Sequence: 1 Stop Time: 0
WARNING:main:......Stop: U1145Z1 Stop Sequence: 2 Stop Time: 0
WARNING:main:......Stop: U1218Z1 Stop Sequence: 3 Stop Time: 0
WARNING:main:......Stop: U1146Z2 Stop Sequence: 4 Stop Time: 0
WARNING:main:......Stop: U1147Z1 Stop Sequence: 5 Stop Time: 0
WARNING:main:......Stop: U1144Z4 Stop Sequence: 6 Stop Time: 0
WARNING:main:......Stop: U1143Z2 Stop Sequence: 7 Stop Time: 0
WARNING:main:......Stop: U687Z4 Stop Sequence: 8 Stop Time: 0
WARNING:main:......Stop: U1148Z2 Stop Sequence: 9 Stop Time: 0
WARNING:main:......Stop: U35Z2 Stop Sequence: 10 Stop Time: 0
WARNING:main:......Stop: U1209Z2 Stop Sequence: 11 Stop Time: 0
WARNING:main:......Stop: U913Z2 Stop Sequence: 12 Stop Time: 0
WARNING:main:......Stop: U178Z2 Stop Sequence: 13 Stop Time: 0
WARNING:main:......Stop: U1141Z1 Stop Sequence: 14 Stop Time: 0

You will notice that all Stop Times are 0. For other GTFS realtime providers the stop time attributes are populated with the expected arrival time at each stop. Golemio do seem to publish the geographic position of the vehicle on the route:

WARNING:main:......Adding position for trip id 306_156_210904 position latitude 50.08327865600586 longitude 14.232150077819824

Unfortunately, as it is currently written, the GTFS-RT integration does not populate the sensor with geographic location unless the stop time is received. So, you can't even show the sensor on a map sorry.

Perhaps you could check with Golemio as to why they are not publishing stop times with the GTFS-RT service.

Regards,

Mark

Answer 2 · 2023-01-09T22:41:40.000Z

Hi,
Were you able to contact Golemio and ascertain why they are not publishing stop times with the GTFS-RT service?
Mark

Answer 3 · 2023-01-12T13:16:59.000Z

Hi,
sorry for the late reply. I didn't have time to play with it.

Thank you for your help and information about what is wrong. If we understood it correctly, there is no error in your integration, but there is an error in the data source.

Unfortunately I have no information from Golemio why they don't publish stop times. I tried to find another end point on the Golemio API:
https://api.golemio.cz/v2/pid/docs/openapi/#/

with the necessary data for your integration, but without success.

In the end, I used the accessible REST API "PID Departure Boards" where you can get a specific stop/stops for the departure times of the nearest lines, including delays, etc... That's exactly what I'm interested in, the position of the vehicle is not important to me.

Thank you and maybe in the future they will start publishing the data correctly and I will use the integration.

Answer 4 · 2023-10-15T15:43:35.000Z

I remember I spoke with Golemio about it but on this specific topic, I did not get any answer. neverthless at the end I've modified the integration and adjusted to be used with Golemio one.

Answer 5 · 2023-10-15T16:32:16.000Z

@mazzy89 can you share? I'm not skilled enough to do it for myself.

Answer 6 · 2023-12-10T13:53:06.000Z

@kalda123 i'm planning to wire a contribution to support that.

Answer 7 · 2024-05-21T12:06:57.000Z

Hi,
I have finally found some time to look at this and work on a solution. GTFS and GTFS Realtime is hard to work with because the standard gives a lot of flexibility to the transport operator as how they implement it. There are very few mandatory attributes but lots of "conditionally required" ones (i.e. provide x when y = z or provide x or y but not both). It makes it hard to write something that is going to work consistently across a number of operators in different countries.
Building on some work done by others I have been working on a new version that incorporates data from the GTFS static files to fill in missing data from the realtime feed, like the stop time in this case.
I have some test code working that caters for the missing stop time and adds some other attributes to the sensor. Here is an example for Prague.

Name: L306 Bus
Route: L175
Stop ID: U738Z1P
Direction ID: 0
Icon: mdi:bus
Service Type: bus
unit_of_measurement: min
Due in: 4
Trip ID: 175_1741_240219
Due at: 21:59
Headsign: Háje
Accessibility: ACCOMODATED
Bikes: NOT_ALLOWED
Route Name: Florenc - Háje
Stop Name: Štěchovická
Latitude: 50.07304000854492
Longitude: 14.48091983795166
Occupancy Status: NO_DATA_AVAILABLE
Next bus: 22:17

I still have some work to do to incorporate in into a new version of the HA integration. And to be honest I have some concerns about how long it will take to refresh and my very ordinary programming skills. For these reasons I will probably do a number of beta releases. Also, it will require the users to periodically provide an updated version of the GTFS static data in a Sqlite database (I will provide instructions on how to do this).

Hope you find this useful.

Mark

Answer 8 · 2024-05-21T21:05:36.000Z

@mark1foley realtime gtfs is well supported and Prague PID have a stable API. I have a forked modified version of this library hosted on my internal Gitea instance which works for Prague PID real-time API and the changes were pretty minimal. I will try to publish. I was using before the static gtfs downloading periodically the zipped file but gtfs is much more reliable.

Answer 9 · 2024-05-27T07:11:07.000Z

@mazzy89, apologies I didn't mean to imply that GTFS is not well supported or stable. Obviously, it is both of those things. My point was that there is a lot of variation in how individual operators have done their implementations because the standard gives them the flexibility to do so. It makes it very hard to write something that works with a lot of different operators.
I would be interested in seeing what you have done. I am not thrilled about relying on the GTFS static file but if the operator chooses not provide the stop time in the realtime update (which is allowed under the standard) then I see little choice then to retrieve it from the static file. If you have a different way then I would love to know more about that.

Answer 10 · 2024-05-27T13:43:10.000Z

Here a gist adapted to support PID Prague Transport using the GTFS RealTime API. For me works well https://gist.github.com/mazzy89/c90ace982492ab7d5ba179027742b7e6