GTFS RT not updating correctly

[mrulrich1]

Describe the bug
Gtfs rt updated at is showing an update but there is no value for Next departure realtime

Steps/data to reproduce the behavior, e.g.

• url to the zip file : https://gtfsbridge.spokanetransit.com/realtime/u/mrulrich
• route ID: 45
• stop ID: PER8THNN
• return

Release used
0.4.3 HAOS in Proxmox VM

Additional
I'm not sure if the integration is reading in the RT data correctly, but it does show that it's updating. I'm confirming actual times against a known working app.

[vingerha]

Hi, the label is a bit misleading... my idea was to indicate that the process to collect data runs. The non-presence of time is usually related to lack of data from the source. I cannot analyse each source as you understand so try and use the service call to download it (protobuf format) to www/gtfs2 and you can then try to find the route/trip in the readable format to see if there indeed is no data

https://github.com/vingerha/gtfs2/wiki/4.-Service-call

[mrulrich1]

so try and use the service call to download it (protobuf format) to www/gtfs2 and you can then try to find the route/trip in the readable format to see if there indeed is no data

Is it possible the integration is dropping data? The stop is included in the real time *.json file from the transit provider. However, when I run a service call and check the converted *_converted.txt file, it's not there. Somehow it's getting dropped...

[vingerha]

I am using an existing library to convert from protobuf to something more readable. I have not yet encountered this before but I for sure cannot guarantee this to be 100% correct...and I sincerely hope this issue is not the case.
At present I cannot find any entry for PER8THNN ...please attach both the rt and txt files where you see discrepancies

[mrulrich1]

Here's an example of the stop being present in both while the integration isn't reporting Next departure realtime

STA.rt.txt
STA_converted.txt

Shouldn't it be reporting? Or am I missing something? (very likely)

[vingerha]

Hi, both files contain 2x PER8THNN. In one group it says 'skipped' so nothing will be shown. The other shows a timestamp of GMT Saturday, 6 April 2024 20:18:55 which I think is 13:18 your time. I am not sure if this was 'old' at the time you sent this. I have at least two RT providers that for some sort of odd reason keep sending historical rt-times. You also need to check if it is for the correct route / trip id which in this case seems ok (45)

trip_id: "982392"
route_id: "45"
start_time: "13:13:00"
start_date: "20240406"

[mrulrich1]

You also need to check if it is for the correct route / trip id which in this case seems ok (45)

45 is definitely correct. I'll take a closer look and see if the provider is reporting a historical time or current.

[mrulrich1]

I've done some more testing. This is straight from the *_converted.txt file.

I put the local time of the service call in bold.

In all cases the Next departure realtime attribute never updated.

In some cases, the entity state would change to the next scheduled time in the GTFS database before the RT data indicated the previous bus was supposed to arrive.

It's possible I've got something configured wrong but don't know what that would be.

7:49
stop_time_update {
stop_sequence: 9
stop_id: "PER8THNN"
departure {
time: 1712589498
}
stop_time_properties {
}

stop_time_update {
stop_sequence: 9
stop_id: "PER8THNN"
departure {
time: 1712588598
}
stop_time_properties {
}

13:24
stop_time_update {
stop_sequence: 9
stop_id: "PER8THNN"
departure {
time: 1712608380
}
stop_time_properties {
}
}

13:30
stop_time_update {
stop_sequence: 9
stop_id: "PER8THNN"
departure {
time: 1712608453
}
stop_time_properties {
}
}

13:32
stop_time_update {
stop_sequence: 9
stop_id: "PER8THNN"
departure {
time: 1712608455
}

13:33
stop_time_update {
stop_sequence: 9
stop_id: "PER8THNN"
departure {
time: 1712608441
}
stop_time_properties {
}

[vingerha]

OK, I need to replicate this. So aside the next_realtime_reparture also the attrib next relatime departureS (with S) is not populated?

[mrulrich1]

OK, I need to replicate this. So aside the next_realtime_reparture also the attrib next relatime departureS (with S) is not populated?

Correct. As far as I can tell, neither have ever populated since I configured the integration and there's a bus at that stop 36 times in a given day. I've been running 0.4.3 for a few weeks now. Thanks for looking into it!

[vingerha]

As of what time of day this line starts rolling? At present I see nothing for your stop

[vingerha]

and which of the links did you use for RT? I see two secition seemingly providing stop-updates, it should be this one normally

[vingerha]

My checks may only be possible end of week as I have no good access to back-end before that

[mrulrich1]

yes, I'm using the one you've highlighted.

That route runs 5:24 to 22:14 local time on weekdays. Less frequent on weekends. The last bus of the night is in about 40 minutes. It should start showing up in the data in the next 20 mins or so.

[vingerha]

So, I found the culprit...like often..the data.
In most cases... since this one it is not 'all'... travel is based on a route and direction (outbound/return) .. how else would you know for which direction the times are valid on stops with the same name. However, in the rt link you sent they are not providing any direction (see below), just the trip+route and this is in principle fine as the trip is more precise.

As my integration tries to look ahead, i.e. not just for the first/single next realtime departure, I am not using trip but "route and direction" which will normally give me all times for route 45 in direction 1....but since the rt has no direction, nothing is returned.

So... seems I need to make it more complex, sadly, and this will also result in the situation that in your case you will only receive the single next-departure and nothing further in the future.

The existing alternative you have is to use the vicinity based approach. i.e. show all departures around a zone...which could be your stop above. This may show more buses/trams/whatever then needed but it should work

For a solution here ...need to think a bit as this may re-introduce complexity that I recently removed :(

id: "982334"
trip_update {
  trip {
    trip_id: "982334"
    route_id: "45"
    start_time: "22:14:00"
    start_date: "20240410"
  }
  vehicle {
    id: "2210"
    label: "2210"
  }
  stop_time_update {
    stop_sequence: 1
    stop_id: "SHPRB4"
    departure {
      time: 1712812440
    }
    stop_time_properties {
    }
  }
....
....

[vingerha]

I will need to try later for further analsysis if that would work.... as I no longer have data due to the TZ differences

[vingerha]

So... with the TZ diff and all it took a bit of time but I also discovered another bug that no-one identified before so it was not 'just-for-you' . I do need to test more but if you know how to do this then you can copy the gtfs_rt_helper.py into the folder on your HA instance
gtfs_rt_helper.zip

EDIT ; after which you need to restart HA

[mrulrich1]

So... seems I need to make it more complex, sadly, and this will also result in the situation that in your case you will only receive the single next-departure and nothing further in the future.

For my use case that's totally fine. I'm only ever interested in the next bus. I can see where having more than one next bus/train would be helpful for others though.

[mrulrich1]

So... with the TZ diff and all it took a bit of time but I also discovered another bug that no-one identified before so it was not 'just-for-you' . I do need to test more but if you know how to do this then you can copy the gtfs_rt_helper.py into the folder on your HA instance gtfs_rt_helper.zip

EDIT ; after which you need to restart HA

Awesome! Thanks for tracking down the issue and creating a fix. I'll do some testing later this evening.

[vingerha]

Jusy to be clear, both fields next_departure and with s...will show the same single slot as this is based on trip. I you want more then again try to use the vicinity based aproach with a zone around your stop

[mrulrich1]

I replaced the gtfs_rt_helper.py file with the new one and am getting this error

Setup failed for custom integration 'gtfs2': Unable to import component: cannot import name 'CONF_OCP_APIM_KEY' from 'custom_components.gtfs2.const' (/config/custom_components/gtfs2/const.py)
5:49:32 PM – (ERROR) setup.py

which is preventing the integration from loading. This is after a restart, of course.

EDIT: I guess I could totally delete the integration and reinstall it with the new .py file?

[vingerha]

Try release 0.4.4.4