Tripview and Sydney Buses Realtime Data

Once I saw that Tripview had Sydney Buses real-time data – I thought this would be fun to play with! One of my friends, Nonie, was quite interested in how the data was disseminated and collected, and I thought it would be rather interesting to look at how this application gets its real-time data.

Unfortunately, it’s late at night, and so there isn’t any real data available. But still, we can try and find out where it comes from.

Step 1: Configure a router as an Access Point

Step 2: Connect AP to second NIC port, bridged to first NIC port with access to the internet.

Step 3: Fire up Wireshark to capture on the AP NIC port.

Step 4: Connect iPad to purpose-set AP.

Step 5: Open up Tripview, and select a stop and a destination and look at the route.

Profit!

Requests are made as follows:

GET /tripview/delays?routes=SB_555_u HTTP/1.1
Host: realtime.grofsoft.com
Connection: keep-alive
Accept-Encoding: gzip, deflate
User-Agent: TripViewLite/212 CFNetwork/609 Darwin/13.0.0
Accept-Language: en-us
Accept: */*

Where in this case, 555 is the route number. I assume SB stands for Sydney Buses. The letter u may stand for the direction – but I don’t know – so far everything has an u appended to it. So it’s pretty much equivalent to requesting http://realtime.grofsoft.com/tripview/delays?routes=SB_555_u. The response is a JSON fomatted reply:

HTTP/1.1 200 OK
Server: ngx_openresty/1.2.3.8
Date: Tue, 18 Dec 2012 12:12:00 GMT
Content-Type: application/json
Transfer-Encoding: chunked
Connection: keep-alive

18
{
 "pollInterval" : 30,

1b
 "timestamp" : 1355832711,

e
 "delays" : [

4

]}
0

If you enter the URL in the browser, you don’t seem to see the random 18, 1b, e, 4 and 0 for some reason, but it is in the Wireshark hex-view of the packet itself. The reply from the server also sets the poll interval on the client – interesting, as they might use this during peak loads to moderate the client poll rates!

In the case of a multi-route request with actual data, the query has the routes separated by commas as so:

GET /tripview/delays?routes=SB_422_u,SB_423_u,SB_426_u,SB_428_u,SB_436_u,SB_438_u,SB_439_u,SB_440_u,SB_443_u,SB_470_u,SB_500_u,SB_501_u,SB_502_u,SB_504_u,SB_506_u,SB_508_u,SB_515_u,SB_520_u,SB_555_d,SB_L23_u,SB_L28_u,SB_L38_u,SB_L39_u,SB_M52_u HTTP/1.1
Host: realtime.grofsoft.com
Connection: keep-alive
Accept-Encoding: gzip, deflate
User-Agent: TripViewLite/212 CFNetwork/609 Darwin/13.0.0
Accept-Language: en-us
Accept: */*

HTTP/1.1 200 OK
Server: ngx_openresty/1.2.3.8
Date: Tue, 18 Dec 2012 12:22:06 GMT
Content-Type: application/json
Transfer-Encoding: chunked
Connection: keep-alive

18
{
 "pollInterval" : 30,

1b
 "timestamp" : 1355833292,

e
 "delays" : [

15e
 {"route":"SB_423_u","tripId":"967107","start":"22:55","offsets":"22:59,8,23:02,7,23:06,6,23:09,5,23:10,6,23:12,7,23:16,6,23:21,7,23:21,6,23:22,7,23:23,8,23:25,7,23:33,6,23:34,7"},
 {"route":"SB_423_u","tripId":"966438","start":"23:21"},
 {"route":"SB_423_u","tripId":"967402","start":"23:40"},
 {"route":"SB_423_u","tripId":"967414","start":"00:10"}

f4
,
 {"route":"SB_426_u","tripId":"967120","start":"22:54","offsets":"22:55,8,22:57,9,23:01,8,23:05,7,23:07,8,23:10,7,23:13,6,23:15,5,23:18,4,23:19,5,23:23,6,23:25,5"},
 {"route":"SB_426_u","tripId":"23053501","start":"23:17","offsets":"23:17,0"}

10b
,
 {"route":"SB_428_u","tripId":"967615","start":"22:55","offsets":"23:00,0,23:01,-1,23:04,-2,23:06,-1,23:07,-2,23:07,-1,23:09,0,23:10,-1,23:13,-2,23:14,-1,23:15,0,23:20,-1,23:22,-2,23:27,-1,23:29,-2,23:30,-1"},
 {"route":"SB_428_u","tripId":"967197","start":"23:20"}

b7
,
 {"route":"SB_436_u","tripId":"77010036","start":"23:05","offsets":"23:08,3,23:10,4,23:14,3,23:15,4,23:17,5,23:18,4,23:22,5,23:23,4,23:27,3,23:28,4,23:32,5,23:33,6,23:35,5,23:36,6"}

11a
,
 {"route":"SB_438_u","tripId":"77010029","start":"23:26"},
 {"route":"SB_438_u","tripId":"77010028","start":"22:46","offsets":"22:47,4,22:48,5,22:51,4,22:52,5,22:53,4,22:55,5,22:56,4,22:59,3,23:00,5,23:02,6,23:04,5,23:05,4,23:09,3,23:12,2,23:15,1,23:16,3,23:17,4,23:20,3,23:21,4"}

f4
,
 {"route":"SB_440_u","tripId":"75112955","start":"23:22"},
 {"route":"SB_440_u","tripId":"75112626","start":"22:53","offsets":"22:54,4,22:56,3,23:02,4,23:03,3,23:05,2,23:11,1,23:14,-1,23:15,0,23:17,-1,23:18,0,23:19,1,23:20,2,23:22,1,23:23,2"}

8a
,
 {"route":"SB_443_u","tripId":"75113491","start":"00:05"},
 {"route":"SB_443_u","tripId":"75113490","start":"23:35","offsets":"23:35,1"}

4f
,
 {"route":"SB_470_u","tripId":"75112305","start":"23:25","offsets":"23:25,0"}

11b
,
 {"route":"SB_501_u","tripId":"76352556","start":"23:05","offsets":"23:06,2,23:10,4,23:13,3,23:14,2,23:16,1,23:21,0,23:25,-1,23:27,-2,23:27,-1,23:30,-2,23:31,-1,23:33,-2,23:36,0,23:36,-1,23:37,0,23:40,-1,23:43,0,23:44,-1"},
 {"route":"SB_501_u","tripId":"76351876","start":"23:35"}

b8
,
 {"route":"SB_506_u","tripId":"76351722","start":"22:53","offsets":"22:55,2,22:59,1,23:00,0,23:01,2,23:02,1,23:03,2,23:06,1,23:13,2,23:14,1,23:19,0,23:24,-1,23:25,1,23:27,0,23:28,1"}

14f
,
 {"route":"SB_520_u","tripId":"76351854","start":"22:47","offsets":"22:49,4,22:49,3,22:51,2,22:56,1,22:57,0,23:00,-1,23:01,0,23:03,-1,23:06,-2,23:08,-1,23:12,0,23:13,1,23:13,0,23:19,1,23:21,0,23:22,-1,23:24,-2,23:26,-3,23:27,-2,23:33,-3,23:38,-4,23:39,-3,23:41,-4,23:47,-3"},
 {"route":"SB_520_u","tripId":"76351856","start":"23:39"}

4

]}
0

So we are returned one array for each route – each array consisting of separate services – each separate service having its timetable time for each of the stops, followed by an offset (likely in minutes) representing the bus performance (late/early) for the stop. It appears that the actual stop lists for each route is separated in the database/app itself, so is not transmitted, making this data a bit hard to interpret without consulting the database (likely contained in the timetable data updates).

So I uninstalled Tripview, reinstalled it and sniffed it while it updated itself – the requests were as follows:

GET /tripview/v15/syd-buses-069.tdb HTTP/1.1
Host: syd.grofsoft.com
Connection: keep-alive
Accept-Encoding: gzip, deflate
User-Agent: TripViewLite/212 CFNetwork/609 Darwin/13.0.0
Accept-Language: en-us
Accept: */*

HTTP/1.1 200 OK
Date: Tue, 18 Dec 2012 12:35:34 GMT
Server: Apache
Last-Modified: Sat, 15 Dec 2012 07:42:56 GMT
Accept-Ranges: bytes
Content-Length: 1429187
Cache-Control: max-age=2592000
Expires: Thu, 17 Jan 2013 12:35:34 GMT
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive
Content-Type: application/tripview-database

And then … it was followed by what it seems is their own propietary format for the database – the first three bytes were TDB – assuming this is “Tripview DataBase”. They even defined a MIME type for it, and gave it an extension of .tdb. It contained no readable strings, so is probably compressed/encrypted. You can request the file by replicating the request – http://syd.grofsoft.com/tripview/v15/syd-buses-069.tdb

Performing a byte-frequency analysis results in there being a peak of 0x00’s – implying that this file might not actually be encrypted (as that tends to distribute the byte frequencies almost identically), but doesn’t give us any clues as to the identity of the file. There are also stand-out peaks at 0x04, 0x60, 0x78 and 0xDA.

And so, I think this is where the discovery ends. I have no real interest in “breaking” the system as such, merely discovering how things are done.

About lui_gough

I'm a bit of a nut for electronics, computing, photography, radio, satellite and other technical hobbies. Click for more about me!
This entry was posted in Computing, Tablet and tagged , . Bookmark the permalink.

3 Responses to Tripview and Sydney Buses Realtime Data

  1. Pingback: Tripview Bus Time Offset Visualization | Gough's Tech Zone

  2. Pingback: Update on Tripview Live Bus Data | Gough's Tech Zone

  3. Pingback: Checking it out: Cityrail Real-Time Train Information | Gough's Tech Zone

Error: Comment is Missing!