To a geek, having an internet connection is everything. Seeing as my budget conscious hotel didn’t offer free Wi-Fi, I decided I would try my luck and see if I could fix that. They did offer Wi-Fi on a paid-for basis, but its coverage was fairly limited and there really wasn’t any good signal within my room. Tell a geek to pay for Wi-Fi and you’re really asking for trouble … as I’d rather just use some of my mobile data and live a data-impoverished life before I’d pay for Wi-Fi.
Anyhow, my room was #106, and this was on the Girrawheen St side of the building – the first balcony above the driveway towards the right of the photo.
This wasn’t an optimal situation, as my “go-to” Telstra Air phone-booth connection was through Haig Park. It was highly unlikely I could make a connection based on that. Having previously been to Floriade, I know that Canberra had rolled out their own free Wi-Fi network called CBRfree WiFi.
A quick check gave me hope, as there was an AP nearby on Mort St, but it’s closer to room #128 than mine, so building shadowing might be an issue.
Yay! Free Data*!
As when I went to Melbourne earlier this year for a conference, I don’t travel without my photographic tripod, a yagi antenna, a USB 2.4Ghz Wi-Fi card, a USB extension lead and a roll of Nitto electrical tape. I decided it wouldn’t hurt to set it up and give it a try, hoping that I could save my precious mobile data megabytes.
I was really pleased when I managed to capture the CBRfree WiFi signal at a fairly decent -66dBm. But could it hear me, transmitting back at a measily 40mW or so?
As it turns out, it did, and fairly well too. I managed to get right on. The existential need for an internet connection has been fulfilled!
The public side IP and reverse DNS makes it clear that it’s a Wi-Fi network which is NAT’ed and in Canberra, hosted by Internode (now iiNet).
IP addresses as allocated are in the 100.72.0.0/16 subnet which makes it a carrier-grade NAT system. It also seems to allocate IPv6 network addresses and have an IPv6 gateway, making it an IPv6+IPv4 dual-stack network if I’m not mistaken.
Connecting with a phone and doing a network scan shows that the network has “AP Isolation” so that peer-to-peer connections are not allowed. This is good news as it stops the broadcast network chatter, and peer-to-peer virus infections. A proper network indeed!
The fun doesn’t last forever though – you get 250MB a day. Or so they say. For some reason, some days, I ended up with less before reaching the block screen. It might be circumvented by MAC address spoofing, but I didn’t have the time for that. I had enough devices on hand anyway! I didn’t encounter any issues with any of my applications – even my SIP phone service connected just fine, although I didn’t check if the UDP media audio worked properly. At least it wasn’t port-blocked as with some other free Wi-Fi offerings, or port-restricted – I managed to get home to my own VPN just fine.
In my experience, at least roaming about in the centre of the city areas, the network coverage was excellent and the signals were strong. Speeds were more than acceptable, which was a surprise as putting in such networks and making them perform well is no trivial undertaking. As a result, I was spurred on to try and understand a little more about how it worked.
In most parts, the hardware that serves the CBRfree network is in plain view. It is based around Cisco branded APs mounted about half-way up a light pole on a tiltable bracket with three antennas sticking out the bottom. Most of the APs have a Dymo-style label tape with the AP identification on it – in the case of the unit above, it is CIVIC_EAST_AP02-1. How this relates to the network topology will be explained later.
In some locations, it is somewhat obscured by the presence of flower pots, although this is merely a side effect of being mounted on the same pole.
While walking around, I noticed that some APs had two conduits connected, whereas others had just one. The extra conduit was connected to a terminal marked fiber, which matches the description that it is a “fibre based” network.
Underneath each AP is also a row of LED indicator lights which indicate its status, uplink, and the state of radio 1 and radio 2 within the AP.
In the case of single-conduit APs, the uplink light is not lit.
In the case of double-conduit APs, the uplink light is lit. Because they are installed outdoors, they really are exposed to the elements and can get quite dirty as well.
Recognize them yet? No? Well, turns out they are Cisco Aironet 1550-series APs. The particular model used in this deployment is the AIR-CAP1552E-N-K9 which features dual SFP fibre or Ethernet backhaul, and dual radios. The radios are capable of mesh network operation with 28dBm (~631mW) power output and N300 two-stream legacy beamforming. The sensitivity figures show it is about 4-6dB more sensitive than most client cards. The unit weighs 7.8kg with an option for a battery (0.7kg) and a bracket (2.8kg). Other options include GPS for positioning and DOCSIS 3.0 cable modem. Each AP is worth about US$4000. Pretty impressive stuff. The units also come fitted with the AIR-ANT2547V-N omni antennas which offers 4dBi gain at 2.4Ghz and 7dBi gain at 5Ghz.
One of the units I saw had a serial number beginning with FTX1824 … which maps to 2014 Week 24 (June).
Surveying the Network Topology
As you’ve probably already guessed, the network topology in this network probably consists of a number of “host” APs with direct fibre uplinks, and a number of mesh “peer” APs which relay the data to their nearest “host”. From the map they have supplied, the indication of host and peer is not at all clear, but as I’ve observed, the AP-relationships are somewhat indicated. AP’s are named something like REGION_DIRECTION_AP##-% where each ## indicates a distinct AP “system”, and any APs with a % number indicate they are most likely peers of the system.
To try and see whether my hypothesis holds true, I decided to go on a decently long walk around the city on my final day with my Garmin eTrex 10 on Waypoint Averaging mode to log the locations and fibre connectivity of each of the AP sets. Because of the annoyance of doing it by hand, the AP names have been abbreviated, and suffix FC indicates fibre connected (and indicated by a green target), and NF indicates no fibre connected (indicated by a yellow target). I plotted this with the data from their map as small pink dots.
I didn’t have the time to visit any more APs, but the 40+ survey effort was not a bad one. On the whole, most of the “suffix-free” APs were fibre connected, however, there are exceptions within the surveyed data! CIVIC_EAST_AP13-1 and CIVIC_EAST_AP_22-2 both had fibre connectivity, which was unexpected, but none of the suffix-free APs did not have fibre connected, so at least that seems to have held for this sample. The reason for this may be because of variances after planning or install, where APs may have been removed or connectivity may have failed and been re-established at a different end of the mesh-set.
The system architecture appears to be “restricted” by their design to a maximum of two “peer” APs per master. This image is of the two systems that cover Mort St. The two APs closest to the middle of the road receive fibre, and the outer APs are “peered” to this. The arrangement of APs are line-of-sight which makes for better connectivity and range.
The design arrangement may have been to maintain a good level of performance and coverage. Too many peered APs risks users at the far end tying up too much air-time as their signals need to be repeated multiple hops and possibly at a lower air-data-rate due to the limited signal between the sets of APs. Keeping the peered APs far enough that they effectively increase coverage without having them too far that the AP-to-AP link gets too slow or unreliable is another optimization point.
I took a walk into Glebe Park, which was somewhat heavily covered by foliage at certain points and was surprised that there was good Wi-Fi coverage in many areas. This was achieved with this arrangement.
One end of the park receives signals from a fibred AP #14, and is meshed to one peer #14-1. The other side of the park is heavily covered by trees, so AP#15 across the road with fibre cannot directly get a good link into the park. Instead, it wirelessly meshes with #15-1 in a “cutting” through the trees, and #15-2 meshes with that to extend the coverage effectively into the park. Rather interesting use, and shows how it’s not just a point-to-multipoint network but more likely to be a one/two-hop mesh.
In the very crowded areas, such as at the bus stop areas, we can see that networks do “overlap” somewhat. In this case, around the Sydney building, the networks “wrap around” corners. In other cases, they are linear systems of three. Other places, they are pairs, whereas a few places, they can “stand alone” as single. This sort of arrangement seems to be quite clever – it maximises coverage with a minimal outlay, but if traffic capacity increases to the point that the mesh wireless backhaul is a problem, it is likely to be possible with minor reconfiguration to “break” the systems down to pairs or individual APs if you can fibre them up.
It’s pretty fascinating, given the number of APs deployed and their cost – there’s a lot of money in there, but it seems to work well. Current stats seem to suggest 5-6Tb/month of user traffic, which isn’t insignificant.
The APs themselves broadcast other SSIDs for iiNet Customer and ACTS. I presume (and it’s sensible to do so) that the network serves at least a dual purpose to allow iiNet customers to evade the quota limits and improve safety of their transmissions with WPA2-Enterprise encryption over the air. The other purpose seems likely to enable the efficient connection of IP-wireless cameras for security as these seem to be scattered anywhere there is one of these APs.
GPS and Map Accuracy
Surveying all these APs gets pretty annoying quickly, especially on a very bright and sunny day. It was the reason I got a good tan on my return to Sydney. Code-tracking GPS units as used by consumers really only have about a 12m accuracy in most cases, even combined GPS+GLONASS units like mine. Sometimes the Garmin is a bit ambitious and claims 2-3m on a good signal, but it’s pretty hard to achieve that just simply due to the method of determining location, the propagation characteristics of the signals and the signals we have at our disposal.
To combat this slightly, the eTrex has a Waypoint Averaging feature which gives you a confidence score and averages positions for as long as you leave it. I took about 5-minutes or so at each location to reach 100% confidence so as to get a better location, and I wanted to see just how good it was.
In this other case near Civic square, the result showed the GPS was off by about the same distance, but the provided location is even further off.
There were some cases where the provided location was more accurate, but not by that much. I was actually quite pleased with the results, but claiming only 2-3m of error on the screen is a little optimistic most of the time. Of course, Google’s map alignment might not be perfect either …
Using Their Data
Given that their data was easily available on Google Maps, and I couldn’t survey all the APs … I decided to use that to see exactly what the network looks like at an overview scale. It doesn’t tell me which points are fibred and which points are not, but at least, it tells me which APs are part of a “system”. I’ve ignored all the indoor APs within their listings for this part.
In the Canberra City area, this is what the AP systems look like. In some high density areas, APs are not meshed (e.g. the sporting ground, along Northborne Ave’s central strip. The longest reach is the spur along Ainslie Ave.
Along this road, the system spans an impressive 231.78m from end to end in a straight line.
The system doesn’t just span Canberra’s central areas, as they have some outcrops in other major cities. The ones without any meshing have not been shown – but this is the set-up at the botanic gardens.
This is the set-up at Belconnen where meshing seems to be quite effectively used in an “interleaved” pattern to cover the waterfront.
In Dickson, it seems that meshing is used along the main road, and parallel streets in-between.
Near Kingston, there isn’t much meshing at all, and at Griffith, there is more meshing but also a cluster of APs in a high density arrangement possibly due to traffic requirements.
In Tuggeranong, the meshing isn’t as heavily used. It may just be that the layout doesn’t give good line of sight.
In more outlying areas such as West Greek and Woden, there aren’t many APs at all yet.
The Odd One Out and Other Things Spotted
While walking around, I found another unit near the theatre that looked like the same model but had neither conduit connected. Instead, it has what appears to be PoE and Ethernet uplink – probably this is part of the CBRfree “indoor” AP network, or maybe it’s another network entirely that just used the same model of AP, but I didn’t bother finding out. I suppose that for their mesh APs, they could theoretically use any other SKU from the same family, maybe sans the Fibre PHYs, but they chose to stick to the same model probably to improve upgradeability, limit stock on hand, and allow for easier interchanging of units.
I saw some LED lighting strung across some areas – some used the Tridonic Talex LCU 035/12 D power supply rated at 12v at 2.92A (35W). Other places used the MeanWell CLG-150-12 which outputs a massive 12V at 11A (132W).
Give me a free Wi-Fi network, and I’ll thank you for the connectivity. But make it visible, and I’ll wonder what it’s made out of and how it works. I feel that my on-foot survey was quite instrumental in working out the points above to satisfy my curiosity and “learn” from the network design by pure observation. In the process, I got to see a few other bits and pieces and test my GPS. Of course, that’s not all – I managed to take quite a few photos of less technology-related things, which will be in the upcoming parts.