Project: ADSB-pi Mark 2 with dump1090, hostapd, dnsmasq & text logging

This is one of the things that I’ve done a long while ago, but never had the time or inclination to actually put together a post. You might remember the first iteration of the ADSB-pi back in 2013, which was a little troublesome, and utilized ad-hoc Wi-Fi networking, isc-dhcp-server and rc.local start-up scheduling.

Of course, since then, I’ve learnt about hostapd and dnsmasq when I built my “captive portal” advertising-pi, and I liked dnsmasq better than isc-dhcp-server, as well as having an infrastructure “softAP” single-Pi usage. As a result, I decided to incorporate this.

The other thing I noted was that dump1090 had some updated forks with more aggressive decoding features, so I decided to change over and use that to get a little more bang for the buck.

The other thing is that normally, such a system is only as interesting as you have time to watch it in real-time. However, I really didn’t have that much time in my day to do that, and it turned out that my access to “sky” and good signals was a little limited, so I wanted some simple logging without the whole fuss of using a mysql database.

This post isn’t going to be a full tutorial, but more so some snippets of configs and issues considered.

Configuration

There was nothing really special to mention – I started with a stock installation of Raspbian, opting for no GUI start-up. I overclocked the Pi to 1Ghz, as the board of choice could handle it, and the added power was to come in handy, and updated all the packages.

From there, I installed hostapd, dnsmasq, libusb, cmake, autotools and cloned the rtl-sdr and dump1090 repositories and built it.

To ensure the rtl-sdr tuner can be accessed by the programs, we need to blacklist the TV-tuner module driver that would otherwise make it a dvb device. To do this, I edited /etc/modprobe.d/blacklist.conf and added the line:

blacklist dvb_usb_rtl28xxu

For the SoftAP, I set a configuration called hapd1.conf in my home directory with the following contents:

interface=wlan0
hw_mode=g
channel=1
ieee80211d=1
country_code=AU
ieee80211n=1
wmm_enabled=1
beacon_int=200
max_num_sta=8

ssid=[YOUR SSID]
auth_algs=1
wpa=2
wpa_key_mgmt=WPA-PSK
rsn_pairwise=CCMP
wpa_passphrase=[YOUR PASSWORD]

Accordingly, because the Pi was to have a static address, I modified /etc/network/interfaces such that the wlan0 section showed the following:

allow-hotplug wlan0
iface wlan0 inet static
address 10.20.30.1
netmask 255.255.255.0
wpa-conf /etc/wpa_supplicant/wpa_supplicant.conf

I chose 10.20.30.1 for the unit to reflect the decision made with the original ADSB-pi, but also because it was easy to remember and unlikely to overlap with other commonly used subnets. This will prove advantageous in the future.

To make things convenient for connecting users, we need to configure the DHCP server, so /etc/dnsmasq.conf was edited to the following, and the service was restarted with sudo dnsmasq restart.

no-resolv
except-interface=eth0
log-dhcp
dhcp-range=10.20.30.2,10.20.30.10,24h
dhcp-option=3
dhcp-option=6
address=/adsb.adsb/10.20.30.1

The configuration is very deliberate. It has no-resolv at the beginning so that DNS resolves are not handled with the configured upstream DNSes. The wired Ethernet interface is excluded, as we only want to serve out of the wireless. DHCP transactions are logged, for debugging purposes and if you want to know who’s connected. Eight addresses are available with a 24 hour lease (matching the maximum 8 STAs of hostapd as configured, to prevent overburdening the pi). DHCP-option 3 and 6 are set to null so that the DHCP server puts out an empty DNS server and Gateway server. This means that any connecting clients know not to resolve IPs through that interface and not to send any non 10.20.30.0/24 traffic down that interface. This means you can have a PC connected to a Wi-Fi network with internet and the adsbpi concurrently without having problems reaching either network and the internet. Pretty cool.

Of course, we still have a few things to fix up – for one, we can only use the unit interactively. To try and collect the data without requiring a heavy-duty solution, I decided I would just record the AVR formatted data into a text file using netcat. I created a script called logadsb.sh in my home folder, which just had the following:

#!/bin/sh
sleep 30
nc 127.0.0.1 30002 > /home/pi/adsblog/$(date +"%Y%m%d-%H%M%S.txt")

The sleep 30 makes the script wait for 30 seconds before executing, so the rest of the start-up has a chance to finish and the tuner initialization is all done. You could shorten it somewhat. The log-file will use the date and time, but because the Pi has no RTC, the time will be incorrect unless it was connected to Ethernet at boot-up and got the right time from NTP. This is no big issue, as it ensures the uniqueness of the file names. Logged files can be downloaded by ssh-ing into 10.20.30.1 and browsing to ~/adsblog/.

To wrap-it-all-up, we need to get it to start-up automatically. Before, I used rc.local, but I now prefer a different route – using the system crontab instead. I modified /etc/crontab directly rather than using any specific tools, and added above the table:

@reboot root cd /home/pi/dump1090 && ./dump1090 --quiet --net --net-beast --fix --phase-enhance --aggressive
@reboot root hostapd -B /home/pi/hapd1.conf
@reboot root /home/pi/logadsb.sh

The @reboot tasks will execute on every reboot, and take a user argument before the command. Simple as that, and no more “hit and miss” with “will it work, or will it not?”

Shutting down is still a matter of ssh-ing in and issuing a sudo shutdown -h now although there are plenty of guides to use a switch to trigger shut-down. I just couldn’t be bothered to do it.

The Mission, and the Unit

Because I didn’t have as good of a reception condition as I liked at home, I wanted to see just how well one could receive ADSB given a good vantage point.

placement

In my case, the best vantage point I could establish with the access I had at the time was the top of the multi-storey Botany Street carpark at UNSW. This was at the upper end of campus, which is on the edge of a hill, and was about as high as I could get with open sky views without any special requirements.

2016070316377914

Nowadays, there are more “modern” versions of Raspberry Pis – now up to the Pi 3. But even then, the old “original” Pis still serve me well, regardless of the fact they are slightly underpowered by comparison and not too power efficient. For my ADSB-pi Mk2, I decided to use my trusty Model B, and go with a clear Multicomp case with a little warning label inside with contact details, just in case it gets discovered and mistaken for something more dangerous than it actually is.

It has a bulk electrolytic capacitor plugged into the 5V line to keep everything nice and smooth in case the contact on the microUSB-B shifts slightly when being positioned. Aside from that and a polyfuse bypass, the board is pretty much “stock”.

The wireless adapter chosen was the Wi-Pi, as they were selling them off for cheap. These don’t quite have the range as the TP-Link WN-722N’s do, but on the “receiving” end, I’m using a laptop with a yagi hooked to the WN-722N, so I can make up for it somewhat. Just like I did when I was on holidays.

The tuner is an ISDB-Mini stick, a fairly old E4000 based tuner, although I did use a full-size R820T as well. It really didn’t matter too much.

2016070316377915

The unit is lashed together with black small cable-ties to stop people from prying it open. The main microSD card is just an old Sandisk 4Gb card, in a non-name micro adapter so as not to protrude out the end (and potentially get moved or stolen).

20150821_120302

A suitable location was found, which was near the fencing behind a row of parked cars. Not too obvious, and hopefully not likely to arouse suspicion, I mounted the unit in the morning and took it down well-after dark. The Xiaomi 16,000mAh power bank easily kept it running more than a day, and even though the car-park was probably about 5-levels up, the Wi-Pi signal with the dongle through the wire fence was still strong enough to connect with a phone from ground level. Only just. At least I could tell without walking up all the floors whether the unit was still safe … (no lifts in this carpark).

Analysing the Results

Using the unit interactively to check on the visible flights over lunch was easy, as was grabbing the files off. It was a matter of connecting, and then configuring the use of 10.20.30.1 as the data source with the appropriate port (depending on needed data) or sshing into the system.

hostapd-ap addresses

When I got home, I downloaded a 40-50Mb text file from the unit. To actually process this proved to be a little challenge. My traditional socat/netcat skills were not enough when dealing with ADSBScope, as it stubbornly refused to accept “bulk” lots of data that way at a fast rate and so nothing got plotted.

Instead, I had to make a small program I called slow.c which reads a line at a time from stdin, and then serves it out of a socket server on the localhost:30002. This was adapted from some C-sockets tutorial online (I forget which):

#include <stdio.h>
#include <unistd.h>
#include <assert.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>

#define PORT_NUMBER 30002
#define DELAY_US 7500
#define LINE_LENGTH 128

int main () {
  char string[LINE_LENGTH] = {0};
  int count = 0;
  int sockfd, newsockfd, clilen, n, cchar;
  struct sockaddr_in serv_addr, cli_addr;
  sockfd = socket(AF_INET, SOCK_STREAM, 0);
  assert(sockfd);
  serv_addr.sin_family = AF_INET;
  serv_addr.sin_addr.s_addr = INADDR_ANY;
  serv_addr.sin_port = htons(PORT_NUMBER);
  assert(!(bind(sockfd, (struct sockaddr *) &serv_addr,sizeof(serv_addr)) < 0));
  listen(sockfd,5);
  clilen = sizeof(cli_addr);
  newsockfd = accept(sockfd, (struct sockaddr *) &cli_addr, &clilen);
  assert(newsockfd);

 while ((cchar = getchar()) != EOF) {
  string[count] = cchar;
  count++;
  assert(count<LINE_LENGTH);
  if(cchar == '\n') {
    string[count] = '\0';
    n = write(newsockfd,string,count);
    assert(n);
    count = 0;
    usleep(DELAY_US);
  }
 }
 return(0);
}

It has a #define which allows the delay time between frames to be configured, and I chose the shortest time which appeared to allow ADSBScope to plot without obvious frame loss in the replay rate diagram (which has dips in case of overflows). Yes, it’s not particularly fool-proof as it has fixed length buffers, but assert should catch overflow. AVR frames aren’t supposed to be that long anyway.

replay-rate

This was enough to allow me to slowly plot out the data which took many hours. Of course, you lose the temporal element of ADSB, but at least you can review the happenings of the day in an accumulated plot without having to remain connected or in range for the day. You don’t even need anything so complex as a mysql server. That being said, I did consider a packet-timestamp and replay system of some sort, but I just didn’t feel it was worth the hassle in my case.

replay-wide

Looking at the wide picture, it’s clear that ADSBScope does have some issues when plot lines aren’t cleared overa long time and some ploy lines might be reused or bad frames may have snuck through resulting in traces jumping in and out of the area. It seems that there was some good tracking out to even 300 – 350km at the high altitudes (FL380+), which is pretty good considering I still have the “shitty” coax of the stock magnetic base, and am using just a simple cut-down stock “stick” antenna.

sydmetro

Because UNSW is not far from Sydney Kingsford Smith airport, we can get good quality traces of the approach and departure flows throughout the day. Looks a little like hair, but I’m quite happy to see that even the “flight training” at Bankstown Airport with their trainees doing circuits, plus their choppers having a nice trail on the map.

chaotic

Moving slightly across, we can see some more chaotic movements near Camden airport and what probably is a chopper doing some surveys or something along roads, pipelines, etc.

bankstown-wide

Zooming into Bankstown, we can see the circuits it seems, as well as some very “elegant” loops left by pilots over the course of the day.

airport-groundlevel

Because of the good vantage point with its tall height, there is a pretty decent “map” of the taxiways on the ground at YSSY, along with the two runways. To have ground level reception is something I just can’t get from home.

weak-wollongong

From home, I see Sydney like we see Wollongong from the carpark – not very clear, sporadic single frames here and there resulting in sharp, angular, jagged traces. But it is impressive to me that we can see flights at that distance, and the “improvements” come at nil expense.

Conclusion

I know RTL-SDR based stuff is no longer considered as cool as it once was, and that Raspberry Pi based ADSB reception is not new. But it can still bring quite a bit of joy, especially when you can see and receive at locations which have much better signal than you do at home. With the advent of “online” pools of receivers, say FlightRadar24 and FlightAware for example, people might rightly question why one would even bother to “roll their own”. I think the adventure and the feeling of satisfaction of knowing that you did (some of it) yourself to be more than worth it.

Aside: The unit was built and run in August 2015, and I couldn’t find the time or inclination to post about it until now. It’s been sitting on my desk for a while, and I connected to it today just to see it was still working as it was before. I also have another similar set-up on another Pi running acarsdec for multi-channel ACARS decoding – much the same principle, but different type of transmission targeted.

About lui_gough

I’m a bit of a nut for electronics, computing, photography, radio, satellite and other technical hobbies. Click for more about me!

This entry was posted in Computing, Raspberry Pi and tagged , , , , , . Bookmark the permalink.

8 Responses to Project: ADSB-pi Mark 2 with dump1090, hostapd, dnsmasq & text logging

  1. Aardvark says:

    ADS-B reception may be no longer be cool but it still is fun and allows me to combine my computer, electronics and radio skills into something useful (I hold an FCC amateur license as well as FCC commercial license). I run two receivers: one at home on the north shore of Long Island and the other 38km away at work on the south shore of Long Island. Both in the metropolitan New York City area and near six busy airports (KJFK, KLGA, KEWR, KHPN, KISP and KFRG). For the longest time, my home receiver was the best of the two. It is subject to attenuation from nearby trees and the hilly terrain of the north shore but performs well out to 100nmi or so. The southern antenna was low and blocked by the side of my office building but gave me good coverage south over the Atlantic Ocean. However, last winter the landlord had a permanent outdoor ladder installed on the side of the building that allowed me to get up to the topmost roof were there are no obstructions. So this spring I moved my ADS-B antenna there. Well the first thing that happened was there were so many aircraft signals that my original Raspberry Pi was running at 100% CPU. Not only do I feed my own Virtual Radar Server installation but also feed FlightRadar24.com, FlightAware.com and PlaneFinder.net. So I upgraded to a Raspberry Pi 3 which is more than sufficient to handle the load including the MLAT traffic. So now I can easily see aircraft out to between 150nmi and 180nmi on that receiver due to the flat south shore terrain and lack of tall trees. It is great but also makes me sad because there is a good chance I will be gone before the end of the year from my current employer, after almost 19 years there, and this great setup, running only since the first week of May, will have to come down. I wish I could get the same coverage at home but that will just not be possible due to the surroundings. I guess I will enjoy it while I have it.

    One other area I am just starting to experiment with now is the USA only Universal Access Transmitter (UAT) protocol operated by some General Aviation aircraft under 18,000 feet on 978MHz. There is a dump978 utility available (https://github.com/mutability/dump978) that can decode the UAT protocol and display the packets. There is not much work going on with development outside the USA because it is strictly a USA standard. Some experimenters have been able to collect the packets and feed them into their existing ADS-B feeds thus making those non-ADS-B UAT enabled aircraft visible without needing MLAT.

    • lui_gough says:

      I did hear of UAT in my travels, although from Australia, I’m fairly sure we won’t see it here so soon partly due to differences in band allocations, but mainly because our airspace is nowhere near as crowded as that of the USA. We have one of the first country-wide ADS-B “secondary” radar coverage deployments, and I think they’re pleased enough with their visibility. My RPi (original) barely even gets up to 60% of CPU load where I had put it, so it’s definitely night-and-day between the two locations. The new Raspberry Pi boards are amazing with more than a clock-for-clock increase in performance and multiple cores – the Pi 3 also has built in Wi-Fi but I still haven’t had one to play with, and neither did I manage to get my hands on a Pi Zero. I’m still happy enough with the original – one of them doing RTL-SDR decoding of other signals has reached >360 days of uptime.

      I wonder if there is any open VDL decoder, since ACARS is dying a slow death and VDL is poised to take over. I only heard that MultiPSK had VDL in its later versions, but as a paid option.

      – Gough

  2. David Sutherland says:

    Aardvark & Gough, if I want to also feed FlightRadar24.com, FlightAware.com and PlaneFinder.net from my home what is the best setup? Is the RPi 3 suffient to do everything? Near post. Thanks.

    • lui_gough says:

      Judging from Aardvark’s reply, it seems possible. I’ve only known most to do one or two at the most, but I think it’s probably just because the first two are more popular. Regardless, the three have some instructions for setting up their “own” server:
      https://www.flightradar24.com/raspberry-pi
      https://flightaware.com/adsb/piaware/install
      https://planefinder.net/sharing/client

      I suspect you will have to investigate them and how they install. Chances are, they all rely on dump1090 as the decoder – so I assume the setup will be one instance of dump1090 connected to multiple “clients” which pass the data onto the services.

      I’ll probably leave the actual implementation for Aardvark to explain, as I don’t actively contribute my data at this time.

      – Gough

      • Aardvark says:

        Yes, all of the applications expect dump1090 to be running. I believe pfclient attempts to install its own but skips it if one is already running. As you noted, there are many forks of dump1090. I settled on FlightAware’s own fork called fadump1090 that can be obtained from https://flightaware.com/adsb/piaware/install. The source is available at https://github.com/flightaware/dump1090/archive/dump1090-v1.2-3.tar.gz if you want to build a copy yourself. I believe it is a fork of the popular “mutability” variant but I am not sure. I went with this version because it adds a command argument called “–forward-mlat” which allows MLAT data from FlightAware to be merged back into your dump1090 data stream for use with your own server such as Virtual Radar Server. The only issue I had with it (and apparently others with similar dump1090 versions) was it occasionally shutting down without any warning or error message. I added a cron job to check every ten minutes to see if it was running and restart it if it was not. It turns out the built in http server in dump1090 is very flaky and random probes to its port from the Internet would crash it from time to time. By adding “–net-http-port 0” to the command line, that effectively disabled it and it has been rock solid ever since. If you are running pfclient, it provides an excellent map as well as statistics pages on port 30053. It is a good alternative to using dump1090’s built in http server.

    • Aardvark says:

      The Raspberry Pi 3 is more than sufficient. I was only running FlightAware and FlightRadar24 on my original Raspberry, along with Dump1090, and hit 100% CPU when I raised my antenna to its current location. That same configuration left about 90% idle CPU time on the RPi 3. Since adding PlaneFinder, the idle still is in the 85-89% range. The most CPU intensive apps are dump1090 and FlightAware’s MLAT client fa-mlat-client process. The piaware, faup1090, fr24feed_arm-rpi_242 (which is old BTW) and pfclient processes use no more than about 4% of any of the four CPU’s in a burst and usually run between 1% and 3%.

      I have been debating about whether or not to add another DVB-T stick to the setup so that I can decode UAT at 978MHz. The biggest issue is probably the powered CPU hub getting overloaded with power demands of the additional device. Besides doing ADS-B feeding, it also has an HP-3110 HD camera connected and feeds a streaming video of the rear parking lot of my office building. Unless someone is actually watching the stream, there is almost no CPU overhead from the mjpg_streamer process. Even if someone is watching it, CPU will be at most 1%.

      The RPi 3 is definitely a powerhouse and excellent for CPU intensive applications. I still use two RPi 1’s for less demanding tasks (dedicated camera feed and my home based ADS-B feeder that does not get overloaded with air traffic like my office feeder). They are still a good computer, especially if you can find them for USD$15-$20. I have a local store that is part of a chain (MicroCenter) that sells the RPi 3’s for USD$29.95 so it was a no-brainer to get them versus the RPi 2 which was priced the same.

  3. spartiatis says:

    Nice work, i recently built a similar unit. Lotsa fun.

Error: Comment is Missing!