Project: DIY Weather Station & Displays

It’s summer in Sydney and we’re definitely feeling it. In the west, there are often strings of 32 degree Celsius or above days coupled with dangerous storm activity, so staying comfortable is not always easy even with the modern conveniences of air conditioning.

To better manage energy usage and keep the house cooler, I’ve long relied on having good outdoor and indoor air temperature readings. This can come from something like a 433Mhz weather station, which I have owned in the past. Unfortunately, as I have found, the cheaper sensors have mostly broken down, the units consume a decent amount of battery, the humidity function often breaks down due to oxidation/electrolysis, the transmission range is sometimes poor and the response time of the thermistors is not entirely swift.

Knowing that more modern digital I2C sensors can be had for low cost with better accuracy and multiple functions, while Wi-Fi capable system-on-a-chip devices such as the ESP8266 are also abundantly available at low cost, I decided to hack up a simple replacement.

The Weather Stations

Being rather lazy, I decided to avoid any complicated protocols, so I decided from an early stage that I would use broadcast UDP to send my weather station data. Each of the stations would send their data in a “sentence” format to 255.255.255.255 (broadcast) on port 9999/9998/9990:

@LOCATIO,TT.TT,HH.HH,SSSSSSSS#

The presence of the format will allow us to easily reject any accidental clashes with other protocols that might use the same port and gives us an easy way to multiplex a number of sensors on the same port (if desired). For now, I’ve decided that my main sensors will be on different ports to prevent any chance of interference.

I decided to keep things extremely cheap by using whatever sensors I had to hand and by not using any RTCs at all on my sensor stations. Instead, the “S” value is based on the sample number sent from the station just as a data-gap detection method (currently unused).

Station 0: Outdoor

The most important station was the first station to be built – the one that went outdoors. In this station, I used a Bosch Sensortec BMP280 (as I got dudded on eBay a few times) which provides temperature and barometric pressure, but not humidity.

As the earliest station, I used an ESP-01 module to provide Wi-Fi which proved slightly annoying to work with due to the limited pins, lack of easy deep-sleep implementation. The compact size proved to be advantageous though.

I built a perf-board based system with a linear regulator for the solar panel, an old 12v 7Ah gel cell (providing about 4.5Ah effective capacity) as the power storage, a step-down switching converter to buck the voltage from 12v to 3.3v and the ESP-01 module.

For weatherproofing, I encapsulated the whole ESP-01 module in hot glue, and put the whole perfboard and battery into an ice-cream tub with a few cutouts for power. I had to add a few holes to the bottom of the tub to let condensation drain out of the unit.

To weatherproof the BMP280, I connected the I2C via a piece of screened USB lead with the ends cut off – four conductors is perfect and screening should add some resilience. The BMP280’s PCB is covered in two layers of heatshrink – any temperature change will take longer to permeate through the plastic, but water should not. So far, I found that if I “squeeze” the ends of the shrink while it is almost overheated, it “seals” like a heat sealed bag. After that, I wrap it in one more layer of overlapping electrical tape for extra security.

No pictures (as I don’t really want to show off the backyard), unfortunately, but it has been continuously connected to the Wi-Fi for over 70 days.

The outdoor station runs its own firmware that emits a packet once every 10 seconds – frequent enough to get a good trend without overburdening the network. The Fingbox also keeps an eye on its connectivity.

Station 1: Indoor

The next most important station is an indoor station to compare temperatures with. I could have used the BMP280, but at the time, I was actually more interested in the Si7021 as it offered temperature and humidity, but not barometric pressure.

I decided to make the indoor station essentially “free” by tacking it on to my Arduino Yun which was already serving as a 433Mhz Easy-Off Sockets controller (an upgraded version of my first one which is only Arduino based which is still in use at the other house).

On the Yun’s firmware, as the Arduino has no direct control over the Wi-Fi (which is handled by the Atheros chip running Linux/OpenWRT), instead we need to rely on passing a system command over the bridge:

cmd+="echo \"@INDOOR_1,"; 
cmd+=lastTemp;
cmd+=",";
cmd+=lastHum;
cmd+=",";
sprintf(timestring,"%010lu",millis());
cmd+=timestring;
cmd+="\#\" | socat -u - UDP4-DATAGRAM:255.255.255.255:9999,sourceport=9999,broadcast";
p.runShellCommand(cmd);
p.flush();

As a legacy of the patchwork design, this one reports the millis() counter raw in the packets rather than a sample number. Still useful to understand the sample-to-sample timing variations, but since this one is indoors, it’s easy to update the code wirelessly as well. As a result, I’m a little less concerned about this one.

So even though I’m using a mixture of hardware, that’s no problem as the system is very much agnostic.

Station 2: Portable

The third sort of sensor came out of a necessity – namely to answer those strange questions such as “how cold is the fridge/freezer?” or “is my room warmer than your room?”

As a result, I built something based on the same ESP-01 board and BMP280 sensor, instead, being really lazy about voltage regulation and powering the unit from two AA batteries instead.

As it turns out, as the BMP280 is rated for 1.8V operation, it will happily give values down to about 1.7V. The ESP8266 is a little more sensitive, but I found it can operate down to about 2V, which is still good enough for getting most out of a pair of Ni-MH cells or alkalines. Best part is avoiding any regulator losses or complexities – if I close the lid, it’s barely bigger than four AA batteries. The sensor is mounted on a cable because the self-heat generated by the ESP8266 is enough to skew the results significantly.

Without any deep sleep, and sending out packets every 10 seconds on port 9990, this unit dubbed “Minimon” will run for about 20 hours from a pair of AA cells. Not great, but not unacceptably bad for a first effort.

The Displays and Logger

While it’s all good and fine to get the data onto the network as a first step, the most useful part of the 433Mhz weather station clocks is that you could actually see the values, and in a convenient way.

The most basic way to do this is just to be on a computer connected to the network with a command line interface you can run netcat on. Something like:

nc -lu 9998

would be enough to get the data starting to show on the screen in a rather nonchalant way – under Windows, I do this with Cygwin. It is possible to like receive and plot the data in MATLAB by using judp – something I had also done. Definitely needlessly “heavy duty” – like taking a chainsaw to cut out a picture from a newspaper, but it worked.

But unfortunately, we’re not always at the computer and it really makes no sense to start up a computer just to see what’s happening with the weather. As a result, it’s time to build a dedicated display.

By then, I had graduated to the NodeMCU Amica 1.0 boards, based on an ESP-12E module with all the I/Os broken out nicely and a Silicon Labs CP2102 USB to UART bridge integrated. A lot less problematic, with more memory and voltage regulators on board while still remaining at a low “sub $5” price. The ESP8266 connects directly to my Wi-Fi network to receive the data from the sensors.

The downside was, as with all the ESP8266 boards, that they are 3.3V I/O and some older things are 5V TTL. For example, I had some 16×2 HD44780-compatible parallel LCD displays lying about which were 5V TTL.

In my desperation, I have it a try with this nasty jumper wiring job and using the LiquidCrystal library, using almost all the available I/Os on the board. In the code, the two UDP ports were bound, and data received and handled separately –

void setup() {
  // Snipped
  Udp1.begin(9999);
  Udp2.begin(9998);
  // Snipped
}

void loop() {
  Udp1.parsePacket();
  Udp2.parsePacket();
  if(var=Udp1.available()) {
    // Snipped
  }
  if(var=Udp2.available()) {
    // Snipped
  }
  delay(0);
}

I found that the LCD absolutely must be powered from 5v as its internal circuitry did not seem to generate sufficient LCD bias voltage – no setting of the contrast potentiometer resulted in any display. But with the power connected to 5v, the display comes up just fine – even with 3.3V data levels on the data lines.

The display above shows indoor (I), outdoor (O) temperatures, humidity (H) and barometric pressure (Q) at a glance. It updates whenever a new packet is received – fitting the bill for an easy display.

Of course, that is quite large and unwieldy, so I decided to make a smaller version to sit above my monitor at my desk so I could always keep an eye on the temperature. In comes an SSD1306-based OLED display for about $3 that connects via I2C for a much simpler connection.

The same information is displayed in full, but the screen is now tiny, measuring under 1″ for a 128×32 pixel display. It is, however, very bright and clear with the full labelling of each variable on display. I stuck it to the backside of the Amica board and wrapped it in heatshrink to make it a very simple unit – I’m not particularly bothered about making enclosures. Maybe not until I get a 3D printer one day.

Also, it is possible to receive the packets in a Termux session on an Android mobile phone with root access. Again, socat comes to the rescue, using the command:

socat -u -v udp4-recv:9998 /dev/null

The command is slightly complicated, mainly to enforce unidirectional transfer (so no packets are accidentally emitted), to put in verbose mode (so socat prints timestamps of reception – compensating for the lack of RTC in the devices) and to throw out the received data to /dev/null (as the verbose print to STDERR includes the raw data).

I’m sure one could build a rather simple app to display this in a nicer form – but as a person that actually likes CLI based programs, I really don’t mind this at all!

For logging purposes, I rely on my resident 24/7 Raspberry Pi (now Asus Tinkerboard) server and its scheduled cron job. The shell command it runs is:

while :
do
socat -u -v udp4-recv:9998 /dev/null 2>&1 | \
grep --line-buffered -o -P '(?<=> ).*(?=  )|\
(?<=OUTDOOR1,).*(?=#)' | while read l1; do read l2; \
echo "$l1 $l2"; done | sed --unbuffered 's/ /\,/g' - \
>> tlog/tlog$(date +"%Y_%m_%d").log
done

The rather complex command basically trims out just the time and the data values and puts it into a log file that is .csv compatible named by the date.

Problems with UDP Broadcast

In my early trials, I found that there were issues with receiving broadcasts on ESP8266 or even on my phone, with only one in about ten packets ever making it through. This is not a flaw with the ESP8266 as some seem to believe but is a consequence of the whole UDP broadcast paradigm as it relates to Wi-Fi.

On the whole, UDP is not designed for applications where delivery must be ensured. Being unacknowledged, its transportation is “best effort”. Broadcast traffic is addressed to all nodes in the subnet, thus for wireless efficiency, broadcast is often sent once to all stations at the basic rate.

Normally, most APs chose the basic rate to be the lowest common supported rate (e.g. 1Mbit/s or 6Mbit/s on 2.4Ghz) and send any broadcast frame just once. This, unfortunately, does not guarantee that reception is successful as the packet could have been interfered with and the frames could have been sent while a device is in power saving mode (if the frames are not buffered, which some APs do not).

It also points to an issue with the whole basic-rate scheme, as a single broadcast packet can really bog down the network as it must be transmitted at the basic rate which might be a lot slower (>10x) than the actual operating rate for an end device. This consumes air time – so on 2.4Ghz, just 1Mbit/s of broadcast traffic might bring a network to its knees, whereas if everything was unicast, maybe 40Mbit/s of traffic could have been carried.

In the case of Mikrotik devices, you have some options to “fix” this.

With multicast helper turned to full, and multicast buffering turned on, the broadcast frames are sent across the air as unicast frames instead. The AP will buffer the broadcast frames and make a copy sent unicast to each connected device. This has the benefit that this delivery is acknowledged (as a 1:1 transmission) at the 802.11 physical level and will run at the link rate rather than basic rate (thus potentially saving air time if you have few clients).

After changing this setting, everything worked just fine. For other APs, I’ve had mixed luck with broadcast UDP reception, thus I can’t really provide a solution, but the ESP8266 is not necessarily the one at fault if you can’t make it work.

Conclusion

As a replacement for the 433Mhz systems which have served me well in the past, the Wi-Fi based alternatives are very much an improvement even despite the ad-hoc nature of the solution. Being integrated into the one IP network means that the temperature data is available on all devices connected to the network at a miniscule cost to bandwidth. With data that doesn’t need assured delivery, UDP broadcast as a connection-less protocol makes for ease of implementation of as many displays/receivers/loggers as needed with no changes to the data source. Spinning up a number of these, it is possible to separate streams by using different port numbers or simply by changing the header.

The downside is that this system doesn’t have any security baked in. Any broadcast or targeted-t0-display emission on the same port number that has a format similar to that used by an actual sensor will cause the displays to show incorrect values. However, this won’t cause any damage to the sensor or the displays, just inconvenience. In the case of most home networks with no malicious actors, this shouldn’t be a problem.

The other downside is potentially waking up a number of Wi-Fi connected devices when they are asleep to receive the broadcast, even though they’re not actively in need of the temperature information. To fix this, it could be possible to change the system to a subscription-based multicast, but then you might be fighting even more quirks in IGMP snooping etc. In my opinion, simplest is best, so I’ve stuck with broadcasting.

In the case of my outdoor sensor, due to the designed transmission duty cycle, the power consumption is quite a bit higher than that of the 433Mhz system. This meant that I had to devote an old 12V 7Ah (actually 4.5Ah usable) battery with a 5W solar panel to keep it running through even the cloudiest string of days. This is more than the two AAA’s which would often last a couple of months, but considering the battery and panel were “left-overs”, that is essentially “free” to me and was an acceptable compromise.

In the end, it provides good information that lets us know whether to open or close windows and allows us to log temperatures inside and out, as well as upstairs and downstairs. Maybe everyone should build one of their own as a weekend project …

About lui_gough

I'm a bit of a nut for electronics, computing, photography, radio, satellite and other technical hobbies. Click for more about me!
This entry was posted in Electronics and tagged , , . Bookmark the permalink.

4 Responses to Project: DIY Weather Station & Displays

  1. MUTMAN says:

    Nice. I’m building something slightly different but along the same lines.
    Using a real “BME” from here.
    https://www.ebay.com.au/itm/Digital-Sensor-Temperature-Barometric-Pressure-Sensor-BMP280-BME280-SPI-1-8-5V/312131095607

  2. darius says:

    Hi Lui,

    you looks to be the only experienced person who can give valuable hints to my problem.

    I need to have a wifi camera to look the front of my car and have Android smartphone/ tablet’s run application to watch video streamed/ broadcasted by such wifi camera.

    I would like to let another driver to watch the same video broadcasted exactly as we watch DVB-T video stream broadcasted one to many, at the same time.

    My idea is to have one standard app coming with a camera ( via Google Play) to p[air/ to connect to camera and to send API codes to camera’s mini server to enable and start video broadcasting 24/365 full time
    and have other car drivers to use sniffer like app to filter out UDP video broadcast frames from wifi noise and play them back as life video of car fronts.

    My idea is to make many Android devices to access the same video broadcasted by a single wifi camera and my connection, pairing time limited to 1 second.

    My friend advised me to avoid autoconnect feature in application, for security reasons.

    First problem is how to make more than one Android device to watch video broadcasted by a single wifi camera (not installing web server to enable multi-user access via http (known IP) protocol).

    Please let me know your opinion.

    darius

    • lui_gough says:

      Unless you’re designing your own camera … you’re likely going to need to put in a server anyway. Likewise, clients that want to join will need to associate and connect with your network anyway (which might be open/no-encryption, but they would still need to connect). In theory, you could make it work without it, but then you’re probably going to run into issues with devices that have wireless chipsets that cannot sniff traffic consistently enough to make it work.

      The problem you’re describing is best solved with multicast video distribution – in fact, this is one of the reasons VLC was actually invented. You can send the stream encapsulated as a transport stream as a UDP multicast to the network, while clients issue IGMP joining messages to subscribe to the stream and receive a copy. Any IGMP snoopers along the way (say in switches and routers) will decide whether to forward or drop the multicast depending if anyone downstream is subscribed.

      The issue is that with wireless, multicast works practically the same as broadcast as long as anyone is connected. As only one copy of the packet is sent into the air (normally), it is sent at the Wi-Fi basic rate configured in the AP which could be as low as 1Mbit/s without any guarantee of delivery. Devices which can “sleep” their Wi-Fi radios may miss these transmissions (in my experience) or they could be interfered with by other transmissions/noise/poor signal. So it might well choke the network to death by forcing low transmit rates and consuming all the air-time. One way to improve this is to use an AP that has Multicast Helper ability which turns multicast frames into several unicast frames (one for each client) which can then be both acknowledged *and* sent at the maximum PHY rate the link supports at the time.

      But there’s a reason why people prefer to set up unicast servers – it’s just much more reliable to negotiate a TCP point-to-point connection to deliver video. This way retransmissions are possible …

      – Gough

  3. darius says:

    Hi Lui,
    wrote much longer question but WP has failed to upload it.

    follow-up

    how to make many car drivers to watch the same video broadcasted by a single wifi camera ( no http server) in open wifi broadcast (or sniffer mode) ?

Leave a Reply to lui_gough Cancel reply