The past few weeks have been quite busy with radiofax monitoring – a small project inspired by the existence of KiwiSDRs and WebSDRs and a desire to preserve what little “analog” utility stations we might have left on the HF/shortwave bands. This post looks at some of the tips I have for other potential monitors and how I’ve been going about my little hobby.
The Problem with kiwifax.py
As much as kiwifax.py has been a godsend to me, there have been a few niggles about its use for DX or monitoring purposes.
For one, it is optimised for use with APT start/stop tones and relies on its detection to initiate and stop printing. Some stations are known not to use the standard tones, resulting in either missing charts entirely or not ending when charts actually finish. The other is that the tones are often not long enough to guarantee successful detection during fading propagation conditions – this results in “missing” one chart in a sequence of charts, for example. Finally, I came across RBW41 which doesn’t use APT tones at all, so how can we get that to print?
There is one way at the moment, which is to use the -F force flag, but this only forces printing from the execution of the program up until the maximum number of lines. You could set the maximum number of lines to a fairly large value, but I find values over about 10,000 cause unacceptable wear on the SSD (as it seems to read the whole file, append a line and write the whole PNG back to disk) and slows down the decoding process dramatically.
As a result, modifications to the kiwifax.py script are probably the best way to overcome this issue and instead turn it into a continuously printing system. This way, it would be easy to leave it running and come back and not miss anything no matter how weak – and with a little post-processing, we can still determine rough start/end times and fix the vertical alignment.
I will repeat what I have mentioned in the past – kiwifax.py (printing or not) occupies one of the receiver slots on the target KiwiSDR. As many are operating in 3-user (20kHz), 4-user (classic) or 8-user (maximum) configurations, responsible use of kiwifax.py is encouraged to prevent occupying too many slots (i.e. concurrent instances) or continuously occupying slots denying others access to the SDR. Also note the existence of per-day maximum use times that some SDRs have which will restrict your capability to monitor “around the clock” – careful selection of SDRs which are less-frequently used, with no such time-outs and/or with higher numbers of slots would probably be best.
To “fix” kiwisdr.py and turn it into something useful for monitoring/DX purposes, it’s necessary to make a few changes.
The first change was to remove the whole start/stop score decision-making code. Deleting all the code highlighted above would cause the program to remain in whatever mode it is in and eliminate its ability to detect whether it should start/stop or phase.
To ensure it would be printing, it would probably be worth “unwrapping” the self._switch_state(‘printing’) line from the if statement. By removing the if-statement, the program will start up as if the force flag has been provided.
Then, we need to pay attention to the _process_row function – by looking at the condition for maximum height, the system changes the state to idle. But we don’t want that. If we just removed it, the PNG would keep growing and growing to no end – so we instead want to start a new file – hence add the two lines above: self._new_roll() and self._switch_state(‘printing’). There is probably a lot of unnecessary code with handling the other conditions, but this was the simplest way without breaking the program to make it do what I wanted it to do.
It is probably also worth changing the “default:” max_height value to something larger to ensure that fewer charts are “broken” across .png files. I normally use either 5,000 or 10,000 – coding it here means you don’t have to supply the command line option.
The good thing is that each PNG slice is “continuous” with the next – if a chart breaks across PNG files, there is no lines lost and you can piece it back together in an image editor. Each PNG slice is labelled with the start time – to know what time the actual fax started, just count the number of lines from the top and divide it by the scanning rate (e.g. 120 lines per minute most commonly).
The only downside seems to be that the code still bails on certain types of exception. While you could handle that within the code, I decided to wrap the whole thing in a .bat file that has a “wait” time before re-invoking the script allowing for unattended monitoring sessions. This is necessary as I’m currently connected via LTE where the session breaks very frequently.
With the right paths coded in the .bat file, you could even have multiple instances in separate folders monitoring the same station across different KiwiSDR receivers. I’ve even set the title of the window to something descriptive to help me keep track of all the instances I’m running. Just note that if you are doing so in the same folder, you might find filename conflicts are the result causing one instance of kiwifax.py to overwrite the results from another – separate folders are recommended.
Having the code available and free to modify is one of the great advantages of open source software – I’m very thankful that when I have a problem, I also have the tools to solve it!
My Radiofax DX/Documentation Workflow
It occurred to me that I never really mentioned exactly how I go about doing my radiofax monitoring and documentation, so it’s probably a good time to do so and explain why it sometimes takes so long …
The above basic flowchart was hurriedly drawn on a sticky-note but describes exactly the steps I take.
- The first step is to research the frequencies, times, transmission power, scanning rate/IOC and locations for the expected transmission. By knowing these parameters, it becomes easier to plan your reception.
- Knowing the location of transmission, one can then look on the map of SDR receivers to find the best candidates by location and see how many alternatives there are. It pays to keep a little notebook to take notes about these receivers. When a known scheduled transmission is on, open up these candidate SDRs and note their web addresses, signal strength, apparent signal to noise and whether there are any “daily use limits”. Sometimes due to out of date schedules, nothing is seen, so we must persist and check back at common fax times (e.g. on the hour) and stepping through the list of known active frequencies.
- Once a first sighting of the signal is spotted, then we know the target station is active and at what time. As a result, we can receive a chart or two to determine whether our information is correct regarding start/stop tones and scan rates. This might take a number of days as we may only know of very few known times of activity.
- The parameters for reception are refined by understanding which frequencies work best with which SDRs, which SDRs give good signals and the slant correction factor gets dialled in progressively.
- All the while, the results are being recorded in separate folders with sometimes overlapping reception via multiple receivers. All of the receptions are then collected and analysed to weed out duplicate receptions by selecting the best result, duplicate chart types by looking at the chart contents and fixing some reception issues (chopped images). Once they have been processed, then they can be uploaded and blogged about along with some research on the side.
As a result, it’s a fairly simple workflow but one that can easily take several weeks for a station especially when signals are being unkind or stations follow erratic schedules. The biggest impact to this invariably comes from stations where only very few candidate receivers exist and they are unreliable – going online and offline very frequently, resulting in a significant amount of frustration as I attempt to find a substitute (rarely possible) or wait it out until they’re online at just the right time with a free slot.
Slant Correction Factors
It is a bit of a miracle that analog radiofax works as well as it does, but one of the key problems often encountered is image slant. This is due to the synchronization of clocks being slightly different on the transmitter versus the receiver – for longer charts to maintain a straight alignment, the synchronization requirements can be quite stringent.
For example, a chart is 1809 pixels wide and each line is 500ms long in 120LPM mode, making each pixel approximately 276.4us. If a short chart like the one NWS/NOAA sends which takes 10 minutes to transmit is to be received in perfect alignment, then the allowable error is only around half-a-pixel or 138.2us in 10 minutes or 0.23ppm! Although to be honest, we could probably tolerate slight imperfections in the slant as long as the margins are not breached.
Lets be a little more generous – lets say we can “spoil” the complete margin area. From my calculations, the margin is 5% of the image line area based on BoM transmission parameters (475ms image, 25ms margin). We can now allow around 91px of slippage or 25ms, which means for a 10 minute fax, a stability of about 42ppm. When you consider the length of the Kyodo Fax news broadcasts, the allowable error might be even less, which is pretty amazing.
This is why slant correction factors are required which allow the decoding software to compensate for differences in timing. Traditionally with a conventional radio and sound card, sharing these figures are nearly meaningless as the error was frequently dominated by the sound card’s own clock error unless it had been calibrated out.
But in the case of kiwifax.py, as many KiwiSDR receivers are GPS-disciplined, this means they are pretty much as accurate as an absolute time reference. As a result, the same slant correction factors can be used across multiple GPS-locked receivers for the same results.
To aid other radiofax enthusiasts – I share my slant correction factors for stations I have received. I do so for your convenience to reduce the slant to acceptable levels out-of-the-box, but I cannot guarantee the accuracy of the factors due to differences in KiwiSDR and transmitting station drifts. These factors have been derived mostly from trial and error to get “close enough”, and are not 100% guaranteed to remove all slant. Note that in my observation, all stations within a given network typically exhibit identical slant behaviour.
Callsign Correction --------------- ---------- CBM/CBV -90.0 DDH/DDK -12.0 GYA +34.0 HLL2 -18.0 HSW64 -11.0 NOJ/NMC/NMF/NMG -4.0 KVM70 -4.0 JFX 0.0 JJC/9VF +3.2 JMH 0.0 RBW41 -17.0 VCO -24.0 ZKLF +11.0
Radiofax Station Status [10th February 2019]
I recently came across the listings from hffax.de from 2002 and a few random frequency lists. Owing to the unreliability and age of the data, I did cross-reference stations which are currently listed, removed those known to be no longer in existence or where more recent data is available.
This now leaves me with a new list split into two – below is the latest compiled list of the stations I know about with the activity status based on my recent monitoring efforts over the December 2018 to February 2019 period. The second portion are based on the historical data obtained and are very unlikely to still be alive – I just haven’t taken the time to confirm they are definitely offline.
Station Status as at 10th February 2019 (based on my observations) CALL COUNTRY STATUS ---- ----------------- -------------------------------------------- ZSJ South Africa ? Unknown, Not Heard JMH Japan - Alive and Well (Published) ??? Chukota Peninsula ? Unknown, Not Heard BMF Taiwan X Confirmed OFFLINE (see above) HLL2 South Korea - Alive and Well (Published) HSW64 Thailand - Alive and Well (Published) JJC Japan - Alive and Well (Published) 9VF Singapore ? Unknown (Relay for JJC) GYA Persian Gulf X Reported Inactive, Not Heard PWZ33 Brazil ? Unknown, Not Heard CB? Chile - Alive and Well (Published) CFH Canada X Reported Inactive, Not Heard VFF Canada ? Seasonal, Not Heard VFR Canada ? Seasonal, Not Heard VCO Canada - Alive and Well (Published) VFA Canada ? Seasonal, Not Heard NOJ USA - Alive and Well (Published) NMC USA - Alive and Well (Published) NMG USA - Alive and Well (Published) NMF USA - Alive and Well (Published) VM? Australia - Alive but (Temporarily) Unwell ZKLF New Zealand - Alive and Well (Published) KVM70 USA - Alive and Well (Published) SVJ4 Greece - Alive and Well (Published) RBW4? Russia - Alive and Well (Published) DD?? Germany - Alive and Well (Published) GYA United Kingdom - Alive and Well (Published) ??? Argentina ? Unknown, Not Heard LSB Argentina ? Unknown, Not Heard CAN6D Chile ? Unknown, Not Heard OXT Denmark ? Unknown, Not Heard ATP?? India X Reported Inactive, Not Heard 5YE? Kenya ? Unknown, Not Heard 6VU Senegal ? Unknown, Not Heard RTH Uzbekistan ? Unknown, Not Heard JFX Japan - Alive and Well (Published) * Published indicates a posting has already been put online for with reception in 2019. For other "alive" stations, a posting will arrive once the fax images have been processed and sorted for publication. HISTORICAL STATIONS from hffax.de (Most Likely Inactive) CALL COUNTRY STATUS ---- ----------------- -------------------------------------------- YMA20 Turkey ? Unknown R??7? Russia ? Unknown BAF?? China ? Unknown VLM Argentina ? Unknown ??? Argentina ? Unknown 3MA?? Taiwan ? Unknown HM??? North Korea ? Unknown, Not Heard R??7? Ukraine ? Unknown CZW Canada ? Unknown 3SD China ? Unknown LOR Argentina ? Unknown BDF China ? Unknown VNA Vietnam ? Unknown, Not Heard
That being said, there is one station on this list I am trying my best to receive and I have some evidence for it being still active. However, I have not yet updated this to reflect this because it’s not an easy station to catch with very limited possibilities to receive – so I don’t want to cause an “SDR traffic jam” just yet until I’ve concluded my investigation.
But as usual … there is more to look forward to – they just won’t be easy stations to find.