BOM Weatherfax, Fax, VoIP and FoIP
A commentary on VoIP and Faxing complications, and the BOM Weatherfax service.
Where should we begin this story other than by starting at the Bureau of Meterology. Back when I was in high school, I used to read the SMH (in paper form of course) and every time I'd look at the weather page, there'd be a number to call if you wanted to receive more information about weather by fax. I was pretty savvy at the time, and I recognised all 1800 numbers were free - and this was no exception. The number that was published was 1800 630 100. I can still remember when I used to call it - the first thing you would hear is a recorded male voice telling you "Welcome to Informatel Weatherfax. This, is a free call. Press the start button on your fax machine, now." Note the placement of the commas which indicate the peculiar way in which the annoucement paused in certain locations. It's a shame I didn't record this ... I can still hear it going in my head. It was surprising to me to think that there was a faxback service which was a free call, after all, these services are almost always classified as premium call services where you would be billed per call or per the minute. After receiving the directory - I looked at the top line, and sure enough, the computer at the other end placed the "standard" header on the top stating that it was billed per minute. I knew that was rubbish. But I got a 4 page fax, with numbers for all their services. Most of them were premium call numbers for other products, but there were some numbers for more detailed services which were also free call numbers. They would be state-wide directories for more detailed listings of services, some for farming products etc. It was a handy number to know - sure I didn't really "buy" any of their products, but it would be nice to test a fax machine with it, test computer faxing software and modems to see that they would work. It would even come in handy, especially since it was multiple pages, to test whether certain lines were capable of at least 14400bps. It was like a Telstra FOLDS test which I could do with my own equipment, anywhere, free of charge (to me at least).
But then things started to change. Subsequent readings of the newspaper, when I had originally forgotten the number, revealed that the number was no longer published in the paper. I found a sheet of paper which I had the number on, and when I rung it back then, it still worked so I thought nothing of it. In fact, you can still see references to the number some places online, and even today in fax product 266 on page 2.
Hence begins this, rather unusual, presevation effort. As you would probably have already seen from my dialup modem sounds pages, I had recently converted to Naked ADSL. What this means is that I don't have an active POTS line anymore, instead, I place my calls through a VoIP service. I myself use a Linksys Sipura PAP2T VoIP ATA as it is one of the most flexible ATAs on the market. I had bought it long before I even had broadband or VoIP just to do experimentation. I am well familiar with the many options it has available, and needless to say, compared with the two other ATAs I had used (a Zyxel Prestige 2302R and a Netcomm V100), it was the best. I had little problem running my own LAN VoIP system - I could call my fax machine downstairs from a box upstairs and fax it a document like it was a printer. It was great for simulating a POTS line so I could do testing on modems and other telephony devices, and even use a cordless phone base as a radio transmitter. All was well when I was experimenting at home.
Now that VoIP is getting more and more common, more and more users are having some sort of experience with it - whether this be in a corporate situation with company desk phones transitioning from propietary PBX systems to those at home getting Naked ADSL like myself. There are a few caveats to this - which some people may or may not be aware of. Many a time, the service providers themselves won't let you call all sorts of numbers on the system - numbers like 18, 13, 19 and even 000 sometimes will not be accepted depending on the provider. This could be a real dangerous situation in the case of an emergency, or a minor inconvenience if you need to reach a company which uses a 13 number.
Secondly, most VoIP services at the moment tend to prefer you choose a compressed audio configuration. What this means is that you are often told to configure your ATA so that you use the G.729a codec to compress the audio down to 8kbps of data. This is for a few reasons - that being that the G.729 codec provides similar quality (it's Mean Opinion Score (or MOS) is only slightly less than uncompressed G.711a/u) and it saves you and the provider some bandwidth. This has additional benefits in the fact that less data means that there's less of a chance for loss. Unfortunately, this often means there's some tradeoff for sound quality, which, while not that noticeable for speech, breaks all ability for data modems to make a connection. The compressed codecs, be it G.729, G.723, G.726, Speex, Ogg or whatever, are just not optimized for the passage of data signals, which are (depending on the modulation used) pure tones or white noise-like randomness. As Shannon's Theorem would dictate - there's no way to carry a 56k connection over a compressed codec that only codes at 8kbit/s! Also note the caveat that while compression does reduce the data down - there are lots of overheads to the VoIP system especially when using UDP to carry the payload data. You can find tables online about this - but essentially, most ATA's will packetise every 20 or 30ms of sound into a separate packet, each with a header and (optionally) a checksum which increases the data rate. The raw ethernet rate for a G.729a call is roughly 20-30kbit/s even though the payload data is just 8kbit/s. Just some food for thought there. Also do note that since the data is packetised into discrete packets, there cannot be any decoding of any sound until the whole packet has arrived - in the ideal circumstance, with no coding algorithmic delay, a packet which contains 20ms of audio means 20ms of delay in the ideal case of no jitter. We'll elaborate more on this in the next few paragraphs.
One of the great annoyances of VoIP which many many users seem to experience is delay. The feeling that you're in some sort of time chamber, with echo in the worst case, or maybe just the fact that you keep talking over the guy at the other end. Delay is the enemy, and unfortunately, a combination of situations means that we can't eliminate it entirely, but we could reduce it with a few tradeoffs. One source of delay is the compression that is being used. Since the compressor needs some data to formulate its trends, it tends to work in frames of discrete lengths - and it cannot produce any output until a whole frame's duration of data has been coded. This is termed algorithmic delay and it can be 30ms for G.729 for example. Secondly, is the packetization - depending on the packet rate you choose, most devices use 20 or 30ms as their default, which can lead to more delays especially if the packetization rate doesn't match the frame rate causing fragmentation. Add onto this, the effect of network jitter. This is where VoIP can begin to fall to bits. Depending on the network on which it is delivered, the state of the links between your gateway and your ATA, you will find your UDP packets traversing networks, routers and computers which are shared by others. Since the internet wasn't really built to deliver information in a time-sensitive manner in the first place, depending on the resources of the routers on the way and the network capacity, you may find that there is variance in the time of delivery. So, in the case of VoIP, we have to account for the delay (latency) of the packet traversing the network, but we also have to store some information (buffer) to prevent interruption in the case of varying delays. Unfortunately, over the public internet, the jitter itself could be anywhere from 5 to 100ms depending on the provider and link conditions. The buffer should be as large as the majority of the delays so that the delivery of audio is smooth - it's not unusual to see jitter buffers at 40-50ms in deployed ATAs, so meaning the total delay could be 50ms in the jitter buffer, 30ms in the coding, and 20ms due to the packetization mismatch giving us a 100ms delay. While this doesn't sound like much, consider that this is only one way. Say a VoIP to VoIP call through two separate gateways linked together via the PSTN and you're looking at a round trip time of 200ms. This could easily cause talkover issues and general annoyance. Cellular calls suffer from this to a limited extent because of the way the GSM compression works in frames, but in VoIP, this can be worse. For the record, according to my data modems, the general latency of a wireline call to Sydney from my house is roughly 15ms at worst, and sometimes it would be just 7ms.
Unfortunately, some ATAs are not very flexible, and don't leave you many options. The Sipura isn't one of them. We have control over many of these variables and we can do our best to optimize them out. For most, I do not recommend you go playing around with your ATA parameters because you could break something instead of fix it, and there might be restrictions imposed by your gateway, I have had fun getting the most out of my VoIP by playing with the settings. First of all, codecs - I prefer G.711a. In fact, I've configured my ATA to use only G.711a. This is essentially an uncompressed codec, which sends 8 bit samples at 8khz sample rate down the line. It has no algorithmic delay whatsoever, but it does incur a bandwidth cost of 64kbit/s of payload. It's significantly more than G.729a, but it does offer the quality required for data transmission, and it does reduce the delay. But do remember, while you can change the codec on your end, you might have to enable "Use Preferred Codec Only" to force the other end to send data back using the codec you've specified. In the worst case, you might find that nothing works anymore, and that you get silence or busy calls since the gateway is unable to fulfil the request using the codec you have specified, so you do need to experiment a bit.
Another thing I've done is I've tuned the packetizer to packet up the data in 10ms increments. Most VSPs will tell you to use the 20 or 30ms defaults - it's probably good for a general voice call and saves on the overheads, but I don't really care so much about overhead. Some VSP gateways won't handle non-standard packet rates so it's another one of those things you'll have to try. What I want is a good, low delay call - this is critical for data. The reason is, the ITU has specified specific timings for certain fax phases - while most machines have a wider tolerance than the specification requires, you have a better chance of success if you keep the quality as high as you can, and the delay as low as you can. This ensures that the signals can fall into the required timing windows - some are no wider than 40ms!
Now, we pay our attention to the jitter buffer - sure, one might be thinking, by reducing the jitter buffer, we can reduce the delay and get what we want. True on some level, but this is where going the other way is a better option. I've set my Jitter Level to High and my Jitter Buffer Adaptive to up only. In a data signal, we essentially don't want an adaptive jitter buffer. An adaptive jitter buffer will grow as the jitter increases on the network and shrinks as the jitter decreases. In a voice call, this is sometimes noticeable as a clipped syllable or a reverb or some wierd sound, or sometimes none at all (if the adjustment happens during silence). In a data call, this would be disasterous - an adjustment would cause lost data or repeated data which would possibly confuse a modem. Some people's rationale is to set the Jitter Buffer to non adaptive - and my reasoning is that it isn't a good idea. In the case that jitter rises above the buffer level, what you will find is that that the audio will be broken and continue to be broken while the jitter rises above the buffer level, every time. Instead, by adjusting upwards only, the jitter buffer will reach a level where it accomodates practically all jitter events and stay there - this is what I would call the "true" minimum jitter level for a data call. This isn't so much of a problem since I find the "discovery" of this jitter level happens usually near the beginning of the call, and so the modems can retrain a few times and then sit stable.
Other options to be careful about are the VAD or Voice Activity Detection. This is a setting that is usually on, by default, which senses if there is any sound and will not transmit any data in case of silence. Good in theory, bad in practice, since the time taken to detect and the possible clipping of sounds would cause a data connection to go flakey. So it's disabled. Along with adaptive Echo Cancellation which attempts to "search" for an echo in the data and digitally remove it to prevent it. Also good in theory, but since data contains many repetitive patterns of tones - most echo cancellers cause data corruption instead. Unfortunately, you can change the setting on your end, but that doesn't mean the setting on the other end will be the same. You can see this happening in the call which is linked later on - the first few identification segments from the remote fax are totally corrupted by the adjustment of the remote Echo Canceller, until it finally drops out and leaves us to get on going! Also, there's an option on some ATAs for Comfort Noise Generation - that is, a slight hiss in the background to let you know you're still connected. I've turned it off - it's nothing but a SNR destroying feature, not a good idea for data calls.
Okay, so so far I've been yabbering on about VoIP problems and data calls - but why, you might ask? Well, remember where I started - it was all about faxing. Most VoIP providers do not officially support faxing over IP - my VSP is no exception, but that being said, someone has already done something about this problem. The answer, it turns out, is called T.38 - otherwise known as the standard for faxing over IP. It's a different sort of VoIP standard, and it's available on the Sipura ATA. Unfortunately, when I tried it on my VSP, when the REINVITE command was sent, the other end didn't know what to do and went silent. So unfortunately the "solution" to the problem wasn't implemented. Essentially T.38 enabled ATA's will listen out for the CED or CNG fax tones, and when it hears them, will switch over to an uncompressed codec. Furthermore, it changes the data transmission so that every line is sent over the UDP end twice, to avoid losses, and commands are sent to produce the HDLC data stream which is used by the faxes to negotiate the speed, mode, pages and acknowledgement. All of this is meant to solve the problem of delays becoming out of spec for the fax machines and ensure seamless working. I can tell you, it does work when going from ATA to ATA, just not my VSP's gateway. They really should support this feature one day ...
So yes, faxing, a bit old yeah? True. It's a bit old, some wonder why it's still around. I personally find them interesting, as with any other form of data transaction. Still, some companies are out there, seemingly bound by legal requirements to have things faxed. E-mail just isn't good enough, scans to PDF at a higher quality will not suffice. It's a bit ironic now that we still use fax when landlines are on the way down, mobiles and internet are taking over the world by storm and snail mail is reserved only for the physical stuff. That being said, since there are some places where the only options are mail or fax - it would be handy to have an ability to fax even without a real landline. Hence all this mucking about to try and get it to work.
So first thing after all those tweaks - I wanted to know - just how well does it work, and does it even work? I fire up my trusty copy of Winfax only to find it stuck with some internal error on this machine that only a contortionist could unravel. So back to my Windows 3.11 roots - back to a piece of software called Cheyenne Bitware. It still works, sort of, under XP. I try to place a poll receive call to the 1800 number only to find it responding busy. I try again, busy again. And again ... and again! I then call it from my mobile, and there, I got to hear the bad news. "The number you have dialled has been disconnected. Please check the number and try again." It was gone. It wasn't published on the Bureau's site anymore. But the service was still available, on a 1902 number only (or so that's what they want you to think).
Since I have some allowance for free national calling, and no ability to call 1902, I look and lo and behold there's a STD number just waiting to be called. Sure it says for international and satellite users, but I don't mind giving it a try. And funnily enough, it picks up, answers and sends. Yay. Fax received. It works, but not without trouble. Listening to the sound below - you can hear the adaptive echo canceller totally screwing up the HDLC data at the beginning. Secondly, not heard in this recording is that about one in eight faxes fail due to jitter buffer adaptation during Phase B or Phase D (i.e. HDLC data) causing some unusual problem. And finally, the problem of corrupted scan lines. Also, another jitter buffer related issue. Essentially, the fax got through, but with a few lines missing. Most fax machines handle lost or damaged scanlines gracefully by just reprinting the previous scanline, however, even in Modified Huffman (MH) transmission mode, there are some faxes which will just blank the rest of the page on the first bad scan line it hits. I guess I can't have all that much confidence in my system - a T.38 complaint gateway should solve all of it. What a shame it's not a complete success - when I had a real landline, bad lines were zero ALWAYS. It was practically a given. That being said, I realized that the number of services available by fax has dwindled significantly. There is now barely one page of directory when there used to be four (and subdirectories). That being said, I have a feeling that this service might not be around for much longer, so today, I called every single fax number listed and got one of each - for posterity. Call me crazy, but that's what I do to remember what was, once, a very special, highly-informative service.
Weather By Fax Services
Dial 1902 935 xxx (cost at $1.38 per minute, more from International, satellite or mobiles) or +61 3 9273 8xxx (for international or satellite users) for the services below:Note that the provided linked files are samples only (obtained September 11, 2010) and are NOT updated. They're there to give you an idea of what was available from this fax service. Hear what it sounds like to call the Informatel Service from a VoIP ATA.
220 NSW Coastal Waters
230 Victorian Coastal Waters
280 South Australia Coastal Waters
710 Queensland Boating Weather
711 SE Queensland Boating Weather
720 Perth Local Waters
290 WA Coastal Waters
214 NT Coastal Waters
240 Tasmanian Coastal Waters
201 Australian Region Infrared
202 Northeast Australia Infrared
204 WA Infrared
Tropical Cyclone Threat Maps
218 Northern Territory
297 Western Australia
002 Four Day Forecast Charts + Satellite Image
266 Australian Swell Charts
210 Mean Sea Level Analysis
292 WA and Perth Forecast (old 292 + 429)