Unfortunately, the cutting edge can quickly become the bleeding edge. Configuration of the codecs was not intuitive, and models and features appeared and disappeared far more quickly than the manufacturer's technical assistance could cover. Maybe it was inevitable that as quickly as it took off, two headline-grabbing events brought our expanding system to its knees.
First, Superstorm Sandy clobbered the Northeast, and the nation's most populated region took a noticeable hit to its Internet infrastructure. For several months after Sandy, our business-class Internet providers could not consistently provide the bandwidth we normally counted on from them. We could go from stable operation to nothing within seconds, then back to stable within hours...or days. The brave new world of public Internet-based audio proved remarkably fragile.
Even half-a-year later, reliability still fluctuates as major Internet providers work to replace damaged fiber and old copper in places like lower Manhattan, and our attempts at finding wireless bandwidth out of our partner venues there have proven fruitless because of Manhattan's famous canyons of tall buildings.
The other news headline that unexpectedly crossed our path was the increase in Internet hacking and denial of service type attacks. Our first inkling of how hackers could hurt us came shortly after installing a fully digital transmitter at one of our stations along the Jersey Shore just weeks before Sandy hit. Very shortly, our major Internet provider there notified us (with less than five minutes warning) they were pulling the plug on our broadband connection because their security department had detected a spam phishing operation emanating from our transmitter building.
Not coincidentally, our operation sat inside the Internet provider's compound, literally hardwired to the head-end of their massive broadband fiber network. The hackers had a huge broadband pipe at their disposal, and a Linux-base control system in our new transmitter to act as their spam server. Life was good...for them.
The vulnerabilities in that particular transmitter had been well publicized in the hacker community (somehow, we missed reading it!), with all the hacking information readily available on the Internet. Unfortunately, without any safeguards or warning from the transmitter's manufacturer, we were just one more sitting duck.
That brief and blunt primer taught us two things about how the digital age has changed broadcasting and the requirements for broadcast engineering. Broadcast manufacturers need to take digital security seriously and practice transparency with their clients about known problems, and IT coursework -- including system security -- is a must for broadcast engineers.
Since our transmitter problem, we have found our IP codec system is as hacker-vulnerable as our transmission systems. The first generation of the codec units -- and these legacy units account for the majority of the units in our system -- have no real security. Their readily available software control system allows anybody who knows the static IP on your codec to easily access it, no password required. In fact, no password protection is available.
It's a throwback to the days of the dial-up remote control system where knowing the right phone number and punching the default password (1-2-3-4-5) into the keypad would get you into many - if not most - transmitter remote control systems in the country. The only differences are that it's easier for hackers to scan for static IPs than it is to find transmitter telemetry system phone numbers, and the dial-up remote controls at least had password protection that could be changed.
Coupled with an already damaged Internet infrastructure, the hacking problem has made our IP system too risky to keep and too costly to update. Dedicated loops quickly make the extensive system more costly than satellite. Sending IT security experts around our network to harden each site is too costly for a public station that must go through public bid procedures before it can cough twice.
So, we're headed back to satellite for our main network distribution. Since we left the geo-stationary orbit guys, their technology has advanced considerably. We can get reasonably good audio for both our classical and jazz networks in the same bandwidth we once used for just mediocre audio for one network. For the time being, satellite audio appears to be beyond the reach of all but state-sponsored hacking operations. When state-sponsored hackers finally manage to seize control of satellite systems, I can only hope the satellite vendor will be more forthcoming with information and solutions than our transmitter and codec vendors.
Meanwhile, we have more than a dozen high-quality, low-security audio codecs that we will use in situations where we can control problems quickly if they occur. Concert venues are where they will serve until they die. In three-hour stints, the known problems are manageable, and they do sound great.
Peter Fretwell is general manager of The Classical Network, based in New Jersey. He is co-author of Lessons from the Hanoi Hilton: Six Characteristics of High-Performance Teams published by Naval Institute Press.