Having worked off-and-on as a network administrator for the past 15+ years, I am often reminded of just what a life-saver the cloud and managed services can be. This past weekend, I happened to step into a situation where ignoring these benefits led to a complete nightmare for one "unlucky" customer.
I was called-in as a favor on what basically turned out to be a disaster recovery operation. The disaster did not come in the way of a flood, fire or earthquake though. It came in the form of a rogue IT employee who basically took down his employer's entire telecommunications network upon his sudden, forced exit from the company.
To protect the innocent, I will not get into too many details about the customer, other than state the fact that it was an (otherwise) fairly modern medical office that depended upon a single employee to handle it servers, network connectivity, hardware, software, back-ups and phone service. What could possibly go wrong?
Hardware, software and telecom services were spread out all over the place. No standardization of operating systems, no network redundancy, pretty much no firewall to speak of, no way to monitor or shape network utilization, a mix of analog phone lines from one company and a VoIP service used for other lines from a different company, a wi-fi network that had access to HIPAA sensitive files, unencrypted and incomplete data backups stored on servers overseas and sky-high expenses on everything including software, telecom hardware, broadband service and external hosting providers. To complicate matters, the IT director had been summarily dismissed before the employer managed to shut off his remote access or recover any of the numerous administrative passwords.
To put it mildly, this customer is in deep trouble. It will take many weeks and thousands of dollars to unravel the enormous problems that a single employee has been allowed to cause. That brings me to the actual point of this article, which is what could have been done to prevent such a nightmare. This is where the real benefits of the cloud and managed services begin to seep in. The below are a list of measures that could have been taken to avoid such an ugly and costly incident. The next time you are consulting a customer on the benefits of moving their business to the cloud and a managed service provider, you may want to keep this story and list in mind.
- A cloud-based phone system operated by a managed service provider could have assured the customer of continued communications without having to pull the plug on their entire network. A managed firewall and monitored network could easily allow access to vital services while blocking unauthorized access from the outside.
- Consolidating services with a single provider would have dramatically lessened the time needed to get in touch with a number of different providers to reset passwords, revoke account access and redirect phone calls.
- A managed broadband circuit with monitoring would have ensured that the customer was subscribed to an adequate amount of bandwidth and perhaps even saved them some money. It definitely would have improved the quality of their VoIP calls, which were previously said to be an issue. As it was, all traffic was basically treated the same and the customer had no idea if they were paying for too much or too little service.
- Integrated Software as a Service could have done away with a multitude of licensing and hardware issues that were found, provided superior customer support and protected the customer from security issues caused by obsolete operating systems and office software. It would also have lessened the impact of losing passwords known only to a single administrator.
- The cloud could have provided complete, encrypted backups of sensitive customer data with greater privacy and perhaps even at a lower cost.
- Outsourced IT management quite possibly would have prevented a single employee from obtaining such a stranglehold on this customer's operations by carefully screening prospective employees and implementing policies that would allow them to easily be replaced without downtime or concern of security breaches.
- Using a knowledgeable agent or cloud broker could have saved this customer thousands of dollars by consolidating lines, preparing competitive quotes from a number of different carriers and coordinating efforts between vendors. An experienced agent would have also recommended a suitable back-up plan in the event that the customer's primary carrier suffered a service disruption.
- Many carriers and vendors have divisions that specialize in helping customers comply with HIPAA regulations, thus protecting their patient's privacy, steering clear of legal problems and safeguarding their licenses. Using one of these providers may have saved them from many headaches that could still result from this incident.
- Physical security of the customer's building was also compromised as a direct result, as the employee evidently had a door key and passcode for the alarm system. Cloud providers and managed service providers neither want nor need this level of access. In addition to cost of addressing IT issues, the customer also had to employ an armed guard, locksmith, private investigator and paid for a field call from their alarm company. Again, none of this would have been necessary if the customer had relied upon the cloud, a broker and a managed service provider.
- Although the customer had a mix of both traditional analog lines and VoIP service (albeit from different providers), neither were integrated with the other to provide any sort of redundancy. Thus, they still had a working lines for faxes, a credit card terminal and alarm system, but they were otherwise unable to make or receive calls until a new firewall was installed and network connectivity was restored. Obviously, bundling of services with a single, integrated provider or adding vendor-recommended hardware to their system could have prevented their blackout of phone service to a large degree.
Unfortunately, this was an "accident" that never had to happen. While the human factor can never be 100% accounted for, using the appropriate technology for the job could have saved this customer an enormous amount of time, money, legal headaches and perhaps even loss of reputation.
When you consider that taking full advantage of the cloud and managed services would have most likely been less expensive for the customer to begin with, you have to wonder why agents and cloud brokers aren't in even higher demand than they are now.
William Van Hefner