Most people imagine SCADA (Supervisory Control and Data Acquisition) as a single computer interface overseeing a utility’s entire remote infrastructure. The reality more often resembles the system found in Ocala, Fla., which was developed over several years by a variety of system integrators.
The budget for the original system only paid for remote telemetry units (RTUs) with proprietary human machine interface (HMI) software at the water and wastewater treatment plants. Later, a different integrator installed another brand of software with programmable logic controllers (PLCs) in the utility’s lift station network. Furthermore, a third combination of hardware and software was later added to irrigation sites. The limited yearly capital budget meant that the municipality had to continually choose between more bolt-on components and the steep cost of starting over with a single system. This resulted in a variety of issues:
- It was expensive to train staff on three different applications;
- Each system could only see part of the process;
- There was no central historical database for reporting and analysis;
- Many servers were required to ensure redundant servers and services;
- Multiple sets of software renewal fees and support contracts.
In 1995, the Ocala utility began a 15-year transformation to a unified SCADA application. Once it found a way to keep all the PLCs and RTUs, it was able to convert one system at a time so that each project did not exceed the yearly budget. The final system took advantage of a new approach to distributed historical data management that allowed the utility to lower costs by reducing the number of manned hours required at some of its remote sites.
What is SCADA?
Every SCADA application is different. However, most share the following core elements:
- Monitoring & control devices to collect data and control equipment at remote sites (e.g. PLCs, RTUs and pump controllers).
- A communication network to relay commands and bring back process data.
- HMI software to provide a graphic representation of the process.
- A database of I/O points (tags) that define important objects in the system.
- A database of historical process information logged by the system.
- A reporting package to generate scheduled or ad-hoc reports.
- An alarm management system to alert on-call personnel and emergency responders to system problems.
1995: Three Systems
To meet the ever-growing needs of its 65,000 customers, the Ocala Water & Sewer Department operates a 25-mgd (million gallons per day) water plant, three water reclamation facilities with a total capacity of 13 mgd, 125 lift stations, approximately 2,000 acres of spray irrigation systems and 15 other assorted water and wastewater sites.
According to Darryl Muse, Ocala’s deputy director of utility systems, the utility became increasingly concerned about the rising cost of maintaining three SCADA systems.
“Primarily, the issue was support,” Muse said. “We had people who specialized in the water side, the wastewater side and the lift stations. We had three groups trying to support the automation side of the house.”
Also, none of the applications could see the whole system, creating blind spots for critical alarms and making it difficult to respond to all issues in a timely fashion. Additionally, the lack of a single historical database complicated the process of generating reliable system-wide reports and trends. Since none of the software packages could communicate with all of their RTUs and PLCs, the utility simply could not standardize on one of those packages.
“One of our biggest issues was the proprietary nature of our software packages,” Muse said. “We had to use specific hardware with each of them. It became problematic especially when we started to experience significant lead times to get equipment.”
The limited yearly capital budget also prevented Ocala from standardizing on one of their brands of remote hardware.
2001: The First Step
To begin this transformation, the utility needed to find an open architecture software solution.? “We had come across VTScada when talking to our neighbor, Gainesville Regional Utilities, who were in conversation with [Trihedral, the manufacturer of the software],” Muse recalled. “We were searching for something that wasn’t proprietary, that we could afford and that wouldn’t involve significant hardware expenditure. VTScada offered all of that.
“We initially moved just the lift stations onto VTScada,” continued Muse. “On the day we installed it, we were literally polling all our sites within a few hours.”
Rather than rebuild the tag database from scratch, a tag conversion utility allowed them to simply convert the existing database. With this, the software could automatically generate standard graphic displays for all data points. The new system no longer required third-party tools for critical components such as reports, trending, hot-backup, remote access and alarm dialer since these are integrated into the software. Those tools grow with the product as new versions of the product are released. However, the problems associated with supporting two SCADA systems remained.
2002: One Unified System
After a year of running the SCADA applications side-by-side, the utility converted its water and wastewater plants as well. The software included graphic development tools that allowed the utility to create customized displays.
“After we proved that the new system was solid and reliable, everyone came by and saw what it could do,” Muse said. “Once we had enough people involved in the process, everyone wanted it at their plant. It wasn’t a hard sell at that point. The flexibility in creating new screens was a major plus for the treatment facilities.”
?
The reclamation facilities soon followed. Now authorized staff could monitor and control any remote site from a single interface and use integrated trending and reporting tools to analyze data from a central historical database (see Figure 1). A redundant hot-backup server in the office provided server and database redundancy. A single set of security user accounts managed access to the whole application. The utility was also free to incorporate other brands of hardware in the future based on features and price.
2010: A New Approach to Historian Management
For eight years, the utility continued to leverage its historical data to identify problems and increase efficiency; leading to a new set of issues.
Each site logged data over the network to a central historian in the main office which was backed up by an onsite redundant server. As a result, when the network was overloaded or offline, remote users could see local live data but could not access the historian. Worse, remote data collected during that period was lost, creating gaps in the permanent record. Further, each remote site required a primary and redundant server in order to provide local fail-over requiring additional maintenance and software licensing.
In 2010, a new city-wide high-speed fiber network combined with new VTScada features inspired the Ocala utility to rethink its approach.
‘We wanted to ensure access to historical data at all our remote facilities where we have folks for 16 hours a day,’ Muse said.
The new network allowed the utility to keep only one server at each location. Each one could take over for any other in the system. Each site also hosted its own local synchronized historical database with a central backup of all databases at the head office. This provided quick local access to live data with the benefits of a centralized database for reporting and analysis. This approach also reduced the server count by 50 percent and increased the levels of server redundancy from two to six (see Figure 2).
These were not the only savings from having distributed historians. “We eventually spoke to the Department of Environmental Protection about our wastewater plants,” Muse said. “We convinced them that we could un-man those sites for eight hours a day because the data was available offsite.”
During those periods, these sites could be monitored and controlled from the water plant which was staffed 24 hours a day. “They were pleased with the availability of the data in case something went wrong on an off shift,” Muse said. “They [Florida DEP] allowed us to go to un-manned shifts at three of our wastewater plants which was a major cost savings for us.”
An Efficient Transition
The combination of time, good planning and non-proprietary SCADA software allowed Ocala to transition to the unified system it needed without exceeding its yearly budget.
“We have been pleased with the system,” Muse said. “It’s flexible and robust enough that we can get some of our technicians some basic training on some of the things we need to do to support the system. We have been impressed that [Trihedral] have continued to grow the product from where it was 15 years ago to what it looks like now.”
Chris Little is a sales and marketing manager and Patrick Cooke is director of marketing, both for Trihedral Engineering Ltd.?
