Can Machine Learning Really Provide A Cost-Effective Desktop Analysis?

By Greg Baird

---

Artificial Intelligence, specifically machine learning, is poised to make a significant impact in underground water infrastructure asset management. Not only does machine learning drive performance optimization, it also increases efficiencies in business processes and planning. In the water utility industry, due to the multitude of data and variables involved, water main condition assessment is an ideal use case for this technology.

Desktop Analysis

The traditional desktop study includes collecting all of the pipe attributes, location and repair and break history, and developing a preliminary risk matrix.

Desktop analysis or computational approaches are by far the most cost effective and least invasive, but many of these methods are based on arbitrary assumptions and weightings and utilize a small number of factors relating to the performance of the pipe. These issues translate into a high error rate which means good pipes could be identified as high risk and face premature replacement.

The industry has adopted a number of different approaches ranging from a simple weighted score approach on an excel spreadsheet, to Cohort Analysis, LEYP, Kanew forecasting and Weibull modeling. More advanced statistical modeling may help decipher differences between variables, although many of these approaches may not have the ability to consider the importance of the spacial proximity, elevation or pipe material characteristics which can distort the overall accuracy.

Machine Learning Vs. Age-Based Models

One of the first steps in evaluating this new machine leaning technology for cost effectiveness and accuracy for the water industry is to compare the traditional age-based methodology of determining water pipe asset life and water main breaks with a machine learning risk assessment model.

The Challenge

As the water industry continues to collect large amounts of data, the old-school methodologies of analyzing that data have only provided a portion of the data’s real value. Age-based or straight-line depreciation methodologies have a very high rate of inaccuracy which have translated into thousands of miles of good pipe being replaced simply because it was “at the end of its aged-based service life.” AI/machine learning leverages a water utility’s collected data and combines more than 1,000 other variables to provide a more accurate predictive model. This model is created for calculating the probability of a water pipe segment failing. Comparing these two types of models reveals very different results as explained in the case study analysis for a large and medium sized utility.

Case Study: Large-Sized Water Utility

Five years of water main break data from a large utility with 3,395 miles of pipe was used to compare how each model would predict the actual pipe’s failures. To do this, part of the data set was withheld from the machine learning model to demonstrate the accuracy of its predictability.

The machine learning model captured 26.2 percent of the historical pipe breaks as part of its analysis of the highest risk or worst 5 percent of pipes that are predicted to fail. This 5 percent of the 3,395 miles of pipe identifies 139.5 miles of pipe as the highest risk pipes that are predicted to fail.

The age-based model captured 26.2 percent of the historical pipe breaks by identifying the worst 7 percent of the pipes. This 7 percent of the 3,395 miles of pipe suggests that 195.4 miles of pipe would need to be replaced to avoid the historical breaks.

In comparing the two models, the machine learning model was 28.5 percent (2 percent/7 percent) more effective in identifying pipe breaks over the age-based model.

• The machine learning model calls for 139.5 miles of pipe to be replaced
• The age-based model calls for 195.4 miles of pipe to be replaced
• The replacement difference is 56 miles of pipes
• If the replacement cost for 1 mile of pipe was $1,000,000 then the age-based model would have spent $56,000,000 more than the machine learning model to prevent the pipe failures.

In order to further test the machine learning model against an age-based model, a new main break data set was used from a medium sized utility following the same comparison methodology as the large utility.

Case Study: Medium-Sized Water Utility

Five years of water main break data from a medium sized utility with 847 miles of pipe was used to compare how each model would predict the actual pipe’s failures. To do this, part of the data set was withheld from the machine learning model to demonstrate the accuracy of its predictability.

The machine learning model captured 10.9 percent of the historical pipe breaks as part of its analysis of the highest risk or worst 1.9 percent of pipes that are predicted to fail. This 1.9 percent of the 847 miles of pipe identifies 14.8 miles of pipe as the highest risk pipes that are predicted to fail.

The age-based model captured 10.9 percent of the historical pipe breaks by identifying the worst 2.4 percent of the pipes. This 2.4 percent of the 847 miles of pipe suggests that 18.7 miles of pipe would need to be replaced to avoid the historical breaks.

In comparing the two models, the machine learning model was 21 percent (1.9 percent/2.4 percent) more effective in identifying pipe breaks over the age-based model.

• The machine learning model calls for 14.8 miles of pipe to be replaced
• The age-based model calls for 18.7 miles of pipe to be replaced
• The replacement difference is 4 miles of pipes
• If the replacement cost for 1 mile of pipe was $1,000,000 then the age-based model would have spent $4,000,000 more than the machine learning model to prevent the pipe failures.

The Water Pipe Condition Assessment Program and Costs

The accuracy and cost effectiveness of the machine learning provides benefits to the entire pipe condition assessment program by focusing more expensive and time-consuming inspection actives to the high-risk pipes for further investigation. Machine learning can be 20 to 30 percent more accurate and provide the same of level of cost efficiencies in identifying the highest-risk pipes. This 20 to 30 percent cost savings can also be passed down to reduce the individual unit costs of direct inspections by only focusing on the pipes and pipe segments as determined by the Machine Learning Pipe Risk Assessment.

---

Asset management is maintaining a desired level of service at the lowest life cycle cost.

---

A condition assessment as a fundamental part of asset management is based on the assumption that materials or infrastructure components deteriorate, with the goal of gathering information to predict the need for repair, rehabilitation, or replacement. The nine main steps of machine learning asset management condition assessment process are:

1. Develop an up-to-date inventory of assets. With water main pipes, a geographic information system (GIS) mobile app can be used to collect the pipe data.
2. Apply machine learning likelihood of failure as a solution to clean and verify the data and identify the probability of each pipe’s failure using hundreds of variables with a 20 to 30 percent improvement over an age-based model.
3. Produce a monetized criticality rating for each pipe segment and conduct risk mitigation efforts.
4. Select and cost effectively deploy direct inspection condition assessment technologies to the high-risk pipes to further determine the internal pipe condition, pipe wall condition, pipe environment condition or leakage.
5. Update the planned maintenance activities in the CMMS.
6. Revise pipe repair and replacement capital plans and re-evaluate water rate increases and future debt needs.
7. Provide high risk pipe locations with GIS maps on mobile devices for field crews.
8. Update results in the water asset management plan.
9. Systematically repeat by updating the machine learning model with new data.

Asset management is maintaining a desired level of service at the lowest life cycle cost. Lowest life cycle cost includes the cost efficiencies gain through machine learning over age-based methodologies. The cost benefits of machine learning extend to additional direct inspection condition assessments, pipe rehabilitation, repair and replacement activities. Asset management is implemented through an asset management program and typically includes a written asset management plan. Sound financial decisions and developing an effective long-term funding strategy are critical to the implementation of an asset management program.

---

Greg Baird is president of the Water Finance Research Foundation. He specializes in long-term utility planning, infrastructure asset management and capital funding strategies for municipal utilities in the United States. He has served as a municipal finance officer in California with rate design and implementation experience and as the CFO of Colorado’s third-largest utility.