Leveraging New Advances in AI, Machine Learning for Aging Underground Infrastructure

The challenges related to managing sewer collection systems are well understood. These include an ever-growing quantity of sewer inspection videos needing to be assessed and acted upon each year. In the total lifespan of this inspection data 30 to 40 percent of the total acquisition time is expended on the tedious work of watching (and in many cases re-watching) sewer video recordings.

In spite of this, the data is still frequently incomplete, inadequate or unused (while it waits in a reviewer’s backlog). Sewer CCTV defect coding consumes a disproportionately large amount of time and cost, currently most often produces deficient data, but is also one of the single most important steps in a utility’s fight to manage aging infrastructure.

Finding a way to automate the sewer CCTV defect coding process has often been heralded as a major boon for sewer service providers because of the impact that it could have on the accuracy, consistency, and quantity of condition data it could make available to asset managers.

RELATED: Water Leak Targeting Using Artificial Intelligence & Machine Learning

Developing such a solution faces many challenges, however, including:

• Interpreting the notoriously low-resolution sewer CCTV video files
• Distinguishing between multiple types of defects, their configuration, pipe materials and size
• Producing a standardized output that can be easily imported into all of the decision-analysis and CMMS software packages commonly in use by sewer utilities

Prior efforts have made progress, but have not completely overcome these challenges:

• Training programs to standardize defect terminology have reduced bias, but the majority of data are still heavily skewed due to human bias.
• Conventional image-mapping software programs worked in-theory, but were overwhelmed by the nearly infinite ways that defects could manifest themselves visually in an image.
• Multi-sensor inspection technologies captured helpful contextual metadata that quantified defects, but the equipment itself is specialized and processing this information is time-consuming and limited to larger diameters.

Recent developments in artificial intelligence (AI) and machine learning (ML) have provided the marketplace with a new technology to bring to the campaign. Not only has the field of computer science evolved, but the accessibility and affordability of powerful computing hardware has also increased exponentially. Commercial processor speeds are orders of magnitude faster than five years ago, and the price of accessing cluster computers and cloud resources has fallen proportionally.

Some of the new approaches to automated sewer CCTV defect detection are built on a data-science-driven approach that leverage AI, ML, and computational vision methods. The foundational building blocks of this technology framework enable the integration of data-driven insights derived from multiple computer models of increasing maturity and sophistication to produce high-quality output, as depicted in Figure 2. More specifically, descriptive, predictive, and prescriptive analytic models are incorporated into the process. Descriptive analytics expose what defects co-occur where, why, and when; predictive analytics detail what defects are likely to be observed; and prescriptive analytics provide engineering-based recommendations and interventions based on similarities with other sewer conveyance systems.

In addition, AI and ML can be leveraged to extract even more value from this data, by taking into account unique defect counts, types, and locations – and running them through a series of intentional algorithms that emulate the same logical decisions engineers make when weighing “Watch? Rehab? Replace?” decisions. AI and ML provide a powerful supplement by crunching the data to calculate optimized scheduling recommendations based on the unique asset information and defect combinations found in the data.

RELATED: Artificial Intelligence & Condition Assessment

On the wave of these technical advances, many service providers have entered the marketplace with competitive offerings, one of which is the Dragonfly solution by Jacobs. Dragonfly is Jacobs’ digital solution co-created with Hitachi, which taps into next-generation artificial intelligence (AI) and machine learning (ML) capabilities to detect and code sewer pipeline defects. Users interact with Dragonfly’s 100 percent web-based user interface, providing easy and secure uploads, processing, and access to results.

To achieve accurate defect detection in sewer CCTV inspection data, Dragonfly utilizes an image classification model that detects the probability of an object (defect) in an image. An AI enabled object, observations and defect classification and localization algorithm then outputs the coordinates of the location of an object within the video frame image and provides a superimposed highlighted defect annotation atop the video frame images, as well as reflecting a confidence measure for the defect’s classification label. These models are capable of ingesting conventional low-resolution CCTV videos, and producing output used to generate the NASSCO Standard Exchange PACP Databases common in the industry.

In order to leverage the capabilities of cluster computing, Dragonfly is based completely in the cloud and runs as an on-demand service. There is no software installation, no yearly license fees, and no minimum quantities.

Jacobs has also incorporated AI/ML into our prescriptive asset management technology, Argon, (formerly known as SCREAM), and joined it with Dragonfly to provide important asset management insights, including recommended reinspection and maintenance schedules and prioritized cost-scheduled rehabilitation steps.

Argon uses AI to go beyond creating condition scores. As depicted in Figure 3, Dragonfly output is passed into Argon for advanced analytics and is fused with any additional asset inventory information uploaded.

Argon’s algorithms estimate immediate and lifecycle costs and suggested rehabilitation methods. A lot goes on behind the scenes in this step, including deciding trench locations and lengths.

Figures 4 and 5 depict how Argon insights target priority pipes, reinspection, and maintenance needs, and produce cost-scheduled-recommendations, and estimated rehabilitation cost estimates and schedules.

AI/ML solutions like Dragonfly have the potential to significantly reduce the level of effort and cost barriers for utilities to accurately detect and classify pipeline defects, which will help optimize pipeline rehabilitation and replacements programs and ultimately help utilities to achieve improved service levels for their customers around the world.

Chris Dermody is a digital solutions and technology leader for Jacobs. Raja Kadiyala is a vice president with Jacobs. Ravi Vennelakanti is vice president and chief innovation strategist, Big Data and Analytics Solutions lab. at Hitachi America Ltd.