Final report: Tech overreliance contributed to bursting water mains
Overreliance on technology caused a major water main break in Woodbury last fall, resulting in nearly 200 insurance claims and costing the city $165,000.
A report released last week concluded the city’s investigation and explained how the technology controlling the system failed to do its job on that early Sunday morning in November 2013.
During the incident, the city’s waterworks system malfunctioned and caused its pumps to continue pumping, building excess pressure and causing water main breaks across the city.
Though the warning signs weren’t obvious, a close inspection by an employee could’ve detected the faults in the system, according to the report.
“We’ve learned a lot about the need to have human hands on these systems,” Utility Superintendent Dan Hansen said. “That’s probably one of the biggest lessons.”
Woodbury’s insurance carrier, the League of Minnesota Cities, paid almost $300,000 in claims as a result of the failure.
The city also repaired water mains and valves for $90,000 and paid a deductible of $75,000 to the insurance company, the report states.
The events that led to the incident are complex, according to the report that explained there was not one point in time or one specific component failure that led to the flooding.
A backup monitoring system partially failed prior to the actual water main leaks, plus the alarms that should’ve alerted public works crew to the problem didn’t detect the failure right away.
Then when the main monitoring program malfunctioned, the switch-off program couldn’t hand off control to the partially failed backup system.
The entire system is connected with the backup monitoring, actual water main valves and emergency alarms. All three failed to notify workers because they didn’t malfunction at the exact same time.
As a result, the city upgraded its system so each component is not dependent on the other.
Public Works Director Klayton Eckles compared operating the technology to flying an airplane, which is both computer and human operated, and overreliance on the automated portions can sometimes lead to crashes.
“To some extent we thought we were operators but we found out we were using that tool to a heavy extent, where it said everything was OK,” he said. “In this case it wasn’t.”
The investigation concluded the sequence of events was human error and that personnel did not see the warning signs so the pumps continued to operate leading to increased pressure and excess water.
Eckles said staff has been trained to inspect all of the tools closely and regularly to avoid future problems.
“This was a hard lesson for our crew,” he said, calling it “a morale shaker.”