29 January 2021

A new update of the Conduit Condition Codes of Australia, WSA 05-2020 Ver 4.1, arrived in late 2020. by Peter Slingsby, Auscodes

A lot of effort went into this update, with the code seeing significant revision, updates and testing to address the shortcomings and ambiguity of previous versions, but more importantly to bring scoring into line with practical applications and expectancy.

In 2021, we hope to see widespread adoption of this code.

Problems associated with 2008 and 2013 codes including scoring, descriptions, conduit materials and the general application of codes have been revised.

Some helpful additions have also been added, for example separating main conduit scores from lateral scores, faster input of codes, refined quantification measurements (for quicker identification and descriptions of problem areas etc.).

The new codes are produced in a digital manual that can be purchased and downloaded via the web here.

Whilst the consistency of language, scoring and application has been improved, this does not automatically translate to improved outcomes unless the assessor is adequately skilled and competent to classify and record information to the standard (or any previous versions of WSA05).

Not only have I personally witnessed some concerning trends in the industry, my peers and colleagues have also expressed deep concern as to the downward trend in skills, attention to detail and overall accuracy of surveys being produced.

A disturbingly large number of inspections being assessed by operators, supervisors, asset owners and artificial intelligence (AI) programs are falling short.

Missed codes, incorrect codes, overstated and understated scores, poor vision, non-observance of proper header information and inadequate camera setups are all far too common.

One AI company boasts an accuracy rate of 80 per cent percent, compared with an average operator accuracy of 48 per cent for structural defects, and 76 per cent (AI) vs 52 per cent (operator) for service defects.

Having been involved in the inspection and training industries for more than 50 years, these numbers are completely unacceptable, and accuracies of more than 90 per cent would be expected at any training course and of any competent operator.

We need to ask ourselves some serious questions as to how we arrived at this point, and how we are to improve? It is clear the slide in quality observed is mostly due to lack of training and accreditation in the current environment.

Downward pressure on contractor pricing, increasing pressure to produce high volumes, and high operator turnover have all contributed to lowering of expectations.

The introduction of AI has shown promise to remove the human element; however, as will be discussed, there are still some serious shortfalls with this approach.

By ignoring or trying to diminish the importance of operator skill and behaviour we set ourselves up on a very slippery slope where quite serious mistakes will be made.

Asset owners and managers typically do not have the time to review hundreds of hours of CCTV footage, so survey accuracy is critically important.

The likely conclusion will lead to major and very expensive, perhaps catastrophic, events such as collapses and sinkholes.

It has happened before in 1993 when two major mains collapsed due to the axing of a preventative maintenance program that caused major pollution of local waterways and extensive infrastructure damage.

Improving Outcomes Proper camera setups An often overlooked and critically important step, the CCTV operator must be trained to select, setup and maintain appropriate equipment.

This will differ for each individual job.

Not all camera systems are created equally, and all have strengths and weaknesses. Lighting must be sufficient as should lens quality and cleanliness, camera position, image clarity and photo chip resolution etc.

Consistency of approach To provide repeatable, objective surveys, the operator should be trained to conform to WSA05 accepted behaviour such as observing speed limits, not panning whilst the camera is in forward motion, stopping at defects and panning for the asset owner to get a good look at the defect, etc.

Suitable training is vital in achieving these outcomes.

If the operator controlling the physical camera is not recording observations and this is to be done later by an assessor or AI software, consistency of motion and capturing quality image of all items of interest is even more important. No matter how it is assessed, this image is the fundamental baseline to assessment and no shortcut should be made.

Correct interpretation of codes The recording of the correct observations, in the correct combinations is crucial to the outcome of a survey.

To be of value to asset owners/managers, a survey must accurately report the conduit’s observed condition and highlight areas of particular concern that need to be addressed, for example intruding services, risks of void generation, ingress and infiltration etc.

It is not uncommon for untrained assessors to ‘over code’ by placing too many irrelevant codes, or to ‘under code’ by using the incorrect descriptions or missing defects entirely.

Both practices may have dramatic effects on final outcomes.

Ability to identify bad surveys, scoring and coding issues All stakeholders from the camera operator on-site, to the ultimate user of the survey information, need to understand the signs to look for to know when something is not quite right with delivered inspections, and be familiar with resources to confirm and interpret results.

Quality should begin with the field operator; however, all levels have a role to play to ensure accuracy.

Correct interpretation of codes Arguably, the best accuracy is achieved by a properly trained and experienced operator.

The human brain is unmatched in its ability to interpret images, make decisions, learn, investigate and highlight items of concern.

Artificial Intelligence (AI) systems are showing promise in this area with some very impressive image matching and classification, but we must be cognisant of the limitations of such a system.

Many of these issues will no doubt be addressed as training datasets increase in size; however, some are more fundamental to the technology currently being employed.

With the introduction of AI computer conduit assessment, the operator would have to use a different technique to capture the video, moving the camera in such a way that the computer can, ‘Identify correctly the conduit defect’, and ‘give the defect the correct location in the pipe’; however, AI is confused by the pan and tilt of a camera head and prefers a straight-ahead field of view.

This, in my view, is a backward step in CCTV surveys.

Pan and tilt camera was an important advance in camera technology 30 years ago as it allowed a thorough inspection of laterals – even if surveying downstream – holes, voids and soil visible defects or structures that could not be properly inspected by ‘field of view’ methods.

This renders the bulk of existing CCTV footage with some significant barriers for AI to assess.

In addition, by not using the full capabilities of the camera system, a clear picture of a defect may not be obtained.

It is critical to be able to look perpendicular to the axis of the conduit to assess joints, extent of holes, voids and other surface defects.

The use of optical zoom – if available on the CCTV camera system – will give an improved level of detail in larger conduits for the measurement of cracks and a closer look at defects for proper assessment.

The AI method codes defects mostly using an extended ‘field of view’ LRP that will be different to ‘head of the camera’ LRP (see illustration 1).

As can be seen by the illustration, if the pre-set distance is set at the head of the camera by the operator and the video is being assessed by AI, in a 600 mm conduit 690 mm would have to be added to the measurement on the video, while in a 900 mm conduit 1.025 m would have to be added.

AI does not allow for this difference of the LRP and takes its defect measurement from the video, AI also uses the reference point ‘C’ ‘Centre of manhole’ as its default for its longitudinal location.

If the conduit video has used the ‘manhole wall’ as its LRP then another 500 mm would have to be added, in a 600 mm pipe this would mean a discrepancy of 1.190 m!

This is totally unacceptable and not within the specifications set out In the conduit condition manual.

See WSA05-2020 ‘1.8 Distance Measurement’ and 2.5.4.5 longitudinal location.

• AI is currently unable to handle continuous codes (i.e., where a defect has an extent greater than 1 m longitudinally). Accurate reporting of extent has quite serious effects on the numerical scores. This would account for AI’s claim that they find more defects than a manual assessor.
• AI assessment coding currently has been trained to and classifies using the WSA05-2103 conduit condition code manual; however, the image classifiers are typically not yet good enough to use the characterisations or quantifications outlined in the manual so, as an example, ‘B-Broken pipe’ is just Broken. AI does not indicate the following characterisations:
  • D – Displaced (where there is visible shear and/or separation of the conduit fabric)
  • M – Broken Missing (where there are pieces of the conduit fabric missing due to a cracking event, as distinct from a hole in the wall)
  • E – Exceptional (where the pipe has experienced external or exceptional trauma, and the defect is not a result of organic deterioration).

Joint Displaced is matched via AI such that nature and extent are not reported.

Characterisations JDA, JDL, JDR are ignored – information required by the lining industry.

Surface Damage Hole (SDH) is not recognised and could lead to a disaster happening for the asset owner.

Many serious defects are quite rare, leading to a chronic lack of training images.

For example, an AI company has published accuracy statistics for defect classes, and whilst ‘Roots’ and ‘Obstruction’ have an 80 per cent recognition accuracy (as they are common defects and the image training pool is large), joint defects only enjoy a 25 per cent accuracy, and ‘other’ (e.g. collapse, holes, surface damage etc) sit at around 20 per cent recognition accuracy.

Over time this will improve; however, in known datasets containing more than 1 million images, the prevalence of some defects (usually the more critical and serious ones) still fall a long way short of the number of images required to accurately train a classification model, so it is conceivable it would be quite some time before enough source material is collected.

In the meantime, these defects are being missed, or mis-classified.

Whilst technology moves at a rapid pace, and these issues are no doubt being addressed, we must be mindful of the impact these and many other issues have on the survey outcomes and how the resultant data is then used to make decisions.

At this point in time, it would be unadvisable to rely on unchecked AI assessment to drive renewal, maintenance, valuation or risk-based programs.

Moving forward

So, what needs to be done to improve the quality of conduit condition assessment?

• It should be the responsibility of the CCTV contractor to make sure whoever is in charge of operating the CCTV inspection camera unit has the required qualifications and certificate to survey the pipe in a manner that correctly interprets and applies the WSA05 manual, also taking into account customer specifications.
• It should be the asset owners responsibility to make sure part of their team has an accreditation in conduit assessment and that the assessors, including AI machine reporting of CCTV reports of any of their conduit assets, have the required accreditation certificate (not just an attendance document) as outlined in the WSA-05 Conduit Inspection Reporting code of Australia and should be an important requirement in the contract.
• AI needs to improve its software to allow the use of pan and tilt, as it is a requirement in the WSA codes. All mainline cameras are built with this capability and it is essential for a complete and accurate survey and would be asked for in the asset owners specifications. See 1.6.1. General (camera) WSA 05 2020-4.1.
• Mandatory auditing of a nominated percentage of any CCTV contract by an accredited assessor would help to keep the CCTV contractor on the ball. This has already been taken up successfully by some asset owners.

It is in the asset owners and the Water Services Association interest to make sure these improvements are made.

Inaccurate reports-coding is as bad as axing a preventative maintenance program and we don’t want that to happen again!