SMS Benchmarking

Over the last three years, with the support of the APIA Research and Standards Committee and the Energy Pipelines CRC, I have been running a project to benchmark the AS 2885 safety management study process against other recognised methods of pipeline risk assessment.  The alternative methods were quantitative risk assessment (QRA) which is required by planning authorities in some states, and reliability based analysis (RBA) which is a relatively new technique developed by C-FER in Canada and included as in option in the Canadian pipeline standard CSA Z662.

SMS compared very well – it produced results no less conservative (and possibly slightly more conservative) than the alternative methods.  I think is a very important outcome.  If there is a serious pipeline failure resulting in multiple fatalities then any subsequent enquiry may well decide to look very closely at the basis of the risk assessment used to justify the pipeline design and operation, which means that the AS 2885 SMS process itself may come under close scrutiny.  This benchmarking study has shown that it should stand up well, in that alternative approaches to risk assessment are unlikely to result in different decisions about whether the risk is tolerable or not.

Notwithstanding that the results of the comparison are similar in terms of deciding what risks are tolerable, the SMS process seems altogether superior as a practical method for use by pipeline engineers.  There are a number of reasons for that:

  • SMS does not require any specialist input but can be implemented by pipeline engineers in the normal course of their work
  • Threat mitigation is an integral part of the process and in fact most mitigation is identified (and threats eliminated) before any risk assessment is necessary
  • Threats identified as presenting an intolerable risk are the obvious targets for measures to reduce risk (in contrast with QRA which provides negligible guidance on what action is necessary if the risk is intolerable)

I presented the results of the benchmarking study at the APIA seminar in Brisbane last week.  A PDF of the presentation is available here.  The full report on the comparison study is titled “RP6.4-01 AS2885 SMS Benchmarking – Final Report” and is available on request to the EPCRC, but only to member companies of the Research and Standards Committee.

This entry was posted in Conferences, Operations, Pipeline design, Research, Risk assessment, Standards. Bookmark the permalink.

7 Responses to SMS Benchmarking

  1. Keith Lenghaus says:

    Peter – I’ve looked at your slides but not read your report, so if my question is covered there I apologise.

    Did you consider the issue of study to study SMS consistency? Appropriately seeded, my understanding is that QRA would give consistent results across sites. And you have shown that QRA matches the SMS results, where the SMS has been (presumably) carried out by personnel skilled in the art. I most definitely do not have any particular example in mind; only that it seems to me that the qualitative component to SMS could result in risks being weighted differently (up or down) study to study where QRA might be more stable.

    • petertuft says:


      No, this study did not look into that. However there are anecdotal reports of independent QRAs on the same Australian pipeline which produced results differing by FOUR orders of magnitude. That is so extreme it sounds hard to believe but the source of the information is highly respected and very credible. So I and others in the pipeline industry are pretty wary about claims of QRA consistency.

      Nevertheless you are right to raise a query about SMS consistency. I was the facilitator for both of the SMSs in the benchmarking study so I’m reasonably confident about the consistency between those cases 🙂 But there may be variations between different facilitators so that could perhaps be a future investigation.

  2. Anonymous says:

    Re the quote about the results varying by four orders of magnitude. Did the actual conclusions vary between studies? That is the key question given risk levels are usually given per million years

    • petertuft says:

      It’s anecdotal and was based on work done probably 15 years ago, maybe more, so I don’t have any more information (but I do trust the source). You raise a valid point about whether the conclusions were the same since deciding whether risk is tolerable or not is the ultimate objective. However regardless of that such a large variation in results from similar analyses has to cast doubt on the validity of the process.

  3. Pingback: Pipeline QRA (quantitative risk assessment) | Pipelines OZ

  4. Pingback: Risk matrix selection | Pipelines OZ

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s