October 2004

Telling the truth about guidelines

by Dr. Terry A. Rondberg

There are two ways to arrive at a set of practice guidelines. One is to assemble a group of experts and have them come to a consensus on how practitioners should care for patients. The other is to conduct systematic reviews and meta‑analyses of all available scientific evidence and to develop a set of recommendations based on that evidence.

A recent article in Chiropractic Economics noted that: "... the Mercy Conference produced the first evidence‑based recommendations for chiropractic services."

This is incorrect. The Mercy guidelines, are consensus‑based, not evidence‑based. The information about the Mercy guidelines, Chiropractic Economics editor Linda Segall told me, came from Joseph Keating, PhD, who later tried to backtrack by saying, "It is fair to call them (the Mercy guidelines) evidence‑based as well as consensus‑based; these are not mutually exclusive concepts."

Although each type has its strengths and weaknesses, they are definitely not the same, as Keating would have readers believe.

The Mercy guidelines were developed by the "Mercy Center Consensus Conference" and the group's results were NOT based primarily on scientific evidence, but on a consensus of opinion by its members.

In trying to re‑write history, Keating may be attempting to "clear the name" of at least one person who was involved in the Mercy Consensus Conference, which was undoubtedly one of the biggest fiascos in chiropractic history (even based solely on the fact that most organizations ‑‑ including the WCA, ICA, FSCO and even the ACA ‑‑ refused to endorse the document). Keating currently works with Mercy Consensus Conference Commission member Arlan Fuhr, DC, of Arizona.

Regardless of his motives, the fact remains that he is incorrect. The Mercy guidelines are consensus‑based guidelines. The Council on Chiropractic Practice (CCP) Clinical Practice Guideline, "Vertebral Subluxation in Chiropractic Practice," is the first and remains the only evidence‑based guidelines in chiropractic.

The difference is more than in semantics.

Most clinical and guideline authorities throughout the world recommend practicing according to evidence‑based guidelines rather than consensus‑based ones.

"Guidelines are not necessarily evidence‑based (in the past, they were often only 'consensus‑based'), but the best ones are evidence‑based," stated Chris Del Mar, Professor and Director, Centre for General Practice, University of Queensland, Brisbane, Australia.

Michael Berger and Ingrid Muhlhauser of the Department of Metabolic Diseases and Nutrition (WHO Collaborating Centre for Diabetes), Heinrich Heine University, Dusseldorf, and Professorial Unit for Health Sciences, University of Hamburg, Germany concluded: "Evidence‑based medicine ... calls for a scientifically proven approach to diagnostic and therapeutic procedures in medical practice. Accordingly, available external evidence needs to be systematically incorporated into clinical decision‑making in consultation with the patient... Neither individual physician preferences nor expert panel-based recommendations of so‑called consensus conferences are accepted within these rigid procedures for clinical decision‑making."

Andy Jagoda, MD, Professor of Emergency Medicine, Mount Sinai School of Medicine,

New York, noted two key problems with consensus‑based guidelines: "Recommendations (are) not necessarily supported by scientific evidence and limited by bias and lack of defined analytic procedures."

The Royal College of Radiologists in London, notes: "Evidence‑based guidelines are based on good research evidence of clinical effectiveness... Consensus guidelines have often been constructed by small and often unrepresentative groups and produced in non‑standard formats."

In chiropractic, we have two major sets of guidelines. One, the Guidelines for Chiropractic Quality Assurance and Practice Parameters was the result of the "Mercy Center Consensus Conference," from which is derived its more common name, the Mercy guidelines.

The Consensus Conference Commission panel of "experts" was carefully chosen by a nine‑member Steering Committee, which included Scott Haldeman, MD, of the World Federation of Chiropractic; attorney David Chapman‑Smith, of the World Federation of Chiropractic; Donald Petersen, Jr., publisher of Dynamic Chiropractic; and members of the administration from four chiropractic colleges. None was a practicing DC.

The panel they choose consisted of 35 doctors of chiropractic, only 23 of whom were in private practice. This group was split up into committees, each given the task of reviewing the appropriate literature and writing the first draft of their assignment chapters. The chapters were to be reviewed by "at least two experts" and then referred to "seven appointed members of the Commission for critical review." Since there was no mandate to choose experts outside the circle of the Commission, the entire process was in essence confined to the 35‑member panel.

The group met in closed sessions at a private retreat in California in 1992 and, according to the information provided in the document, "after three days, at five chapters a day, all topics had been reviewed." After six days ‑‑ from Jan 25‑30 ‑‑ the document was finalized and within months it was in the hands of the insurance industry and, at about the same time, doctors got their first look at it.

That's the consensus development process. It's not necessarily a bad process, although it is susceptible to bias and reflects the prejudice of panel members, particularly when members are specifically chosen because they share certain viewpoints or opinions.

On the other hand, evidence‑based guidelines are developed through a strict and thorough examination and analysis of scientific evidence, as devoid as possible of subjective opinion and interpretation. Results from scientific research are compiled and examined, and these results ‑‑ not a consensus of personal opinions from selected experts ‑‑ form the basis for the guideline recommendations.

The Council on Chiropractic Practice (CCP) Clinical Practice Guideline, "Vertebral Subluxation in Chiropractic Practice," is an example of evidence‑based guidelines.

"The first endeavor was to analyze available scientific evidence revolving around a model which depicts the safest and most efficacious delivery of chiropractic care to the consumer," notes the CCP document.

The CCP called for and received input from representatives of more than 35 named techniques in order to make sure it had access to all available evidence on chiropractic practice. An open forum was held where all interested individuals could participate and provide input.

The draft of the guidelines, based on the evidence reviewed, was submitted to 195 peer reviewers in 12 countries. Only then was it finalized and published.

That's the process behind evidence‑based guidelines. As conducted by the CCP, it was an inclusive and open process that resulted in a document that is uniquely chiropractic. They are, in fact, the only chiropractic guidelines to be accepted for inclusion in the National Guideline Clearinghouse (NGC ‑‑ www.ngc.gov). The NGC is a comprehensive electronic database administered by the AHRQ (Agency for Healthcare Research and Quality), which allows access to accepted guidelines from various health care disciplines.

Regardless of which guidelines ‑‑ or guideline developers ‑‑ Keating supports, he should be accurate when discussing them and not try to convince the profession that the document written by the Mercy Center Consensus Conference was not consensus‑based.

The undeniable historical truth is that the Mercy guidelines are the result of the consensus of 35 people in a closed room, while the CCP guidelines are based on all available evidence with the input of hundreds of doctors throughout the world.

There are important differences between consensus‑based and evidence‑based guidelines, and no amount of double talk or revisionist history can change that fact.

*************