November 2004

Evidence or consensus?

by Dr. Christopher Kent

Several disturbing reports have come to my attention. It appears that there is an effort to re‑write history, and represent the consensus Mercy Guidelines as evidence‑based guidelines. The Mercy Guidelines, despite being over a decade old, are still being promoted by certain factions in the profession. "Who cares?" you ask. "Does it matter?" The answer is a resounding "yes." Consensus guidelines create an illusion of clinical certainty and scientific support that simply does not exist.

Consensus guidelines

Consensus guidelines represent the opinions of the guideline developers. Jagoda [1] listed the characteristics of a formal consensus process:

‑‑ Group of experts assemble

‑‑ Appropriate literature reviewed

‑‑ Recommendations not necessarily supported by scientific evidence

‑‑ Limited by bias and lack of defined analytic procedures

It is important to realize that the Mercy Guidelines were based upon consensus, not necessarily evidence. In this regard, Powers has stated, "As the strength of the evidence declines, the composition of the panel and the process it follows become increasingly important determinants of the recommendations." [2]

Critics of consensus methods have suggested that developing formalized standards of practice leads to the practice of "cookbook medicine." It is feared that the unique circumstances of the patient, the condition of the patient, and the clinical insights of the attending doctor are subservient to the standards promulgated in the "cookbook." [3]

In addition to charges of "cookbook medicine," the selection of participants will significantly affect the outcome of the process. Sackman [4] describes a "halo effect" where participants "bask under the warm glow of a kind of mutual admiration society."

As Shekelle [5] has observed, acceptance of practice standards has been poor. He cites some significant shortcomings of previous methods of constructing standards. Most commonly, an inadequate review of literature and/or an implicit method of achieving consensus were to blame.

An example of an opinion masquerading as a fact is the frequently repeated Mercy recommendation that an "adequate trial of treatment/care" is: "A course of two weeks each of two different types of manual procedures (four weeks total) after which, in the absence of documented improvement, manual procedures are no longer indicated." [6]

In a RAND document, we are told, "an adequate trial of spinal manipulation is a course of two weeks for each of two different types of spinal manipulation (four weeks total), after which, in the absence of documented improvement, spinal manipulation is no longer indicated." A favorable response to manipulation is defined by RAND as "an improvement in symptoms." [7]

Objective physiologic measurements and imaging findings do not enter into their definition. What is the scientific basis for such caps? According to RAND, "There exists almost no data to support or refute these values for treatment frequency and duration, and they should be regarded as reflecting the personal opinions of these nine particular panelists." [7]

In short, consensus processes frequently result in the promulgation of practice guidelines which are based upon the opinions and biases of the participants in the absence of clinical data. The guidelines produced are a function of the composition of the group.

Evidence‑based guidelines

Jagoda [1] describes the process of evidence‑based guideline development:

‑‑ Literature search

‑‑ Secondary search of references

‑‑ Articles graded

‑‑ Recommendation based on strength of evidence

‑‑ Multi‑specialty and peer review

Evidence‑based clinical practice is defined as "The conscientious, explicit, and judicious use of the current best evidence in making decisions about the care of individual patients...(it) is not restricted to randomized trials and metaanalyses. It involves tracking down the best external evidence with which to answer our clinical questions." [8]

The National Health and Medical Research Council [9] has made it clear that opinions are not evidence. "The current levels (of evidence) exclude expert opinion and consensus from an expert committee as they do not arise directly from scientific investigation."

According to Rosner [10], Bogduk was equally emphatic: consensus or expert opinion is no longer to be accepted as a form of evidence.

The Council on Chiropractic Practice has developed evidence‑based practice guidelines for vertebral subluxation with the active participation of field doctors, consultants, seminar leaders, and technique experts. In addition, the Council has utilized the services of interdisciplinary experts in Agency for Health Care Policy and Research (AHCPR) guidelines development, research design, literature review, law, clinical assessment, and clinical chiropractic.

The purpose of these guidelines is to provide the doctor of chiropractic with a "user friendly" compendium of recommendations based upon the best available evidence. It is designed to facilitate, not replace, clinical judgment.

As Sackett wrote, "External clinical evidence can inform, but can never replace, individual clinical expertise, and it is this expertise that decides whether the external evidence applies to the individual patient at all and, if so, how it should be integrated into a clinical decision. Similarly, any external guideline must be integrated with individual clinical expertise in deciding whether and how it matches the patient's clinical state, predicament, and preferences, and thereby whether it should be applied." [8]

Consensus guidelines are not synonymous with evidence‑based guidelines. Do not be mislead by individuals who do not understand the difference.

References

1. Jagoda A: "Clinical policies' development and applications." ACEP 2004. http://www.ferne.org

2. Powers EJ: "From the Congressional Office of Technology Assessment." JAMA 1995;274(3):205.

3. Fink A, Kosecoff J, Chassin M, Brook RH: "Consensus methods: characteristics and guidelines for use." Am J Public Health 74(9):979, 1984.

4. Sackman H: "Delphi Critique." Lexington Books. Lexington, MA, 1975.

5. Shekelle P: "Current status of standards of care." Chiropractic Technique 2(3):86, 1990.

6. Haldeman S, Chapman‑Smith D, Petersen D, eds. "Guidelines for Chiropractic Quality Assurance and Practice Parameters: Proceedings of the Mercy Center Consensus Conference." Aspen Publishers, Inc. Gaithersburg, MD, 1992.

7. Shekelle PG, Adams AH, Chassin MR, et al: "The Appropriateness of Spinal Manipulation for Low Back Pain. Indications and Ratings of a Multidisciplinary Expert Panel." RAND Corporation. Santa Monica, CA, 1991.

8. Sackett DL: Editorial. "Evidence‑based medicine." Spine 1998;23(10):1085.

9. "How to use the evidence: assessment and application of scientific evidence." National Health and Medical Research Council. Commmonwealth of Australia. 2000.

10. Rosner AL: "Evidence‑based clinical guidelines for the management of acute low back pain: Response to the guidelines prepared for the Australian Medical Health and Research Council." JMPT 2001;24(3):214.

(WCA Vice President Dr. Christopher Kent, president of the Council on Chiropractic Practice, is a 1973 graduate of Palmer College of Chiropractic. The WCA's "Chiropractic Researcher of the Year" in 1994, and recipient of that honor from the ICA in 1991, he was also named ICA "Chiropractor of the Year" in 1998. He is director of research and a co‑founder of Chiropractic Leadership Alliance. With Dr. Patrick Gentempo, Jr., Dr. Kent produces a monthly audio series, "On Purpose," covering current events in science, politics and philosophy of vital interest to the practicing chiropractor. For subscription information call 800‑892‑6463.)