How we rate the efficacy of our treatments or how to know if our treatments actually work.
Efficacy vs. Clinical Effectiveness :
Efficacy is determined by evaluating formal studies done on each disorder. When a study is done, the treatment is very carefully standardized, the people doing the interventions should have great expertise in the treatment and the disorder, and patients are very carefully selected. In the real clinical environment, the patients may have many problems in addition to the one they are being treated for (which would affect the chances of the treatment doing well), may be given many overlapping treatments at once (so you can't tell how much help any one treatment was), and the therapist may not be as experienced as the people running the research study. Thus, a treatment's efficacy may be greater or lesser than its effectiveness in the real clinical world.
The Association for Applied Psychophysiology has developed the following criteria for setting the level of evidence for efficacy (Moss and Gunkelman 2002, LaVaque et al 2002): It is very similar to the rating schemes developed by other organizations such as the American Psychological Association. Please note that the efficacy ratings made based on these criteria are from formal studies. Please see these citations for an explanation of how the ratings were arrived at and a discussion of the weaknesses of double blind studies for several of the techniques evaluated.
LaVaque, T., Hammond, D., Trudeau, D., Monastra, V., Perry, J., Lehrer, P., Matheson, D., & Sherman, R. (2002). Template for developing guidelines for the evaluation of the clinical efficacy of psychophysiological evaluations. Applied Psychophysiology and Biofeedback, 27(4), 273-281.
Co-published in Journal of Neurotherapy, 6, 11-23. Moss, D. & Gunkelman, J. (2002). Task force report on methodology and empirically supported treatments: Introduction. Applied Psychophysiology and Biofeedback, 27, 261-262.
Level 1: Not empirically supported: Supported only by anecdotal reports and/or case studies in non-peer reviewed venues.
Level 2: Possibly Efficacious: At least one study of sufficient statistical power with well identified outcome measures, but lacking randomized assignment to a control condition internal to the study.
Level 3: Probably Efficacious: Multiple observational studies, clinical studies, wait list controlled studies, and within subject and intrasubject replication studies that demonstrate efficacy.
Level 4: Efficacious:
a.) In a comparison with a no-treatment control group, alternative treatment group, or sham (placebo) control utilizing randomized assignment, the investigational treatment is shown to be statistically significantly superior to the control condition or the investigational treatment is equivalent to a treatment of established efficacy in a study with sufficient power to detect moderate differences, and
b.) The studies have been conducted with a population treated for a specific problem, for whom inclusion criteria are delineated in a reliable, operationally defined manner, and
c.) The study used valid and clearly specified outcome measures related to the problem being treated and
d.) The data are subjected to appropriate data analysis, and
e.) The diagnostic and treatment variables and procedures are clearly defined in a manner that permits replication of the study by independent researchers, and
f.) The superiority or equivalence of the investigational treatment have been shown in at least two independent research settings.
Level 5: Efficacious and specific: The investigational treatment has been shown to be statistically superior to credible sham therapy, pill, or alternative bona fide treatment in at least two independent research settings.
Efficacy ratings for biofeedback based treatments:
Please see Yucha and Gilbert's 2004 review of efficacy ratings for most of the disorders biofeedback is used to treat. (Yucha, C. & Gilbert, C. (2004) Evidence-Based Practice in Biofeedback and Neurofeedback. Association for Applied Psychophysiology and Biofeedback. Colorado Springs, CO. It can be purchased from the AAPB's bookstore on this site.)
The following efficacy ratings are taken from the Yucha/Gilbert text. Additional references are available in the text.
Efficacious and specific (Fifth level):
- Urinary Incontinence in Females
Efficacious (Fourth Level) :
- Attention Deficit Disorder
- Headache- Adult
- Temporomandibular Disorders
- Urinary Incontinence in Males
Probably efficacious (Third level):
- Alcoholism/Substance Abuse
- Chronic Pain
- Fecal Elimination Disorders
- Headache- Pediatric Migraines
- Traumatic Brain Injury
- Vulvar Vestibulitis
Possibly Efficacious (Second level):
- Cancer and HIV, Effect on Immune Function
- Cerebral Palsy
- Chronic Obstructive Pulmonary Disease
- Depressive Disorders
- Diabetes Mellitus
- Foot Ulcers
- Hand Dystonia
- Irritable Bowel Syndrome
- Mechanical Ventilation
- Motion Sickness
- Myocardial Infarction
- Post Traumatic Stress disorder
- Raynaud's Disease
- Repetitive Strain Injury
- Urinary Incontinence in Children
Not empirically supported (First level of evidence):
- Eating Disorders
- Multiple Sclerosis
- Spinal Cord Injury