2.0 Introduction
Research studies often have a narrow focus, but sometimes it can be too narrow. When too many patients are left out, those who remain may not be not representative of the types of patients you will encounter. When you are trying to figure out who was left out and what impact this has, ask the following questions:
2.1 Who was excluded from the study?
2.2 Who refused to join the study?
2.3 Who dropped out during the study?
2.4 Who stopped or switched therapies?2.0.1 Case study: Nicotine patches
The Journal of Pediatrics published a study of adolescent smokers in 1996 (Smith 1996). The researchers recruited 22 volunteers from five public high schools in the Rochester, MN area for participation in a smoking cessation program involving behavioral counseling, group therapy, and nicotine patches. Researchers measured the number of cigarettes smoked, side effects, and blood levels of nicotine.
The purpose of the research was to evaluate "the safety, tolerance, and efficacy of 22 mg/d nicotine patch therapy in smokers younger than 18 years who were trying to stop smoking." The authors also listed a secondary goal, "to compare blood cotinine levels, nicotine withdrawal scores, and adverse experiences with those of adults obtained in previous patch studies." Cotinine is a metabolite of nicotine and provides a useful objective measure of cigarette smoking. It also allowed the authors to examine whether nicotine toxicity was an issue.
This study did not include major segments of the teenage smoking population. The study included only white subjects because there were too few minority students in the Rochester area. Subjects had to get parental permission, excluding smokers who wished to keep their habit secret from their parents. Subjects were also volunteers, and thus could be considered more motivated to quit than the typical teenage smoker.
The study also had a serious drop out rate. Of the presumably thousands of teenage smokers in the Rochester Minnesota area, only 71 volunteers responded to the initial call for subjects. Of the 71 volunteers, 55% met inclusion criteria. Of the remaining 39, 44% declined to attend the initial meeting. Of the remaining 22, 14% were non-compliant. Of the remaining 18, 39% failed to respond to the one year survey. Only 11 completed the entire study (50% of those who started the study; 28% of those meeting inclusion criteria; 15% of the initial volunteers.)
This study had a serious problem with who was left out. The large number of subjects who did not get into the study or who did not complete the study makes it hard to generalize the findings of this research.
2.1 Who was excluded at the start of the study?
Researchers, trying to minimize variation, will use exclusion criteria to create more homogenous groups. While minimizing variability is good, too much homogeneity can backfire. It’s difficult to extrapolate results from a very tightly controlled and homogenous clinical trial to the variation of patients seen in your practice. Ask yourself the question "How similar are my patients?"
Watch out for exclusion criteria that leave out large groups of patients. Sometimes this exclusion is subtle. For example, if you wanted to study adolescent drug use, you might consider a survey of high school students. This survey, though, would exclude anyone who dropped out of school. The dropouts have a far higher rate of drug usage than teenagers who stay in school. If you are interested in all adolescents, but your research design excludes dropouts, you will seriously underestimate drug use (Swaim 1997).
In a different situation, of course, this might not be a terrible problem. It depends on your perspective. A principal trying to understand patterns of drug use in her high school, for example, might actually prefer to exclude dropouts.
You might encounter other subtle exclusions based on the geographic location or the type of health care setting, which places restrictions on the type of patients seen. A study of Midwest hospitals will not have a representative number of Hispanic patients compared to the Southwest. Tertiary care centers only see patients that are extremely ill.
There are three very common and very serious exclusions in medical research, however, that deserve special attention: exclusion of elderly patients, exclusion of women, and exclusion of children.
2.1.1 Exclusion of elderly patients
If you are elderly, pat yourself on the back. Your demographic group drives the healthcare economy. You are, by far, the largest consumers of new medications and new therapies. Yet, far too often, these new medications and new therapies are tested on patients much younger (Bayer 2000).
There's a simple reason for this exclusion. When researchers design their experiments, they want a nice clean sample.
Researchers want patients who are ill with one and only one disease. But with older people, several things will break down at the same time (Schellevis 1993).
Researchers don't want patients who are taking a lot of other medications. But older people take so many different drugs that they often qualify for bulk discounts at Walgreen's.
Finally, researchers want patients who are likely to stay alive for the duration of the research study. But older people are likely to die from conditions unrelated to disease being studied.
Although the reasons for excluding elderly patients are understandable, they are still not justifiable. Research done on younger patients cannot be easily generalized to older patients.
2.1.2 Exclusion of women
Several decades ago, there was a large study of aspirin as a primary prevention against heart attacks (Physicians Health Study Research Group 1989). This study recruited over 20 thousand physicians and asked them to take either a small dose of aspirin every day or take a placebo. They had to follow these physicians for five to ten years because they wouldn't cooperate and have heart attacks faster. At the completion of the study, the researchers announced that aspirin was highly successful at preventing heart attacks.
There was one major problem with the research sample, though. Every single one of the physicians studied was male. Not a single female was included in the sample. It's not as this was a problem only for men. Heart disease kills more women than any other condition.
There are some legitimate concerns when testing drugs that might harm a developing fetus, but you can handle this with careful restrictions to women who are not sexually active and/or who are using an effective form of birth control. In addition, some conditions, such as prostate cancer cannot be tested in women.
There is some dispute over whether gender bias exists, with one study arguing that it still occurs (Ramasubbu 2001) and another arguing that it does not (Meinert 2001). When exclusions of women does occur, it raises troubling questions and hinder your ability to generalize the results of the research.
2.1.3 Exclusion of children
At the opposite extreme from the elderly are children. This group, sadly, is also left out too often (www.aap.org/advocacy/washing/offlabel.htm).
Children are not little adults. The liver in a child will process drugs quite differently from the liver of an adult. The nutritional demands of a growing child are quite different than those of a fully grown adult. And if you thought that your children became unpredictable as they went through puberty, try looking at them from a medical perspective!
No one wants to see our children used as guinea pigs, and there are special ethical reviews and safeguards that
