Volunteer bias can be especially troublesome when you are examining issues that are considered by some people to be embarrassing or personal. Two American researchers examined the characteristics of people who were willing and unwilling to volunteer for studies about sexuality (Strassberg 1995). Volunteers had a more positive attitude towards sex, less guilt, and more sexual experiences.
2.3.5 Dropouts: What to look for.
It would be a rare research study that had absolutely no dropouts, so you don't want to be too fussy.
- First, you need to look for the proportion of patients who drop out.
- Second, look for a description of who dropped out. Is this group different from those who completed the study?
- Third, can you infer something about the dropouts and impute a reasonable value for their outcome?
2.4 Who stopped or switched therapies?
When you give a new drug to your patients, unless you watch them as they swallow the pill, you have no guarantee that they took the drug. This is also true for most research studies. The research subjects may not comply with the demands of the study. They may take only some of the medication, may stop taking the medication entirely, or may even switch to the competing medicine. Issues involving compliance are difficult to handle and there is no perfect way to analyze these patients.
Problems with compliance will usually end up diluting the impact of the new therapy. At the extreme, if 100% of your patients are non-compliant in both arms of the study, then you will surely see no difference between any two drugs. Although I discuss compliance from the perspective of a drug study, it is also an issue in non-drug studies. If a patient fails to show up for therapy sessions, or forgoes a required operation, that has the same issues and problems as noncompliance with a drug regimen.
2.4.1 Intention to treat
The intuitive approach is to remove from your study any patients who fail to comply with the protocol. This approach has its merits, but is generally avoided. What most researchers use instead is an "intention to treat" (ITT) approach. With ITT, the patients are analyzed in the groups to which they were originally randomized regardless of how much or how little medication they have taken. In fact, if some of the patients have the opportunity to switch to the competing drug (or therapy) and do so, with ITT, you still analyze them as if they took the drug they were originally assigned to.
There are several reasons why many researchers use ITT. First, researchers will often go to a lot of trouble to ensure randomized assignment in the study. Researchers in surgery have been known to take a sterilized coin into the operating room to choose which surgery to perform (Hollis 1999). When you go to such great lengths to use randomization, you don't want to abandon it without a fight. And when patient choices about whether they comply with the protocol start to determine who gets analyzed in which group, you lose randomization and all the benefits that it confers.
Second, with ITT, you get a more realistic picture of the new drug or therapy. If a drug or therapy is difficult to comply with, then that difficulty ought to be considered as part of the whole package. If noncompliance for a difficult to tolerate drug dilutes the impact of that drug, then that's worth knowing. Keep the noncompliant patients in because you will likely encounter the same patients among those who you regularly treat.
Third, ITT can prevent some serious biases in the research. Consider a new surgical therapy which is being compared to a standard non-surgical therapy. Some patients randomized to the surgical therapy might die prior to receiving the therapy. This is the most extreme form of non-compliance. These patients should still be analyzed as part of the surgical therapy group. Otherwise the rapidly dying patients will be excluded from the treatment group, but not from the control group, leading to serious bias.
As a general rule, noncompliant patients will usually have worse outcomes than compliant patients. In fact, there is solid evidence that patients who fail to comply with a placebo have worse outcomes than patients who comply with a placebo (Coronary Drug Project Research Group 1980; Horwitz 1990). I was quite amazed when I first saw evidence of this, but it actually makes sense. Patients who comply poorly with a placebo probably have other poor self care habits.
2.4.2 What an analysis that excludes noncompliant patients will tell you.
Even though ITT is widely used, there still is a place for the analysis that excludes noncompliant patients. This analysis answers the question, what will happen if I prescribe this drug to a group of patients who all take the drug regularly? The ITT analysis answers a different question: what will happen if I prescribe this drug to a group of patients that includes both compliant and noncompliant patients? It may help to know the answers to both questions.
Example: The MRFIT trial was a randomized comparison of a special intervention to usual care (Cutler 1991). The special intervention encouraged smoking cessation and dietary changes. A comparison of the groups as they were randomized to represented a comparison of special encouragement to change. A comparison of the groups that actually changed represented a different comparison, because some of the people in the special intervention ignored the advice and some of the people in the usual care group changed their habits on their own. This second comparison was of nonrandomized groups, since the patients themselves determined which group they belonged to. Nevertheless, it was interesting, because it involved a comparison, not of the encouragement itself, but of the actual changes that were being encouraged.
2.4.3 Excluding noncompliant patients before the study starts.
Since noncompliant patients can dilute the impact of a new drug, one dubious approach that researchers take is to not let these noncompliant patients into the study at all (Senn 1997). A placebo drug is given to all patients during a single blind run-in period, and anyone who does not comply with the placebo is excluded from the study.
The intent of this exclusion seems good on the surface. Problems with compliance will tend to dilute the effectiveness of a new therapy. At the extreme of 0% compliance, there is no possible way to distinguish the effectiveness. So excluding noncompliant patients before the study starts will avoid this dilution effect.
The problem is that the researchers have jumped from the frying pan of compliance problems into the fire of poor generalizability. Unlike the researchers, you do not have the option of only treating patients who are compliant. And you will not have any reasonable way to screen out those noncompliant patients for special handling. So excluding noncompliant patients causes the same problems as excluding children, women, or the elderly.
2.4.4 Intention to treat: What to look for.
When you are looking at compliance issues, consider the following issues:
- Was any attempt made to assess compliance?
- Was the compliance level similar to patients seen in your practice?
- Would additional analysis using the treatment actually received answer a different, but still important question?
2.5 Summary - Who was left out?
Exclusion of subjects can make the study biased or less generalizable.
Who was excluded at the start of the study? Excessively strict entry criteria in a research study can make it difficult to extrapolate to the types of patients that you normally see.
Who refused to join the study? Do the volunteers differ substantially from refusers in ways that might influence the outcome of the study?
Who dropped out during the study? Did these dropouts have a different prognosis?
Who stopped
