Recent enhancements to MEDLINE have allowed for more sophisticated search techniques and HIRU has developed and tested strategies for improving the yield of clinically important and valid information from MEDLINE. First, the wider use of more informative on-line abstracts has been promoted. Most clinicians draw conclusions about the applicability of study results directly from the on-line abstract. More informative abstracts of original research (Ad Hoc, 1987; Haynes, 1990a), review articles (Mulrow, 1988), and practice guidelines (Hayward, 1993) give key information about the study purpose, patients, settings, methods, interventions, results, and conclusions. The structured summaries provide more information than conventional abstracts (Taddio, 1994; Narine, 1991; Counams, 1990), highlight methodologic considerations and facilitate more effective textword searching.
To evaluate the effectiveness of methodology-based MEDLINE searching, HIRU (Haynes, 1994a) studied the retrieval characteristics of index terms and textwords alone and in combination. We compared MEDLINE retrievals for each citation with the results of a manual review of 10 core medical and internal medicine journals (American Journal of Medicine, Annals of Internal Medicine, Archives of Internal Medicine, BMJ, Circulation, Diabetes Care, Journal of Internal Medicine, Journal of the American Medical Association, Lancet, and New England Journal of Medicine) for the years 1986 and 1991. Our results show that various combinations of index terms and textwords can be used to increase the number of relevant citations or the proportion of relevant citations retrieved. For example, in searching for studies that evaluate diagnostic procedures in adult medicine, the one-term strategy SENSITIVITY (TEXTWORD) retrieved up to 57% of relevant articles; the strategy EXPLODE SENSITIVITY AND SPECIFICITY (INDEX TERM) OR SENSITIVITY (TEXTWORD) OR DIAGNOSIS (PRE-EXPLODED SUBHEADING) OR DIAGNOSTIC USE (SUBHEADING) OR SPECIFICITY (TEXTWORD) retrieved 92% of the relevant citations with 27% irrelevant citations. To maximize the proportion of relevant citations, the strategy EXPLODE SENSITIVITY AND SPECIFICITY OR PREDICTIVE AND VALUE (TEXTWORD) retrieved 55% of the relevant citations available with less than 2% irrelevant citations. These results may be artificially high because only methodology terms were tested, content was not included in the searching strategies, and only core journals were included in the manual review.
MEDLINE, however, has three major drawbacks that limit its utility in busy patient care settings. First, high-yield searches can take as much as 30 minutes to complete --much longer than physicians have to spend with each patient. Second, individuals studies rarely offer conclusive information and so results from several studies must be combined. Finally, the patient characteristics, settings, disease severity and comorbidity featured in high quality studies may differ significantly from the sorts of patients seen in typical practice. Clinicians need digests of the clinically relevant literature. These digests serve EBM to the extent that they highlight the validity, importance and applicability of multiple studies of a particular health problem.
Textbooks are traditional sources of synthesized information. Unfortunately, textbooks are often out of date even at the time of publication. Most lack a systematic approach to evidence surveillance, summarization, and citation. A notable exception is Scientific American Medicine (Dale, 1995). Its large team of authors regularly updates chapters and its subscribers regularly receive packets of updates. An electronic version, on CD-ROM, is updated 4 times a year. Chapter authors must comply with strict editor's rules about the relevance and significance of citations to the recent medical literature. Less comprehensive evidence-based texts now serve specific areas of medicine. Examples include three monographs about screening and diagnostic tests (Sox, 1987; Eddy, 1991; Panzer, 1991). The Effective Care in Pregnancy and Childbirth (Chalmers, 1989) and the Effective Care of the Newborn Infant (Sinclair, 1992) volumes from McMaster University, Oxford University and several other institutions also emphasize critical appraisal of the best available evidence from the medical literature.
ACP Journal Club
The ACP Journal Club (1995) exemplifies a new breed of EBM publications. J Club is a bimonthly abstract journal for internists published since 1991 by the American College of Physicians (ACP) with the editorial office at HIRU. Its stated purpose is "to select published articles according to strict criteria and to abstract those studies and reviews that warrant immediate attention by physicians attempting to keep pace with important advances in treatment, prevention, diagnosis, cause, prognosis, and economics of the disorders managed by internists." J Club staff at McMaster review about 50 journals of relevance to internal medicine and select only those articles that meet explicit criteria for scientific merit and clinical importance. They then prepare a structured abstract in accordance with a detailed protocol and solicit clinical feedback from experts in the subject area of the study. The abstract and commentary are published on a single page for easy reading and use.
The selection criteria for studies abstracted in ACP Journal Club emphasize the quality of evidence that might justify changes in clinical practice. For example, a study reporting the progress and outcome of a specific disease over time (prognosis) must have recruited patients at an early and uniform stage of their disease (inception cohort). This could be at first diagnosis for persons with diabetes mellitus or rheumatoid arthritis or at first symptoms for Alzheimer disease or myocardial infarction (heart attack). The cohort must be followed long enough for the health condition to progress (e.g., five years or more for multiple sclerosis) and at least 80% of the participants must have been accounted for at the end of the study. Different criteria pertain for articles that answer different types of clinical questions. ACP Journal Club inclusion criteria for review articles require a statement of methods that describes how primary articles were defined, retrieved, and screened. Such statements must be specific enough that the search strategy could be repeated, with duplicate results. Application of strict methodological and clinical inclusion criteria for ACP Journal Club has the beneficial effect of radically reducing the number of articles that merit careful attention from practitioners, and fastidious abstraction by ACP Journal Club staff. However, the cost is a huge effort to properly screen all journal articles for potential interest to internists.
Reactions to the J Club have been very favorable; a recent survey conducted by the ACP suggests that it is one of the few medical journals read by near 100% of its recipients. In addition, the NLM now indexes J Club commentaries, facilitating access to J Club resources through MEDLINE. An electronic ACP Journal Club will be released in 1995 and promises to optimize this evidence-based information tool for searching, in addition to browsing for important advances in medical practice. After five years of steady growth, the accumulated database of structured abstracts is now large enough to afford useful answers to a wide range of clinical questions with over 750 studies abstracted. Other medical disciplines such as pediatrics (Current, 1995) and sports medicine (Mohtadi, 1994) have developed their own abstracting sections. The ACP is working with the British Medical Association (BMA) to publish a companion product for all areas of medicine. The new journal, called EBM, will supplement a core of J Club articles with abstracts of studies from psychiatry, surgery, obstetrics and gynecology, family medicine, public health, and pediatrics. A sample issue will appear in September, 1995 and full production begins in January 1996.
Systematic Overviews
Another rapidly maturing subset of the medical literature is secondary publications that summarize original research appearing in primary journal articles. Good quality overviews help clinicians by doing much of the work of critical appraisal pertaining to specific health conditions. The NLM helps clinicians by publishing its Bibliography of Review Articles at in the front of each volume of Index Medicus and by indexing different types of reviews with specific "publication type" tags in MEDLINE.
Standards are emerging for evidence-based systematic overviews of evidence and meta-analyses of data from multiple studies (Mulrow, 1987). Meta-analysis was developed in the humanities and education fields in the early 1970s (Glass, 1981) and has been embraced by medicine as a strategy for extending the power and applicability of results from large numbers of primary studies. Overviews and meta-analyse
