Asking a Question

The first step in practicing EBM is asking a well-constructed clinical question. To benefit the patient and aid the clinician, clinical questions must be both directly relevant to patients’ problems and constructed in a way that guides an efficient literature search to relevant and precise answers. The Centre for Evidence Based Medicine (CEBM) in Oxford, England, provides an excellent description of the four essential elements of an EBM question, summarized in Table 227-1.

TABLE 227-1 Four Essential Elements of a Well-Constructed Clinical Question

Developing a specific, thoughtful question leads to a much more efficient search for the answer. Search results themselves can be used to further refine a question. For example, too many results may indicate the question is too broad, and too few results often necessitate a broader description of the patient population, intervention, or outcome.

Types of Evidence

After the question is formulated, one must consider the type of question being asked. Different types of studies, based on their size, design, and methodology, provide evidence of differing quality and relevance to a research question. For example, is the question about therapy, prevention, etiology, or harm? A randomized controlled trial (RCT) or (better yet) systematic review of RCTs will provide the best evidence for this kind of question. Is the investigator interested in the prevalence of a specific disease or symptom in the general population? If so, a large cohort study will best answer this question.

Randomized clinical trials, also referred to as experimental or interventional studies, are the cornerstones of medical evidence. Physicians place considerable faith in the results of randomized control trials.^4,⁵ This faith is placed with good reason, as randomization remains perhaps the best solution to avoid misinterpreting the effect of a therapy in the presence of confounding variables.⁶ When participants are randomly allocated to groups, factors other than the variable of interest (e.g., a new therapy for sepsis) that are likely to affect the outcome of interest are usually distributed equally to both groups. For example, with randomization, the number of patients with underlying comorbidity that may adversely affect outcome should be similar in each study arm, presuming sample size is appropriate. A special advantage of randomization is that this equal distribution will occur for all variables (excluding the intervention) whether these variables are identified by the researcher or not, thus maximizing the ability to determine the effect of the intervention.

However, RCTs are expensive, difficult, and sometimes unethical to conduct, with the consequence that less than 20% of clinical practice is based on the results of RCTs.⁷ Moreover, many important questions such as determining the optimal timing of a new therapy or determining the effects of health care practices cannot practically be studied by RCTs.

Observational Studies

The principal alternative approach to the RCT involves observation rather than experimentation. Prior experience has biased us to favor RCTs, but partly in response to the increasing need to answer questions unanswerable by the RCT, the design and execution of observational outcomes studies have become much more sophisticated.

Observational outcomes studies are very powerful tools for addressing many questions that RCTs cannot address, including measuring the effect of harmful substances (e.g., smoking and other carcinogens), organizational structures (e.g., payer status, open versus closed ICUs), or geography (e.g., rural versus urban access to health care). Because of their cost and the regulatory demands on drug and device manufacturers, RCTs are frequently designed as efficacy studies in highly defined patient populations with experienced providers and therefore provide little evidence about effectiveness in the “real” world.⁸ Alternatively, observational studies can generate hypotheses about the effectiveness of treatments that can be tested using other research methods.⁸ Investigators have also explored the effects of different therapies that are already accepted but used variably in clinical practice.⁹

There are a number of different kinds of observational studies, each designed to address a different type of clinical question. These include case-control, cross-sectional surveys, and cohort studies. Case-control studies compare a group of patients with a disease or symptom of interest to a selected control group. They have the advantage of being quick and relatively inexpensive to perform and are often the only feasible study method for very rare disorders or when the lag time between an exposure and the related disease is very long. They can also be conducted with a relatively small number of patients. Cross-sectional studies provide a snapshot of a population at one point in time. They can also be conducted inexpensively and in a short time. Cohort studies prospectively identify an at-risk group (the inception cohort) and follow them through time, recording exposures and development (or not) of the disease under investigation. Cohort studies have a number of strengths, including the ability to match subjects to controls for some confounders, establish the timing and sequence of events, and standardize eligibility criteria and outcome assessments; they are easier and less expensive to conduct than RCTs.

However, observational studies have several significant limitations. First, the data source must be considered. Observational outcomes studies are often performed on large data sets wherein the data were collected for purposes other than research. This can lead to error owing to either a lack of pertinent information or bias in the information recorded.¹⁰ Second, one must consider how the authors attempt to control for confounding. The measured effect size of a variable on outcome (e.g., the effect of the pulmonary artery catheter on mortality rate) can be confounded by the distribution of other known and unknown variables. More specifically, case-control studies are subject to recall and selection bias, and the selection of an appropriate control group can be difficult. Cross-sectional studies can only establish association (at most), not causality, and are also subject to recall bias. Cohort studies have a number of limitations, including difficulty in finding appropriate controls and difficulty determining whether the exposure being studied is linked to a hidden confounder, and the requirement of large sample size or long follow-up to sufficiently answer a research question can be timely and expensive.

Case Reports or Case Series

The last form of primary research is the case report or case series. A case is a published account of a single or small number of patients and their response to a particular therapeutic intervention. The inability to generalize from a case report makes it the weakest form of clinical evidence available. However, case reports may be the only available or practical information in support of a therapeutic strategy, especially in the case of rare diseases when the evolution of the therapy predates the common use of randomized study designs in medical practice. This is also true for new therapies that have not yet been tested in clinical trials.

Summaries of Primary Research

Another valuable source of information, especially for the busy clinician with limited time for reading and research, is primary research that has already been summarized and evaluated. There are a number of high-quality, peer-reviewed sources of summary information, including those that summarize the results of individual trials and those that combine and summarize the results of multiple trials addressing the same topic. The following is a description of the most common types of literature summaries.

Single-Study Results—Critically Appraised Topics

Determining which studies provide information useful in the care of patients is largely a question of deciding whether a study is valid and, if so, can its results be applied to the patients in question. One format for appraising individual studies is the critically appraised topic (CAT) format that has been popularized as part of EBM. The purpose of the CAT is to evaluate a given study or set of studies using a standardized approach. Studies that address diagnosis, prognosis, etiology, therapy, and cost-effectiveness all have a separate CAT format.³ An example is shown in Box 227-1 for studies that address therapy. The CAT format for studies on therapy asks several questions intended to address the issues of validity and clinical utility. Studies that fail to achieve these measures are not generally useful, although studies do not necessarily have to fulfill every criterion, depending on the nature of the topic. For example, a study that examined the effect of walking once a day for the prevention of stroke would not be expected to include a detailed examination of side effects or a cost-effectiveness analysis. However, a study comparing streptokinase to placebo for treatment of stroke would likely be required to include a detailed examination of side effects and a cost-effectiveness analysis because of the excessive risks and costs associated with such therapy. Similarly, blinding may not always be possible, and the effects of the investigators being unblinded can be minimized by separating them from the clinicians making the treatment decisions or by establishing standard treatment protocols that are applied equally to both the study and control groups. Alternatively, a study would be “fatally flawed” if it failed in terms of randomization or was not analyzed as “intention to treat.” There are a number of other useful tools for assessing study design and for quantifying effect size and cost-effectiveness. In general, these are the tools of epidemiology and biostatistics, and their discussion is beyond the scope of this chapter. A basic primer and glossary of terms is included in Table 227-2.

Box 227-1

Critical Appraisal of the Literature