Clinical research is an essential feature of modern medical practice. Although most areas of medicine present challenges to conducting high-quality controlled studies, the prehospital and emergency care environments are particularly challenging due to numerous uncontrolled variables inherent in developing and implementing meaningful and ethical consent processes. Despite this, an ever-growing body of research in the area of EMS Medicine has helped solidify the subspecialty and moved prehospital care toward a more evidence-based practice. EMS physicians and medical directors must be able to interpret the literature and should aid in its advancement whenever possible.

• 
  

  Describe basic research concepts and definitions.

  
  
• 
  

  Describe patient groups and study designs.

  
  
• 
  

  Describe scientific levels of evidence and grades of recommendation.

  
  
• 
  

  Standardized reporting of clinical trials.

  
  
• 
  

  Describe the development of EMS research.

  
  
• 
  

  National Research EMS Agenda.

  
  
• 
  

  Describe the design of an EMS study, including development of the research question.

  
  
• 
  

  Describe the usual IRB process and approach to clinical research.

  
  
• 
  

  Describe the concept of emergency exception for informed consent.

  
  
• 
  

  Describe community consultation and public disclosure.

  
  
• 
  

  Define basic statistical terms.

  
  
• 
  

  Describe some basic research pitfalls.

  
  
• 
  

  List existing EMS research databases and discuss how to request access to the data.

  
  
• 
  

  List sources of potential funding for EMS-related research.

Scientific research is meant to draw investigators and readers of the scientific literature closer to “the truth.” Study design and limitations of conducting research in the clinical environment can drastically affect the results of clinical research, potentially altering the truth being sought. In cases where meticulous design considerations have limited the presence of bias and error, it is possible to utilize study results to advance the practice of medicine. This is the basis for the concept of evidence-based medicine. Acknowledging that not all research study results lead to applicable medical practices, it is important to consider how to best interpret current EMS research results and design future studies.

In order for EMS physicians to provide state-of-the-art medical direction and oversight of EMS systems, a basic grasp of the methodology behind clinical research is required. Research concepts and the definitions of common research terms are prerequisites for this understanding.

EFFICACY

Efficacy is a description of how well the treatment works in clinical trials (“explanatory”).

EFFECTIVENESS

Effectiveness refers to how well the treatment works in the practice of medicine (“pragmatic”).

VALIDITY

Validity in research is the degree to which a tool measures what it claims to measure. The validity of a study refers to determining the likelihood that the conclusions drawn from the study are correct or reasonable. A valid study asks the appropriate questions, uses the correct sample (in size and character), collects the correct outcome measures, and utilizes correct statistical methods. This is a very complex process that requires strict adherence and reassessments to produce a truly valid study.

Internal validity: Internal validity considers the direct effect of one variable (the independent variable) on another variable (the dependent variable). It is important in studies designed to show cause-and-effect relationships. At times, even properly designed studies may have confounding variables that interfere with internal validity.

Confounding variables: There are eight types of confounding variables that interfere with internal validity: (1) history—specific events occurring between measurements (in addition to any experimental variables); (2) maturation—participant changes over time (eg, becoming tired, growing older, etc); (3) testing effect—the effect that taking the first test has on taking any additional testing; (4) instrumentation— refers to changes in measurement tool calibration or the changes in observers that may change measurements; (5) statistical regression—when selected groups are selected based on their extreme score; (6) selection bias—results from the differential (nonrandom) selection of respondents for the comparison groups; (7) experimental mortality—the loss of participants from comparison groups; (8) selection-maturation interaction—occurs when participant variables (eg, hair or skin color) and time variables (eg, age, obesity) interact.¹

External validity: Relates to the extent to which the results of a (internally valid) study remain true in other cases (ie, different populations, places, or times). Can the study findings be generalized? Are the research participants representative of the general population? Many studies are performed in a single geographic area, with smaller samples or patients, who also possess unique characteristics or are representative of a specific population only (ie, volunteers, military cadets, medical students). Studies that are not generalizable have low external validity. Other factors adversely affect external validity include (1) testing effect; (2) selection bias; (3) experimental arrangements— which are not generalizable to patients in a nonexperimental setting; and (4) multiple-treatments interferences—effects of previous treatments that interfere with present testing and are not erasable.

Internal versus external validity: It might appear as if internal and external validity contradicts each other. If a strict adherence to experimental designs control as many variables as possible, the study may have high internal validity. Yet, this highly artificial setting lowers the external validity. Alternatively, in performing observational research, it is difficult to control for interfering variables and lowers the low internal validity. However, the study of environmental or other measures in a natural setting results in higher external validity. Fortunately, these apparent contradictions are resolvable, as a great many studies primarily wish to deductively test a theory, in which the major consideration is the rigor (internal validity) of the study.

BLINDING

Blinding refers to procedures undertaken to ensure that neither the study participants nor any member of the study team know to which group the participant belongs (treatment or nontreatment). Some studies have been classified as single, double, or triple blinded, depending on whether it was the participants, care team, or outcome assessors that were blinded. It is currently accepted that investigators should refrain from this terminology and simply state the type of blinding within the test of the paper.²

• 
  

  Unblinded/open/open label: These are trials without blinding.

INCLUSION CRITERIA

Inclusion criteria are conditions that must be met for the appropriate recruitment of subjects into a clinical study.

EXCLUSION CRITERIA

Exclusion criteria are conditions that must be met for the appropriate rejection of subjects from a clinical study.

STUDY ANALYSES

There are different paradigms for performing analysis of data from clinical studies. When attempting to limit bias and evaluate the effect of introducing a clinical intervention on a particular population it is best to utilize a study design that incorporates intention-to-treat analysis. Other poststudy analysis and interim analysis may be appropriate in some circumstances, but the conclusions drawn from these may be less accurate.

Intention–to-treat (ITT) analysis: The objective is to analyze each group exactly as they existed upon randomization. A true ITT analysis is possible only when complete outcome data are available for all randomized subjects. This means to include all subjects, including those that drop out. ITT analyses decrease outcome bias.

Subgroup analysis: Analyzing groups within the groups being studied. Subgroup analyses are discouraged because multiple comparisons may lead to false-positive findings that cannot be confirmed.

Interim analysis: A pretrial strategy for stopping a trial early if the results show large outcome differences between groups. It allows for periodic assessments for beneficial or harmful effect of treatment compared to concurrent placebo or control group while a study is ongoing. It is used as a cost-saving measure and importantly the ethical obligation that only the minimum number of patients should be entered into a trial to achieve the study’s primary objective and reduce the participants’ exposure to the inferior treatment. This analysis may occur following the inclusion of a certain number of treatments or after a set period of time. However, the way in which the interim analysis is to be conducted must be expressly stated in the study protocol. Additionally, the results of the interim analysis should be evaluated by an independent data monitoring committee.³ Other reasons to stop an interim analysis are that there are unacceptable side effects or toxicity, accumulation is so slow that the trial is no longer sufficient, outside information makes the trial unnecessary or unethical, poor execution compromises the studies’ ability to meet its objectives, or disastrous fraud or misconduct.

SCIENTIFIC BIAS

Bias usually refers to any unintended influence that a particular facet of the study design may have that will alter or skew the results. Typically investigators seek to limit bias as much as possible; however, much of the medical literature is affected by bias in some form.

Selection bias: Usually results from an error in choosing the individuals or groups to participate in a study. It distorts the statistical analyses and may result in drawing incorrect conclusions regarding the study outcome(s). It weakens internal validity.

Sampling bias: A systematic error in a study that occurs because the participants do not represent a random sampling of the population. This occurs in some instances due to participant self-selection or prescreening of trial participants. The result is that some members of the population are less likely to be included than others. It weakens external validity.

Attrition bias: A kind of selection bias caused by the loss of participants. It includes patients that dropout or do not respond to a survey (nonresponders), or who withdraw or deviate from the study protocol. It results in biased results because a study intervention or nonintervention is unequal or underrepresented in the outcome.

Publication bias: A bias regarding what is most likely to be published, positive or negative findings. If negative findings are underreported, it leads to a misleading bias in the overall published literature. Studies suggest that positive studies are three times more likely to be published than negative studies. Trial registration is now required by many journals to ensure that unfavorable results are not withheld from publication.⁴

RANDOMIZATION

Participants are arbitrarily assigned to a treatment (intervention) or nontreatment (control) group. Randomization eliminates bias in group assignment, aids in blinding the investigator, participants, and other assessors from knowing the grouping of study participants, and allows the use of probability theory to express the likelihood that any outcome differences between groups merely indicate a chance finding.

Cluster randomization: A preexisting group of study participants (schools, poisoning victims, families) are randomly selected to receive (or not receive) an intervention. Cluster randomization is sometimes done due to factors related to study participants (ie, all family members are placed in the same treatment or nontreatment group)

Factorial randomization: Each participant is randomly assigned to a treatment (or nontreatment group) that receives a particular combination of interventions (or noninterventions).

Randomization procedure: Generation of an unpredictable sequence of allocations to be distributed to participants to treatment or control groups using an element of chance, following the patient’s evaluation of eligibility and recruitment into the study.

Allocation concealment: Very strict protocols to ensure that patient group assignments are not revealed prior to their allocation to a group. Sequentially numbered, opaque, sealed envelopes (SNOSE) is a type of allocation concealment

TRIAL REGISTRATION

As of July 1, 2005, the International Committee of Medical Journal Editors (ICMJE) announced that all RCTs must be registered to be considered for publication in member journals. This registration may occur late.⁵

PATIENT GROUPS

During the design phase of a study, it is important to define groups. Based on the study type, there may be several different types of groups needed for a particular study.

Control group: A patient group that receives no treatment

Placebo control group: A patient group that receives no treatment, but will receive a “sham” or “placebo” treatment that will mimic what is being performed in the “test group” but without physiological or real effect.

Parallel group: Participants are randomly allocated to a group, and study participants either receive or do not receive an intervention.

Crossover group: Participants are randomly allocated to a group, and, over time, all study participants receive and do not receive an intervention in a random sequence.

Certain attributes of studies produce better evidence than others. It is important to consider the design of the study when determining how to interpret a study. Some studies may have seemingly ideal designs, whereas others do not. The constraints of design related to the clinical environment many times limit the investigators’ ability to design a study with all of the ideal parameters. In general, the following attributes can be used to distinguish between stronger and weaker level of evidence.

• 
  

  Randomized studies are superior to nonrandomized ones.

  
  
• 
  

  Prospective studies are superior to retrospective studies.

  
  
• 
  

  Blinded studies (in which patients, and clinicians and data analysts where possible, do not know which intervention is being used) are superior to unblinded studies.

  
  
• 
  

  Controlled studies are superior to uncontrolled ones.

  
  
• 
  

  Experimental study designs are superior to observational study designs.

  
  
• 
  

  Contemporaneous (occurring at the same time) control groups are superior to historical control groups.

  
  
• 
  

  Internal control groups (ie, managed within the study) are superior to studies with external control groups.

  
  
• 
  

  Large studies (ie, involving enough patients to detect with acceptable confidence levels any true treatment effects) are superior to small studies (ie, properly powered studies are superior to underpowered studies).

  
  
• 
  

  Studies that clearly define patient populations, interventions, and outcome measures are superior to those that do not clearly define these parameters.

RANDOMIZED CONTROLLED TRIAL

In a randomized controlled trial (RCT), subjects are assessed for eligibility and recruitment into the trial. Then, before the intervention or treatment begins, the patients are randomly allocated to receive one treatment or another of the treatments being studied. After the patients are randomized to one treatment arm or the other(s), they are followed in exactly the same way, with the only difference between the two groups being the randomly allocated treatment(s). RCTs are the gold standard for a clinical trial. They are often used to test the efficacy or effectiveness of various types of medical interventions within a patient population. An RCT must contain a control group or a previously tested treatment (a positive-control study). The advantage of an RCT is that they are so strictly controlled that they reduce study bias and the confounder of causality. The disadvantages of RCTs are that they are very expensive, time consuming, may have lower external validity, and perhaps unethical for certain orphan diseases (rare) or rapidly morbid or mortal conditions or injuries which do not lend themselves to RCT evaluation. The types of RCTs are listed below.

Superiority trials: Most RCTs are superiority trials, wherein one intervention is hypothesized to be significantly superior to another.

Noninferiority trials: These RCTs seek to determine whether a new treatment is no worse than an established treatment.

Equivalence trials: These RCTs seek to determine whether two interventions are indistinguishable from each other.

RANDOMIZED (UNCONTROLLED) CLINICAL/COMPARATIVE TRIAL

A randomized (uncontrolled) clinical/comparative trial is a trial that compares multiple treatment groups with each other in the absence of a control group.

NONRANDOMIZED TRIAL

These types of studies also called quasi-experimental studies. They do not have random assignments to either a control or treatment group. In these studies, the researcher controls the assignment to a grouping (ie, sicker patients receive the treatment) or circumstances dictate grouping (ie, patients presenting on nights and weekends are allocated to the control arm). These studies typically have lower internal validity because the two groups may not have comparable baselines.

OBSERVATIONAL STUDIES

Observational studies typically are designed to follow patients of a particular type, or whom have specific common features or risk factors.

Cohort studies: These studies are longitudinal studies, in that they look at an effect of an intervention on a patient population over time. It may assess risk factors in a group of patients who share common characteristics (ie, diabetes or heart disease risk factors) and it may or may not compare the effect of the study intervention to a comparison group (ie, a similar group of patients without the intervention) or to historical groupings previously studied. Several types of cohort studies are discussed below.

Prospective cohort studies: A study that sets out to evaluate patients that present forward from some set time. They usually seek to determine risk factors that affect the studied group over time. These are important studies in that it is unethical to perform such as study as an RCT and purposely expose patients to risk factors. Prospective studies are of a higher level of evidence than retrospective studies.

Retrospective cohort studies: These studies are also called historic (post hoc) cohort studies. They review the records of patients that have already been treated or an event that has already taken place. These studies are easier to complete than prospective studies, less expensive and may allow for the study of rare diseases. The disadvantages are that some important statistical measures cannot be made and significant biases (selection and informational) may confound the results. Moreover, these studies rely on precise record taking that has already taken place, which may have key data points, epidemiological, or treatment information missing or unavailable.

TIME SERIES STUDIES

Time series studies are studies run through a period of time, in order to test a hypothesis or make an observation.

Case-controlled study: This is an observational study in which two groups with different outcomes are identified and compared. These studies are often performed to identify contributing factors in patients with a particular condition with patients who do not have that particular condition. They are simpler to perform than other studies and less expensive. Additionally, they are usually used as preliminary studies for research questions related to topics that have limited known information. However, the conclusions drawn may be weaker than more rigorously performed studies.

Nested case-control study: This variation of a case-controlled study evaluates only a subset of controls from the cohort to the incident cases. This subset is selected to match risk sets with the incident cases. In a case-cohort study all incident cases are compared. These studies may be analyzed using methods that take missing covariates into consideration.

Cross-sectional studies: These studies are also called cross-sectional analyses, transversal studies, or prevalence studies. These are observational studies involving either all members of a population or a representative subset of that population, at a specific point in time.

Ecological study: This is an epidemiological study in which large populations, rather than individuals are evaluated. These studies are considered inferior to cohort and case-controlled studies because of the concept of ecological fallacy (a false interpretation of the data made because inferences are made regarding individuals based on inferences from the group to which those individuals belong).

This is a ranking system used in evidence-based medicine to assign a strength rating to a clinical trial or research study. There are five levels of evidence (I, II, III, IV, and V) with alphanumeric subsets (Table 10-1).