Why Evidence in Critical Care Is Often Insufficient

Many clinicians hold strong beliefs regarding the efficacy of ECLS for patients with severe ARDS despite a paucity of credible evidence. The highest quality evidence to guide decisions about ECLS for adults with ARDS comes from four randomized controlled trials (RCTs; Table 11-1 ). An actively enrolling prospective RCT (the Extracorporeal Membrane Oxygenation for Severe Acute Respiratory Distress Syndrome [EOLIA] trial) may also advance our understanding ( http://www.clinicaltrials.gov/ct2/show/NCT01470703?term=eolia&rank=1 ; http://revaweb.org/gb/etudes.php#e2 ). However, current ECLS literature is dominated by observational studies, clinical experiences, clinical reports, and opinions. Observational studies and trials of ECLS with low credibility are difficult to interpret for various reasons that have not changed over the past 30 years. These reasons, detailed later, are central to the controversy surrounding ECLS today.

It is generally accepted that scientifically rigorous clinical experiments provide the best foundation for the evaluation of the efficacy of clinical interventions. Personal clinical experiences, including observational case series and case reports, provide important preliminary information that can stimulate thinking and create hypotheses, but they fall short of rigorous compelling evidence that a therapy is either effective or efficacious. Unfortunately, rigorous experiments addressing the benefit for ECLS in ARDS are uncommon. Even clinical trial evidence, our most credible source of evidence, is often of low quality.

A persistent threat to the credibility of critical care trial results is the introduction of both random and systematic error. Systematic error (bias) is the more challenging and requires careful attention in experimental design. The belief that bias plays little role in clinical trials is incorrect for many critical care experiments. Differential (between-group) bias frequently exists because of uneven distribution of confounders, but it can also exist because of uneven distribution of the experimental intervention, especially in nonblinded (open) clinical trials. In ECLS trials, variable application of mechanical ventilation in both the intervention and control arm can be a confounder. Confounders introduced after subject assignment to the clinical trial groups are better termed “co-interventions” and should be distinguished from confounders that exist before subject allocation to the experimental groups.

Clinical trials that test complex and multifaceted interventions such as ECLS are particularly vulnerable to confounding from management issues such as transfusion or mechanical ventilation practices that are not uniform among experimental groups. Co-interventions in clinical trials are frequently neither controlled nor measured, and this deficiency threatens the internal validity of critical care clinical trials. In nonblinded (open) scientifically rigorous critical care clinical trials, all experimental arms require well-defined protocols that contain enough detail to standardize clinician decisions about both the experimental intervention and important co-interventions. In clinical studies of ECLS, between-group nonuniformity can occur in the management of the ECLS itself (the experimental intervention) because in the past, the methods used were commonly not replicable.

Inconsistency in study subject selection is an additional cause of variability in clinical ECLS study results. Randomization cannot account for differences between the study subjects and the population of interest from which they are derived. The patients who arrive at study institutions constitute a convenience sample; even a multicenter study may be seen roughly as a conglomerate of convenience samples. The process of obtaining consent may also result in a selection bias; many critically ill patients are unable to provide their own consent. A myriad of technical and personnel ECLS aspects must be considered. Therefore the link between any given patient and the ECLS study population may vary. This variation produces questions about generalizability (external validity) with almost all clinical studies. Consequently, clinicians who try to apply study results must always ask whether the patient under consideration belongs to the subset of subjects from which the study results were obtained and whether their setting is similar to the study setting, before using a study intervention. Guides to assess the evaluation of external validity have been published.

Meta-analyses are meant to overcome some of the limitations of many clinical studies by pooling their results, but the quality of a meta-analysis depends on the quality of the clinical studies on which it is based. In addition, there are several steps that must be performed to obtain reliable results through meta-analysis, but few analyses follow all the appropriate procedures. Many meta-analyses may produce positive results simply because of an insufficient sample size, without proper adjustment for multiple comparisons. Scientists expect experimental results that properly describe the way the world works to be independently reproduced by other investigators. For such results to be reproduced, it is required that the methods of the experiment be replicable. Unfortunately, the methods of most ECLS studies lack detail and are not replicable. Even the use of Bayesian methods (which adjust for uncertainty) would not overcome the limitations imposed by some methodological inadequacies of included studies. Therefore conclusions of meta-analysis results must be interpreted with caution.

Importance of Adequately Explicit Protocols for Clinical Trials

We define an adequately explicit protocol as a protocol with enough detail to respond consistently to changing patient conditions. Adequately explicit protocols generate specific instructions (patient-specific or personalized) that do not require judgments by the clinician. Although adequately explicit computerized protocols often contain the greatest detail, paper-based versions can also contain enough detail to be adequately explicit.

Most clinical study protocols are not adequately explicit. Even systematic and scholarly collections of flow diagrams commonly lack necessary detail and do not standardize clinician decisions. Most protocols can elicit different clinical decisions from different clinicians because clinician decision makers must fill in the gaps in the insufficiently detailed protocol logic. Clinicians’ judgments will vary with their backgrounds and experience, as will their choices of the rules and variables they use to fill in the gaps of inadequately explicit guidelines and protocols. This is a major contributor to unwarranted variation in clinical care. Protocols and flow diagrams are commonly but inappropriately called algorithms.

An algorithm in mathematics or engineering is a precise solution, although its definition allows for the more liberal use common in medicine (“a set of rules for solving a problem in a finite number of steps”). “Solving a problem” is the operative concept—our current techniques have not solved the problem. It is important to make this distinction between adequately explicit protocols and the more common guidelines and protocols because it may help us to develop more scientifically rigorous clinical trials for ECLS. Adequately explicit protocols can enable replicable clinical trial methods in multicenter trials of ECLS, thus enhancing the quality and reproducibility of future ECLS trial results.

History of ARDS Patient Survival and Management

ARDS therapy is usually supportive. Mechanical ventilation strategy, positive end-expiratory pressure (PEEP), inspired oxygen (O ₂ ), and breathing mechanics play central roles. Other therapeutic interventions that should be assessed include prone positioning, neuromuscular blockade, and intravenous fluid administration. Although diffuse, ARDS injury is not uniform, but this was not widely appreciated in early studies. The static thoracic compliance of ARDS patients appears to be directly proportional to the fraction of aerated lung, and only a small fraction of the lung appears to receive the tidal volume. After this understanding, newer therapeutic approaches focused on reducing the vigor of mechanical ventilation. These included intravenous oxygenation, permissive hypercapnia, pressure-controlled inverse ratio ventilation (PCIRV), low tidal volume mechanical ventilation, high frequency oscillatory ventilation, and airway pressure release ventilation, among others.

Survival of ARDS patients is highly variable. Survival from severe ARDS in the 1970s was as low as 10% to 15%. After 1988, survival from ARDS was higher. More recently, reported survival from ARDS has increased to 60% to 80%. This secular increase in survival, combined with advances in mechanical ventilation and in ECLS technology, makes historical comparisons challenging. Changes in ARDS definitions, including the new Berlin definition, and in trial enrollment criteria also make comparison between studies difficult. Few recent studies include clear patient criteria for ECLS therapy. Uniformly applied extracorporeal membrane oxygenation (ECMO) entry criteria, similar to those used in the 1970s trial of ECMO, could enhance replicability of subject selection in future ECLS trials.

The H1N1 epidemic was associated with an increase in ARDS incidence and stimulated use of newer therapeutic approaches, including ECLS, for ARDS. Unfortunately, the observational studies examining ECLS for ARDS after H1N1 influenza have the limitations described earlier. Two national ARDS registry reports led to conflicting conclusions regarding ECLS therapy for H1N1-induced respiratory failure. A retrospective cohort study matched 80 patients with H1N1 influenza–associated ARDS who were referred to one of four ECLS centers in the United Kingdom. Patients referred to an ECLS center survived at nearly twice the rate of the group who were not referred to an ECLS center. The results were consistent with three different methods of statistical matching. A major limitation to this design is the inability to control for confounders such as mechanical ventilation strategies and ECMO patient selection criteria. Patients with mechanically ventilated lungs from the ARDS Network H1N1 influenza registry were treated with lung-protective mechanical ventilation strategy (6 mL/kg predicted body weight), and some received ECLS. Survival of ARDS Network H1N1 influenza registry patients who received ECLS did not seem to differ from those who met ECLS eligibility criteria but were not treated with ECLS. ECLS eligibility was determined by the presence of severe hypoxemia within the first 7 days of mechanical ventilation. Of 600 adult patients with H1N1 and requiring mechanical ventilation, 31 patients received ECLS and 569 did not receive ECLS. Ninety-one (16%) of these 569 were deemed eligible for ECLS. Unadjusted 60-day survival did not differ between the ECLS-eligible group (66%) and the group that actually received ECLS (52%). In summary, survival of ARDS Network H1N1 registry patients who were eligible for ECLS but were treated with conventional therapy appeared similar to survival of ECLS-supported H1N1 patients in the United Kingdom and in the previously reported H1N1 patient survival in Australia and New Zealand.

Lung-protective ventilation (6 mL/kg predicted body weight tidal volume strategy) with the appropriate application of PEEP is the most credible evidence-based approach for management of ARDS. Theoretically, ECLS technology could allow almost complete lung rest. However, if or how total lung rest with ECLS might further increase survival remains unknown. A retrospective examination of mechanical ventilation practices during ECMO suggested that the higher PEEP levels in the first 3 days of ECMO therapy were associated with increased survival. The authors of that study concluded that further research on appropriate mechanical ventilation practice during ECLS was needed. More recently, researchers have described ECLS as a “super protective” mechanical ventilation strategy. The ECLS organization registry and reported case series indicate that prolonged (9 to 14 days) mechanical ventilation before ECLS support is common and is associated with decreased survival in adult patients. ECLS supporters argue that early intervention with ECLS increases survival by reducing mechanical ventilation exposure. These observations underscore the importance of both co-intervention protocols and unambiguous patient section criteria for ECLS therapy rather than reliance on clinician judgment for ventilator management or patient selection.

Unfortunately, the accuracy with which clinicians predict survival in individual patients with ARDS (compared with large groups) is frequently low. Using defined fraction of inspired O ₂ (F io ₂ ) and PEEP conditions for determining the ratio of partial pressure of arterial O ₂ (Pa o ₂ )/F io ₂ might enhance patient selection because F io ₂ and PEEP seem to predict patient outcome. In fact, this selection strategy was used in the 1970s ECMO trial, the first ECLS clinical trial. An important conclusion to be drawn from this discussion of variation in survival over time is that the use of historical controls for estimating ECLS efficacy is dangerous. This emphasizes the need for carefully crafted randomized controlled clinical trials.

Economic Feasibility of ECLS in Patients with ARDS

There are important unresolved cost-effectiveness issues that present practical barriers to widespread ECLS use for respiratory failure. Although the prices for ECLS systems are falling, training and maintaining a center is expensive and requires significant resources. Therefore it is not surprising that ECLS access across the United States is variable. Although the CESAR (Conventional Ventilation or ECMO for Severe Adult Respiratory Failure) trial authors asserted that ECLS could be cost-effective, their results should be interpreted with caution; the study design did not allow credible conclusions about the efficacy of ECLS. The cost of ECLS-related complications must also be incorporated. Zangrillo’s meta-analysis noted a 54% ECMO patient mortality with frequent complications, including renal failure, pneumonia, sepsis, and bleeding. In a recent hypothetical cost-effective simulation of ECLS with Markov chain analysis, Park showed that ECLS could be associated with acceptable costs, although the analysis did not account for training and personnel costs. The costs of keeping an ECLS team ready in places with low demand may be unjustified.

Some Principles and Objectives of ECLS

ECLS can support patient gas exchange (oxygenation and alveolar ventilation) and hemodynamic function with two general strategies of circulatory access: veno-venous (VV) or venoarterial (VA). With VA cannulation, blood is drained from the right atrium via the central venous system and returned to the proximal arterial system. VA support bypasses both ventricles and the intervening pulmonary system, unloading the patient’s natural heart and lung and providing gas exchange and hemodynamic support. In most cases, partial support is achieved, with some residual pulmonary blood flow present in the natural lung.

With VV cannulation, blood is drained and subsequently returned via the right internal jugular vein or femoral veins. VV support has its origins in the work of Kolobow, Gattinoni, and others, who introduced VV cannulation for extracorporeal CO ₂ removal (ECCO ₂ R). Newer cannulation techniques allow for higher blood flows and minimal recirculation and also can provide adequate support of oxygenation. Although VV support does not directly provide hemodynamic support, improved oxygen delivery may improve myocardial performance.

ECLS now includes older techniques that emphasize CO ₂ removal (ECCO ₂ R) or arterial oxygenation (ECMO). Modern ECLS equipment enables both of these and blurs the distinction between them. Technologic advances have improved cannula flows and mechanics, and VV has begun to supplant the VA approach, unless concomitant cardiac failure exists. Although VA cannulation can support lung and cardiac failure, VV cannulation is often preferred for patients who have adequate intrinsic cardiac function. VV ECMO use is increasing.

Patient selection criteria for ECLS vary considerably. These criteria also determine the ECLS technique (e.g., VV or VA ECMO or ECCO ₂ R). ECCO ₂ R has been used primarily in patients without refractory hypoxemia. Different cannulations, pump systems, and complication rates are reported for the different techniques. Therefore direct comparison of study results is complicated by the use of VV and VA ECLS techniques within individual published case series. This limits the ability of such case series to accurately inform readers and clinicians about the true effects, risks, and complications they may encounter if and when extracorporeal support is attempted at their local institutions.

Technical advances and extensive clinical experience have made it clear that patients with ARDS can be supported successfully with ECLS. However, technical accomplishment does not equal clinical efficacy.

Clinical Trial Evidence Addressing the Use of ECLS for ARDS

Because randomized controlled clinical trials provide the most compelling evidence for clinical decision-making, it is pertinent to note that only four RCTs of ECLS in adult respiratory failure have been published to date ( Table 11-1 ). ECLS enthusiasts often disregard the first two trials, noting their use of more complicated equipment, older techniques, and inexperience as the reasons for the negative trial results. They argue that current ECLS techniques save patients who are otherwise destined to die. However, Roger Bone cautioned against the early adoption of ECLS and suggested the need for clear diagnostic criteria and measurement of anticipated adverse effects before the widespread use of ECLS could be justified.

The first ECLS RCT in adults, the randomized multicenter trial of ECMO for ARDS, selected a subset of ARDS patients with severe disease and poor outcome—only 8 (9%) of 90 randomized patients survived with no difference in survival between ECMO and conventional care. Efforts to introduce widespread clinical use of ECMO for adults with ARDS were thereafter abandoned.

Kolobow, Gattinoni, and their colleagues subsequently introduced the concept of “lung rest.” The need to ventilate the injured natural lung could be reduced in proportion to the CO ₂ removed by a spiral silicone membrane (ECCO ₂ R). The ECCO ₂ R relieved the natural lung of some of its ventilatory burden. They intended to increase patient survival after reduction of the intensity of mechanical ventilation and of the consequent putative iatrogenic lung damage. The intermediate goal of their low-frequency positive-pressure ventilation extracorporeal CO ₂ removal (LFPPV-ECCO ₂ R) was to reduce the motion of the diseased lung to a minimum with almost complete elimination of ventilation (with only 3 to 5 breaths/min). This technique has shown benefit in recent small studies of patients with chronic obstructive pulmonary disease. The management of the natural lungs of the randomized patients in the National Institutes of Health (NIH) collaborative ECMO trial of 1974-1977 did not adhere to these principles of lung rest. Therefore a superimposed iatrogenic lung injury due to higher end-inspiratory pressures or tidal volumes to ARDS lungs of the study subjects might have introduced enough bias to affect the ECMO trial outcome.

Gattinoni et al. reported an increase in survival of ARDS patients after use of PCIRV followed by LFPPV-ECCO ₂ R, but their observational study was an uncontrolled clinical application. In the second RCT of ECLS in adults, Morris et al. subsequently observed a similar survival between a control and interventional group with an overall increased survival of all patients when compared with historical controls of the 1970s. Unexpectedly, the 42% survival of their control patients supported with continuous positive-pressure ventilation was not statistically significantly different from the 33% survival of patients supported with PCIRV/LFPPV-ECCO ₂ R.

A trial in which LFPPV-ECCO ₂ R was used as its primary intervention represents a significantly different intervention than the primary intervention of ECMO in the 1974-1977 trial ( Table 11-1 ). The higher survival of ARDS patients after support with LFPPV-ECCO ₂ R is intriguing. Pulmonary blood flow is likely an important determinant of lung response to injury. Pulmonary blood flow is preserved in VV LFPPV-ECCO ₂ R, whereas the 1974-1977 VA ECMO technique markedly reduced pulmonary blood flow to the natural lung. The preservation of natural lung blood flow, with the low natural lung ventilation, leads to a low overall ventilation-perfusion (V/Q) ratio during VV LFPPV-ECCO ₂ R. The 1974-1977 VA ECMO technique produced an oligemic natural lung with a high overall V/Q ratio. On the basis of observations in animals, a high overall V/Q ratio might cause lung necrosis in patients with ARDS. Both preserved pulmonary blood flow and lung rest are two significant differences between LFPPV-ECCO ₂ R and ECMO that may be important contributors to the difference in patient survival between the LFPPV-ECCO ₂ R and the 1974-1977 ECMO clinical trials. However, the high survival rate in the control group during the ECCO ₂ R trial suggests that other variables than the differences between ECCO ₂ R and ECMO contributed more significantly to the higher survival.

The highest quality randomized clinical trials of extracorporeal support for adults with ARDS enrolled only 90, 40, and 79 patients. The power to detect a real difference between control and LFPPV-ECCO ₂ R therapy group survival depends on the number of patients studied. Assuming that the observed survival rates of 42% for the control group and 33% for the LFPPV-ECCO ₂ R group represent the true survival rates of these two treatment groups, the number of study patients required to detect this difference in survival 80% of the time (power = 0.8) is approximately 400 in each treatment group. Only multicenter trials can provide sufficient patient enrollment to make such studies feasible. Detailed or adequately explicit protocols could enable the multiple clinical sites, in such a multicenter trial, to function as an extended laboratory with replicable methods.

The largest completed prospective ECLS trial to date, CESAR, was a multicenter trial. However, the methods of CESAR seemed to lack detailed co-intervention protocols and reproducible patient selection criteria for requiring ECLS. Central to their study was the identification of “potentially reversible respiratory failure” patients. Many patients randomized to the ECLS arm did not receive ECLS once transferred to the ECLS center, and several ECLS patients required VA rather than VV ECMO. However, the methods do not indicate exactly how this identification was achieved. Fundamental to the success of the CESAR trial was the trial design that included referral to a facility with ECLS expertise. The astute researchers recognized the need for ECLS-specific training, exposure, and experience. Peek et al. report an increase in 6-month disability-free survival in the ECLS group. Although some may find it reasonable to conclude from CESAR that ECLS in the hands of an experienced team poses no additional harm to adult patients with ARDS, we conclude that the trial, which was designed for effectiveness, not efficacy, does not provide adequate information to inform either regarding ECLS support for ARDS patients. There was no within-site allocation of patients into the two therapy arms. As a result, the likely substantial intersite variations of mechanical ventilation strategy could have created a design bias that significantly influenced the results. Patients treated at the ECLS referral center may have been more likely to receive standardized, lung-protective mechanical ventilation than those treated in other ICUs. Conventional care was neither protocolized nor clearly defined and was not likely the same at all referring hospitals.

In the Xtravent-study, Bein and colleagues asked if ECLS for CO ₂ removal plus very low tidal volume would improve ventilator-free days in patients with established ARDS. In this high-quality multicenter RCT of 79 patients with ARDS and a high plateau pressure (>25 cm H ₂ O), patients were randomized to ECCO ₂ R plus 3 mL/kg tidal volume versus conventional 6 mL/kg tidal volume ventilation. Clear inclusion and exclusion criteria ensured patient uniformity. Identical ventilator protocols (except for the tidal volume breaths [3 mL and 6 mL/kg]) served to standardize mechanical ventilation, an important study co-intervention. Despite the design strengths of this study, a time limitation was imposed on the investigators, and they were unable to meet their prespecified sample size. They concluded that ventilation with 3 mL/kg tidal volume and ECCO ₂ R was safe and feasible and not associated with a significant reduction in ventilator-free days at 28 days.

After the 2009 H1N1 novel influenza pandemic, several institutions reported observational data suggesting ECLS increased survival compared with historical survival data. Most of this literature comes from ECLS-positive case series and observational studies that likely include publication bias. More recently, two meta-analyses addressed ECLS for severe respiratory failure in adult patients. Zampieri et al. used a strict evaluation score of quality and only five studies based on a potential 49 were considered for the final analysis. The analysis included three studies (353 patients total) including the CEASAR RCT but excluded the older previously mentioned RCTs and included one retrospective case series and one case control analysis. Munshi included 10 studies (RCT and observational). Despite differences in conception and in study selection, both meta-analyses reached similar conclusions. More important, both were sensitive to the statistical method. A benefit of ECLS was not found in the main analysis and could only be found in subgroup or sensitivity analyses.

We repeated Zampieri’s meta-analysis with the highest quality evidence available and included the RCTs by Zapol, Morris, and Bein. In this analysis, the data from the CESAR study were included as presented in the “intention-to-treat” analysis (therefore not all patients in fact received ECLS), and data from a study by Noah et al. were added as reported by the authors in the propensity analysis with replacement (thereby assuring that both groups would be paired for illness severity). ⁶² This analysis workflow maximized group balance. The random-effects model again showed no survival benefit with ECLS with an odds ratio of 0.79 ( Fig. 11-1 ). Including the highest quality evidence (four RCTs and two case control studies), the survival benefit of ECLS for adults with severe respiratory failure remains undemonstrated.

Only gold members can continue reading. Log In or Register to continue

Share this:

• Click to share on Twitter (Opens in new window)
• Click to share on Facebook (Opens in new window)

Related

Related posts:

What Lessons Have Intensivists Learned During the Evidence-Based Medicine Era? What Is the Role of Invasive Hemodynamic Monitoring in Critical Care? What Fluids Should I Give to the Critically Ill Patient? What Fluids Should I Avoid? Should Exacerbations of COPD Be Managed in the Intensive Care Unit? Are Anti-inflammatory Therapies in ARDS Effective? Has Outcome in Sepsis Improved? What Has Worked? What Has Not Worked?