Abstract
Purpose
Ineffective respiratory efforts during expiration (IEE) are a problem during mechanical ventilation (MV). The goal of this study is to validate mathematical algorithms that automatically detect IEE in a computerized (Better Care®) system that obtains and processes data from intensive care unit (ICU) ventilators in real time.
Methods
The Better Care® system, integrated with ICU health information systems, synchronizes and processes data from bedside technology. Algorithms were developed to analyze airflow waveforms during expiration to determine IEE. Data from 2,608,800 breaths from eight patients were recorded. From these breaths 1,024 were randomly selected. Five experts independently analyzed the selected breaths and classified them as IEE or not IEE. Better Care® evaluated the same 1,024 breaths and assigned a score to each one. The IEE score cutoff point was determined based on the experts’ analysis. The IEE algorithm was subsequently validated using the electrical activity of the diaphragm (EAdi) signal to analyze 9,600 breaths in eight additional patients.
Results
Optimal sensitivity and specificity were achieved by setting the cutoff point for IEE by Better Care® at 42%. A score >42% was classified as an IEE with 91.5% sensitivity, 91.7% specificity, 80.3% positive predictive value (PPV), 96.7% negative predictive value (NPV), and 79.7% Kappa index [confidence interval (CI) (95%) = (75.6%; 83.8%)]. Compared with the EAdi, the IEE algorithm had 65.2% sensitivity, 99.3% specificity, 90.8% PPV, 96.5% NPV, and 73.9% Kappa index [CI (95%) = (71.3%; 76.3%)].
Conclusions
In this pilot, Better Care® classified breaths as IEE in close agreement with experts and the EAdi signal.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
Introduction
Patient–ventilator dyssynchrony affects 35–43% of mechanically ventilated patients [1–3]. Patient–ventilator dyssynchrony has been associated with patient outcomes: increasing duration of mechanical ventilation (MV), need for tracheostomy, and respiratory muscle injury [1, 4–6]. The most common patient–ventilator dyssynchronies are autotriggering, ineffective inspiratory efforts during expiration (IEE), and double triggering [1, 6]. IEE are defined as muscular contractions of the inspiratory muscles, primarily the diaphragm, which are unable to trigger the ventilator to inspiration. Currently, IEE can only be detected at the bedside by directly observing the patient’s inspiratory muscle contraction, by observing physiologic waveforms displayed by the ventilator, or by applying dedicated algorithms in investigational studies [3, 6–10]. Esophageal pressure tracings are also used for this purpose, but this technique is invasive, tracings are contaminated by artifacts that often require repositioning of the esophageal catheter, and interpretation requires special skills [11].
Neurally adjusted ventilatory assist (NAVA) is a new mode of assisted MV [12–14]. During NAVA, the electrical activity of the diaphragm (EAdi) is assessed by a special nasogastric tube. NAVA detects a representative EAdi signal that can be used to trigger the ventilator and manage gas delivery or to monitor the activity of the diaphragm [15]. We hypothesized that IEE can be detected by continuous automated monitoring of airway pressure and flow waveforms (Better Care®). The goal of this pilot study is to determine the accuracy of the Better Care® system for automatic detection of IEE compared with expert clinicians and the diaphragmatic EAdi signal. Presented here are pilot data on the first application of the Better Care® system to a group of critically ill mechanically ventilated patients. Preliminary results of this study have been reported in abstract form [16].
Materials and methods
Software
We developed software (Better Care®, Barcelona, Spain) capable of acquiring, standardizing, synchronizing, analyzing, and recording signals from digital medical devices, mainly bedside monitors and mechanical ventilators. Better Care® interacts with the output signal of medical devices rather than directly with patients. Specific drivers were also developed to allow connection to each medical device. Mechanical ventilators and bedside monitors were connected to Better Care® using an ED41000P2-01 remote access server (Lantronix, Irvine, CA). Better Care® standardizes the signals (associates each recorded curve) and resamples the signals at 200 Hz (i.e., 200 samples per second). Standardized signals are then analyzed, tagged, converted to digital imaging and communications in medicine (DICOM) format, and stored in the hospital picture archiving and communication system.
Patients
Data were prospectively obtained from eight patients older than 18 years of age admitted consecutively to four specific beds in a general intensive care unit (ICU) equipped with Better Care® in Hospital de Sabadell (Sabadell, Spain). All patients were ventilated for at least 24 h. Table 1 summarizes clinical characteristics, ventilator settings, and respiratory mechanics [also see Electronic Supplementary Material (ESM) Table 1] of these patients. Subsequently, we studied eight selected patients admitted to the Department of Perioperative Medicine, Intensive Care and Emergency, Cattinara Hospital (Trieste, Italy) and ventilated with Servo i (Maquet, Sweden) with a NAVA EAdi catheter recording EAdi data [15]. For this analysis, patients with suspected phrenic nerve dysfunction, neuromuscular disorders capable of altering EAdi signal quality, or gastroesophageal surgery were excluded, and only stable patients with good EAdi signal were included. This ensured that the EAdi data were appropriate for analysis. The EAdi data were time-synchronized with the Better Care® system. In addition, use of the EAdi catheter and NAVA in spontaneously breathing patients is the standard of practice for the unit. Table 2 summarizes clinical characteristics, ventilator settings, and respiratory mechanics (also see ESM Table 2) of these patients.
The investigators were not involved in any clinical decisions. The Institutional Ethics Committees at both institutions approved the study and decided to waive informed consent because the study was observational, signals were encrypted to ensure privacy, and no extra effort for personnel or changes to usual treatment were required.
Medical devices
Patients were ventilated with one of two ventilators: the Evita 4 (Dräger, Germany) or the Servo i (Maquet, Sweden), and mainly in two modes: volume assist control (VCV) and pressure support (PS), as described in Tables 1 and 2. We recorded a total of 2,608,800 breaths from the eight patients monitored with Better Care®. In the eight patients ventilated with the EAdi catheter, 57,323 breaths were collected. From these, 9,600 breaths were randomly selected and analyzed with Better Care®.
IEE analysis with Better Care® software
For each breath, the Better Care® software calculated a theoretical mono-exponential expiratory flow curve and compared it with the actual expiratory flow curve by evaluating its percentage deviation (0%, no deviation; 100%, maximum deviation). The method used to identify an IEE was as follows: First, the patient’s theoretical expiratory flow curve was estimated by examining expirations in which there was no deviation that could be an IEE. These curves were then averaged to produce the ideal curve. Second, the ideal curve was then compared with the patient’s actual expiratory flow curve. Third, during this analysis four deviations that represented the configuration of the missed inspiration and the subsequent expiration were compared with the ideal expiratory curve. Fourth, each of these four comparisons were given a weight that was eventually converted into a percentage deviation. Fifth, in addition, the algorithm also identified the presence of secretions and corrected the IEE identification algorithm to account for their presence. Sixth, based on our expert comparison, if the level of deviation equaled or exceeded 42%, the breath was considered an IEE. Figure 1 illustrates a number of breaths with different levels of deviation from the theoretical mono-exponential flow curve. From the recorded breaths, we selected the first 128 breaths spanning the whole range of deviation from the ideal expiratory curve from each of the eight patients to obtain a sample of 1,024 breaths. We considered the 1,024 breaths a fair estimate of the number of breaths an expert can evaluate without getting tired or losing concentration. A randomized sequence of the 1,024 JPEG images showing airflow and airway pressure was presented to the study team (Fig. 2). After the percentage deviation of each breath was determined and each record was labeled, we compared the results of the Better Care® software analysis with the experts’ decision about each of the 1,024 breaths. We did not analyze the first 100 ms of expiration to avoid confusion with prolonged inspiration due to premature termination of mechanical inspiration with IEE [6].
IEE analysis by experts
A team of five expert intensivists, each trained in identification of IEE, analyzed the previously selected 1,024 breaths using software designed for this purpose. Individually, each expert categorized each selected breath as IEE, not IEE, or unknown (Fig. 2). The team was instructed to classify a breath as an IEE if a positive tidal swing occurred during expiration whether or not a small negative swing was present in the airway pressure tracing without triggering the ventilator to mechanical inspiration. The study team was unaware of the system’s evaluation of the breath until the end of the data analysis. Provided there were at least three valid responses from the experts (no abstentions), the ad hoc program labeled the breath as IEE or not IEE. Otherwise, the breath was then evaluated in consensus. During consensus evaluation, the breaths were presented to all the members of the study team on a large screen and they were encouraged to discuss why they considered it was or was not an IEE (Fig. 2). After discussion, the members were again asked for a decision. Breaths in which at least three members did not provide a valid vote were discarded from analysis. Breaths in which the winning decision did not achieve at least four votes were excluded from the analysis.
IEE analysis with EAdi signal
Increases in EAdi of >1 μV from basal expiratory EAdi not followed by a ventilator breath during expiration were considered IEE. The cutoff point for EAdi of >1 μV was arbitrarily determined and corresponds to the mean value of the NAVA trigger [14, 17]. Airway pressure, flow, and EAdi were acquired at 100 Hz from the ventilator via a RS232 interface connected to a computer using commercially available software (Servo i RCR, version 2; Maquet Critical Care). Once captured, the signals were converted to the extensible markup language (XML) standard used by Better Care®. The processing module of the software was modified to detect a value higher than 1 μV in the EAdi curve during expiration, so we were able to test the Better Care® software to detect IEE against the EAdi value during expiration for every single breath (Fig. 3).
Statistical analysis
We used a receiver operating characteristic (ROC) curve to determine the diagnostic accuracy and the optimal cutoff point for the IEE score. Agreement between experts regarding the presence of IEE in expiratory airflow was assessed with the Kappa statistic. A Kappa statistic >0.75 is considered excellent agreement [18].
A true-positive result was defined as the presence of an IEE in the Better Care® software analysis of expiratory airflow and a positive decision by experts. A true-negative result was defined as the absence of an IEE in the Better Care® software analysis of expiratory airflow and a negative decision by experts. A false-positive result was defined as the presence of an IEE in the Better Care® software analysis of expiratory airflow and a negative decision by experts. A false-negative result was defined as the absence of an IEE in the Better Care® software analysis of expiratory airflow and a positive decision by experts. Standard formulas were used to calculate sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV).
For the eight additional patients ventilated with an EAdi catheter in place, a true-positive result was defined as an IEE detected with the Better Care® software and a positive EAdi signal. A true-negative result was defined as the absence of an IEE detected by the Better Care® software and a negative EAdi signal. A false-positive result was defined as an IEE detected with the Better Care® software and a negative EAdi signal. A false-negative result was defined as the absence of an IEE detected with the Better Care® software and a positive EAdi signal.
Results
Tables 1 and 2 describe patient characteristics, ventilator setup, and respiratory mechanics (also see ESM Tables 1 and 2). Of the total 1,024 breaths analyzed by the experts, 87 were sent to consensus, and experts failed to reach a consensus on 17 of these 87 breaths. These 17 breaths were excluded from analysis. The experts found 271 IEE in the 1,007 breaths analyzed. Table 3 shows the contingency table for the comparison between IEE analysis by the expert intensivists and Better Care®. From the expert study, the area under the curve (AUC) of the receiver operating characteristic (ROC) curve was 0.964 [CI (95%) = (0.952; 0.975)], demonstrating good performance of the IEE algorithm (Fig. 4). When an IEE score of 42% (percentage deviation from the ideal expiratory curve) was used as a cutoff, Better Care® achieved sensitivity of 91.5%, specificity of 91.7%, PPV of 80.3%, NPV of 96.7%, and Kappa index of 79.7% [CI (95%) = (75.6%; 83.8%)].
When compared with an EAdi change of >1 μV from baseline expiratory EAdi during expiration and not followed by a ventilator breath during expiration, Better Care® yielded sensitivity of 65.2%, specificity of 99.3%, PPV of 90.8%, NPV of 96.5%, and Kappa index of 73.9% [CI (95%) = (71.3%; 76.3%)]. Table 4 shows the contingency table for the comparison between EAdi data used to identify IEE and Better Care®.
Discussion
Our pilot study in a limited number of patients demonstrates that Better Care® can identify IEE during MV with similar accuracy to clinicians observing the waveforms displayed on the ventilator’s interface. Moreover, all records of IEE can be stored for later processing. The Better Care® system is designed to work with any ventilator.
The capabilities and performance of mechanical ventilators have evolved dramatically over the last 30 years; however, they are still unable to detect patient–ventilator asynchronies. IEE may increase duration of MV, ICU stay, number of tracheostomies, and costs [1, 5]. Furthermore, even in an ideal situation in which a highly trained physician or respiratory therapist would always be available in the ICU, patient–ventilator synchrony cannot be ensured at all times because the patient’s needs change over time. Thus, an automated monitoring system capable of detecting asynchrony could have a significant clinical impact. Moreover, de Wit et al. [19] found that deep sedation is a predictor of ineffective triggering, highlighting the need for automated systems that can detect IEE in comatose patients.
The Better Care® IEE algorithm analyzes perturbations in the expiratory flow pattern and detects IEE with good accuracy. During exhalation, however, other perturbations in the flow pattern can occur (noise) [20, 21]. So, the cutoff point for the algorithm’s identification of IEE must be chosen to minimize false positives while keeping the rate of undetected IEE low. Colombo et al. [10] found that the ability to recognize patient–ventilator asynchronies by visual inspection of flow and airway pressure tracings was overall quite low, although clinician expertise increased sensitivity for breath-by-breath analysis. This article highlights the need for computerized systems to facilitate recognition of these events. Chen et al. [3] also designed software that analyzes pressure and flow signals to detect IEE. They connected patients to an external monitoring system and recorded flow and pressure values for 10–30 min. Subsequently, they used changes in the slope of the airway pressure and flow curves to detect IEE. This method, however, is highly affected by noise. The authors found good sensitivity (91.5%) and specificity (96.2%), but patient’s airways were carefully suctioned before the measurements. As secretions and/or cardiogenic oscillations are the main source of noise [20, 21], specificity could have been overestimated since the method was not evaluated in the presence of secretions. Mulqueeny et al. [8] used a pattern classification algorithm to detect IEE by analyzing flow waveforms in 23 patients ventilated invasively and noninvasively with PS in a general ward. Data were limited to 10–20 min of ventilatory records. They found good specificity (98.7%) and overall accuracy (94.5%) but low sensitivity (58.7%). The low sensitivity was attributed to timed PS breaths (ventilators that intentionally ignored patient’s inspiratory efforts) and early cycling. In our study, variations from the theoretical mono-exponential expiratory flow curve in the first 100 ms of expiration were not analyzed to avoid confusion from prolonged inspiration due to premature termination of mechanical inspiration and IEE [6, 22]. Finally, other authors have reported good accuracy in detecting IEE and double cycling with software developed for a specific noninvasive ventilator [7] or using the equation of motion to generate a signal in real time that reflects respiratory muscle pressure output [9].
The main condition for Better Care® to detect an IEE is the presence of an abrupt increase in expiratory flow preceded by an abrupt decrease in expiratory flow. So, an abrupt increase in expiratory flow (because alveolar pressure is increased by inspiratory muscle relaxation) with a previous abrupt decrease in expiratory flow (because alveolar pressure is decreased by inspiratory muscle contraction) will be considered an IEE by the system if not followed by a ventilator breath [22]. However, an abrupt increase in expiratory flow (expiratory muscle contraction) followed, instead of preceded, by an abrupt decrease in expiratory flow (expiratory muscle relaxation) will not reach the 42% deviation threshold and will be ignored by the Better Care® system. Our approach considers the airflow curve from a dataset recorded continuously, 24 h a day under normal conditions. Better Care® is not limited to a specific device or ventilation mode. Although the ability of Better Care® to detect IEE has been validated against both human experts’ judgment and physiological measurements obtained using NAVA technology in this pilot, it still requires validation in a much larger cohort of patients.
The current study has some limitations. First, although the breaths analyzed were obtained over a period of several days, suggesting they could be representative of many clinical situations, they came from a limited number of patients. Thus, there is a risk that the study population is not fully representative of the ICU population. Clearly, studies with more patients at more centers are necessary to fully validate Better Care®. Second, instead of using recordings of esophageal pressure tracings to track variations in intrathoracic pressure originating in the diaphragm or in any other inspiratory muscle, we used EAdi, which has limitations, including diaphragm paresis, neuromuscular weakness, and technical problems related to the position of the EAdi catheter in critically ill patients [15, 23]. We sought to minimize this problem by excluding patients with neuromuscular disorders, when the EAdi signal-to-noise ratio was too low to ascertain whether a neural effort was present, and ensuring patients were stable during data acquisition. Interestingly, the Kappa statistic indicated excellent agreement between experts. This indirect evidence suggests that IEE were correctly detected by the specialists, and these findings were further confirmed after comparing Better Care® with the EAdi signal. In fact, waveforms obtained from ventilators’ digital outputs can be used to noninvasively detect IEE and alert professionals not necessarily present at the bedside. Third, the algorithm will not be able to distinguish whether or not an IEE is followed by an expiratory muscle contraction, because in both situations an increase in the expiratory flow will be preceded by a decrease in the expiratory flow. However, when the IEE algorithm built into the Better Care® system was further validated with EAdi in an independent group of patients, specificity was still very high with only a moderate decrease in sensitivity. Finally, we have studied only patients with invasive MV; therefore, further investigation is necessary before we can extrapolate our results to patients receiving noninvasive MV.
In conclusion, in this pilot, Better Care® reliably detected IEE in patients receiving MV with accuracy similar to that of expert intensivists and the EAdi signal. However, further validation on a larger group of patients is needed.
References
Thille AW, Rodriguez P, Cabello B, Lellouche F, Brochard L (2006) Patient-ventilator asynchrony during assisted mechanical ventilation. Intensive Care Med 32:1515–1522
Vignaux L, Vargas F, Roeseler J, Tassaux D, Thille AW, Kossowsky MP, Brochard L, Jolliet P (2009) Patient-ventilator asynchrony during non-invasive ventilation for acute respiratory failure: a multicenter study. Intensive Care Med 35:840–846
Chen CW, Lin WC, Hsu CH, Cheng KS, Lo CS (2008) Detecting ineffective triggering in the expiratory phase in mechanically ventilated patients based on airway flow and pressure deflection: feasibility of using a computer algorithm. Crit Care Med 36:455–461
Sassoon CS, Foster GT (2001) Patient-ventilator asynchrony. Curr Opin Crit Care 7:28–33
de Wit M, Miller KB, Green DA, Ostman HE, Gennings C, Epstein SK (2009) Ineffective triggering predicts increased duration of mechanical ventilation. Crit Care Med 37:2740–2745
Kondili E, Prinianakis G, Georgopoulos D (2003) Patient-ventilator interaction. Br J Anaesth 91:106–119
Mulqueeny Q, Ceriana P, Carlucci A, Fanfulla F, Delmastro M, Nava S (2007) Automatic detection of ineffective triggering and double triggering during mechanical ventilation. Intensive Care Med 33:2014–2018
Mulqueeny Q, Redmond SJ, Tassaux D, Vignaux L, Jolliet P, Ceriana P, Nava S, Schindhelm K, Lovell NH (2009) Automated detection of asynchrony in patient-ventilator interaction. Conf Proc IEEE Eng Med Biol Soc 2009:5324–5327
Younes M, Brochard L, Grasso S, Kun J, Mancebo J, Ranieri M, Richard JC, Younes H (2007) A method for monitoring and improving patient: ventilator interaction. Intensive Care Med 33:1337–1346
Colombo D, Cammarota G, Alemani M, Carenzo L, Barra F, Vaschetto R, Slutsky AS, Della Corte F, Navalesi P (2011) Efficacy of ventilator waveforms observation in detecting patient-ventilator asynchrony. Crit Care Med Jun 23 (Epub ahead of print)
Zin WA, Milic-Emili J (1997) Esophageal pressure measurement. In: Tobin MJ (ed) Principles and practice of intensive care monitoring. McGraw-Hill, New York, pp 545–552
Sinderby C, Navalesi P, Beck J, Skrobik Y, Comtois N, Friberg S, Gottfried SB, Lindström L (1999) Neural control of mechanical ventilation in respiratory failure. Nat Med 5:1433–1436
Lecomte F, Brander L, Jalde F, Beck J, Qui H, Elie C, Slutsky AS, Brunet F, Sinderby C (2009) Physiological response to increasing levels of neurally adjusted ventilatory assist (NAVA). Respir Physiol Neurobiol 166:117–124
Piquilloud L, Vignaux L, Bialais E, Roeseler J, Sottiaux T, Laterre PF, Jolliet P, Tassaux D (2011) Neurally adjusted ventilatory assist improves patient-ventilator interaction. Intensive Care Med 37:263–271
Barwing J, Ambold M, Linden N, Quintel M, Moerer O (2009) Evaluation of the catheter positioning for neurally adjusted ventilatory assist. Intensive Care Med 35:1809–1814
Garcia-Esquirol O, Sales B, Montanya J, Chacon E, Estruga A, Borelli M, Villagra A, Lucangelo U, Murias G, Fernandez R, Gonzalez J, Blanch L (2010) Validation of an automatic continuous system to detect expiatory asynchronies during mechanical ventilation. Intensive Care Med 36(2):S349
Suarez-Sipmann F, Pérez Márquez M, González Arenas P (2008) New modes of ventilation: NAVA. Med Intensiva 32:398–403
Fleiss JL (1981) The measurement of interrater agreement. In: Statistical methods for rates and proportions, 2nd ed. John Wiley, New York, pp 212–236
de Wit M, Pedram S, Best AM, Epstein SK (2009) Observational study of patient-ventilator asynchrony and relationship to sedation level. J Crit Care 24:74–80
Imanaka H, Nishimura M, Takeuchi M, Kimball WR, Yahagi N, Kumon K (2000) Autotriggering caused by cardiogenic oscillation during flow-triggered mechanical ventilation. Crit Care Med 28:402–407
Jubran A, Tobin MJ (1994) Use of flow-volume curves in detecting secretions in ventilator-dependent patients. Am J Respir Crit Care Med 150:766–769
Georgopoulos D, Prinianakis G, Kondili E (2006) Bedside waveforms interpretation as a tool to identify patient-ventilator asynchronies. Intensive Care Med 32:34–47
Coisel Y, Chanques G, Jung B, Constantin JM, Capdevila X, Matecki S, Grasso S, Jaber S (2010) Neurally adjusted ventilatory assist in critically ill postoperative patients: a crossover randomized study. Anesthesiology 113:925–935
Acknowledgments
This work is partially funded by: Fundació Parc Taulí, Caixa Sabadell, ISCIII PI09/91074, CIBERes, Plan Avanza TSI-020302-2008-38, MCYIN, and MITYC (Spain).
Author information
Authors and Affiliations
Corresponding author
Additional information
This article is discussed in the editorial available at: doi:10.1007/s00134-012-2497-0.
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Blanch, L., Sales, B., Montanya, J. et al. Validation of the Better Care® system to detect ineffective efforts during expiration in mechanically ventilated patients: a pilot study. Intensive Care Med 38, 772–780 (2012). https://doi.org/10.1007/s00134-012-2493-4
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00134-012-2493-4