xue-et-al-2020-automated-inline-analysis-of-myocardial-perfusion-mri-with-deep-learning
xue-et-al-2020-automated-inline-analysis-of-myocardial-perfusion-mri-with-deep-learning
Purpose: To develop a deep neural network–based computational workflow for inline myocardial perfusion analysis that automatically
delineates the myocardium, which improves the clinical workflow and offers a “one-click” solution.
Materials and Methods: In this retrospective study, consecutive adenosine stress and rest perfusion scans were acquired from three hospitals
between October 1, 2018 and February 27, 2019. The training and validation set included 1825 perfusion series from 1034 patients
(mean age, 60.6 years 6 14.2 [standard deviation]). The independent test set included 200 scans from 105 patients (mean age, 59.1
years 6 12.5). A convolutional neural network (CNN) model was trained to segment the left ventricular cavity, myocardium, and
right ventricle by processing an incoming time series of perfusion images. Model outputs were compared with manual ground truth for
accuracy of segmentation and flow measures derived on a global and per-sector basis with t test performed for statistical significance.
The trained models were integrated onto MR scanners for effective inference.
Results: The mean Dice ratio of automatic and manual segmentation was 0.93 6 0.04. The CNN performed similarly to manual
segmentation and flow measures for mean stress myocardial blood flow (MBF; 2.25 mL/min/g 6 0.59 vs 2.24 mL/min/g 6 0.59, P =
.94) and mean rest MBF (1.08 mL/min/g 6 0.23 vs 1.07 mL/min/g 6 0.23, P = .83). The per-sector MBF values showed no differ-
ence between the CNN and manual assessment (P = .92). A central processing unit–based model inference on the MR scanner took
less than 1 second for a typical perfusion scan of three slices.
Conclusion: The described CNN was capable of cardiac perfusion mapping and integrated an automated inline implementation on the
MR scanner, enabling one-click analysis and reporting in a manner comparable to manual assessment.
This copy is for personal use only. To order printed copies, contact [email protected]
Automated Inline Analysis of Myocardial Perfusion MRI
Figure 2: Schematic plot of the convolutional neural network (CNN) trained in this study. This network consists of downs-
ampling and upsampling layers. Each layer includes a number of ResNet blocks. More layers and blocks can be inserted
into the CNN to increase its depth. In the example illustration, two layers are used with two blocks for each layer. The total
number of convolution blocks is 23. BN = batch normalization, C = concatenate filter response, CONV = convolution, LV =
left ventricle, RELU = rectified linear units, RV = right ventricle.
segment the RV; however, the myocardium was properly seg- proximately 250 msec per slice. For a typical three short-axis
mented. A second rest case had one apical slice in which the acquisition, inline analysis took less than 1 second on the cen-
myocardium segmentation included blood pool. There was ap- tral processing unit. On the older Xeon E5, model loading
parent through-plane motion that was uncorrected. No other time was approximately 130 msec and applying models took
segmentation failures were found. 370 msec per slice.
Figure 3: Screenshot in a patient undergoing an adenosine stress study demonstrates the proposed inline analysis solution on an MR
scanner. Stress maps show regional flow reduction in septal and inferior sectors. The determined right ventricular insertion was used to split
myocardium to American Heart Association (AHA) sectors, with the contours overlaid to mark territories. The inline reporting further produced
a 16-sector AHA bulls-eye plot with global and per-sector flow measures reported in a table.
derived myocardial measures were computed and reported and myocardial perfusion reserve measured automatically using
inline on the MR scanner, taking just 1 additional second of artificial intelligence quantification of cardiac MR perfusion (2).
inline processing time. Quantitative evaluation in this initial This study demonstrates the relevance of automated myocardial
study demonstrated performance of myocardial segmentation segmentation in cardiac MR stress perfusion.
and sector-based analysis that was well matched to a human Automated cardiac MR image analysis has been attempted
expert. This study used stress and rest data from seven scanners over a long period of time (21). Most work focused on cine im-
at three sites at two field strengths, using more than 1800 con- age analysis (eg, MICCAI 2017 ACDC challenge [22]). The first
secutive scans for training and 200 for test. Bland-Altman anal- deep learning study, which was based on a large data cohort and
ysis demonstrated a 95% CI for global MBF of 0.05 mmol/ reported performance matching human level, was published in
min/g compared with manual labeling, which was sufficient for 2018 (23). Since then, deep neural nets were applied to other
automated detection and reporting. Prior work on segment- cardiac MR imaging applications, such as T1 mapping and car-
ing perfusion (19) used much smaller datasets that resulted in diac late enhancement segmentation (24). Our approach used
much higher variance. A weighted sum loss function was used the temporal information through the whole bolus passage
in this study and gave good accuracy. There are indeed many (2D+T) to exploit the contrast dynamics for detecting both RV
other alternatives, such as soft Dice ratio or focal loss (20), and LV which enabled finding the RV insertion point. The epi-
which can be effective in perfusion segmentation task. Which cardial fat, showing no dynamic intensity changes, was correctly
loss function is the best may vary for different applications. excluded from segmentation which would have been more dif-
A comprehensive overview and implementation of many loss ficult to avoid on a single static image.
functions can be found at https://github.com/JunMa11/SegLoss. This study had several limitations. First, the presented study
The prognostic significance of proposed artificial intelligence was conducted on MR scanners from a single vendor. Although
application was studied and recently published in Knott et al (8). the specific imaging dual sequence used may not be available on
In this study, 1049 patients with known or suspected coronary other platforms, the proposed segmentation method and CNN
artery disease underwent stress MR perfusion scans and were an- models may be still applicable. Second, although the proposed
alyzed with the proposed CNN models, showing reduced MBF algorithm works well for most cases, a few cases are challenging.
Figure 4: Adenosine stress perfusion images and myocardial blood flow (MBF) maps illustrate segmentation in the format of derived American
Heart Association sector contours overlaid on flow maps. For each case, the first row is the images in gadolinium units and the second row is the MBF
maps. Sector contours were overlaid to mark three territories for left anterior coronary artery (yellow), right coronary artery (green), and left circumflex
(red). (a) Patient with single-vessel obstructive coronary artery disease in right coronary artery territory. Papillary muscle was not included in segmenta-
tion. (b) Patient with hypertrophic cardiomyopathy illustrates that the convolutional neural network–based segmentation works with thick myocardium
and small cavity. The epicardial fat was correctly excluded.
For example, in the case of severe hypertrophy, some slices may extend segmentation to long-axis slices. In the case where the
not exhibit any blood pool (ie, complete extinction) where no basal slice may cover some portion of the outflow tract, the
endocardial contours are drawn. Third, the CNN models are proposed algorithm will avoid the blood pool and divide a seg-
currently trained for short-axis slices and cannot be applied to ment accordingly or may skip a segment entirely, resulting in
long-axis views. New training and test datasets are needed to incomplete segmentation. Fourth, in cases of severe respiratory
Figure 5: Bland-Altman plots for independent test dataset (a) global mean myocardial blood flow (MBF) and (b) per-
sector measures. No significant differences were found between convolutional neural network (CNN)–derived results and
manual measures. The dotted lines mark the 95% confidence range.
motion that is beyond the capacity of the in-plane retrospective sure any questions related to the work are appropriately resolved, all authors; litera-
ture research, H.X., M.F., P.K.; clinical studies, L.A.E.B., K.D.K., T.K., M.F., S.P.,
methods, portions of the myocardium may be blurred, where J.C.M.; experimental studies, H.X., R.H.D., P.K.; statistical analysis, H.X., P.K.;
CNN segmentation can perform poorly. However, in these cases and manuscript editing, H.X., R.H.D., L.A.E.B., K.D.K., M.F., S.P., J.C.M., P.K.
manual segmentation is difficult as well.
Another limitation of this study was the single operator for data Disclosures of Conflicts of Interest: H.X. disclosed no relevant relationships. R.H.D.
disclosed no relevant relationships. L.A.E.B. disclosed no relevant relationships. K.D.K.
labeling. The interoperator reliability was not tested in this article. disclosed no relevant relationships. T.K. disclosed no relevant relationships. M.F. dis-
Because this solution had been deployed to MR scanners, clinical closed no relevant relationships. S.P. disclosed no relevant relationships. J.C.M. Activi-
collaborators have started to use this solution (8,25), but more ties related to the present article: institution receives grants from British Heart Founda-
tion for research fellows to develop perfusion mapping as a clinical tool. Activities not
clinical validation is required to further validate this solution. related to the present article: disclosed no relevant relationships. Other relationships:
In conclusion, we demonstrated automated analysis can be disclosed no relevant relationships. P.K. disclosed no relevant relationships.
achieved on clinical scanners for perfusion MRI. Deep learning
enabled inline analysis immediately after data acquisition as part References
of imaging computation, therefore making the process more ob- 1. Nagel E, Greenwood JP, McCann GP, et al. Magnetic resonance perfusion or
jective, convenient, and faster, reducing clinical burden. fractional flow reserve in coronary disease. N Engl J Med 2019;380(25):2418–
2428.
2. Kotecha T, Martinez-Naharro A, Boldrini M, et al. Automated Pixel-Wise
Author contributions: Guarantors of integrity of entire study, H.X., M.F.; study Quantitative Myocardial Perfusion Mapping by CMR to Detect Obstruc-
concepts/study design or data acquisition or data analysis/interpretation, all authors; tive Coronary Artery Disease and Coronary Microvascular Dysfunction:
manuscript drafting or manuscript revision for important intellectual content, all Validation Against Invasive Coronary Physiology. JACC Cardiovasc Imaging
authors; approval of final version of submitted manuscript, all authors; agrees to en- 2019;12(10):1958–1969.
3. Kellman P, Hansen MS, Nielles-Vallespin S, et al. Myocardial perfusion International Conference on Machine Learning (ICML). Volume 37. Lillie,
cardiovascular magnetic resonance: optimized dual sequence and reconstruc- France: 2015;448–456.
tion for quantification. J Cardiovasc Magn Reson 2017;19(1):43. 15. Maas AL, Hannun AY, Ng AY. Rectifier nonlinearities improve neural
4. Xue H, Brown LAE, Nielles-Vallespin S, Plein S, Kellman P. Automatic network acoustic models. Proceedings of the 30th International Conference
in-line quantitative myocardial perfusion mapping: Processing algorithm on Machine Learning (ICML). Volume 28. Atlanta, Ga: 2013.
and implementation. Magn Reson Med 2020;83(2):712–730. 16. Paszke A, Gross S, Chintala S, et al. Automatic differentiation in pytorch.
5. Zorach B, Shaw PW, Bourque J, et al. Quantitative cardiovascular magnetic 31st Conf Neural Inf Process Syst, 2017; 1–8.
resonance perfusion imaging identifies reduced flow reserve in microvascular 17. Xue H, Shah S, Greiser A, et al. Motion correction for myocardial T1 map-
coronary artery disease. J Cardiovasc Magn Reson 2018;20(1):14. ping using image registration with synthetic image estimation. Magn Reson
6. Acampa W, Petretta M, Evangelista L, et al. Myocardial perfusion imaging Med 2012;67(6):1644–1655.
and risk classification for coronary heart disease in diabetic patients. The 18. Huttenlocher DP, Rucklidge WJ, Klanderman GA. Comparing images
IDIS study: a prospective, multicentre trial. Eur J Nucl Med Mol Imaging using the Hausdorff distance under translation. Proceedings 1992 IEEE
2012;39(3):387–395. Computer Society Conference on Computer Vision and Pattern Recognition,
7. Xue H, Davies R, Hansen D, et al. Gadgetron Inline AI: Effective Model Champaign, IL, June 15–18, 1992. Piscataway, NJ: IEEE, 1992; 654–656.
inference on MR scanner [abstr]. In: Proceedings of the Twenty-Seventh 19. Scannell CM, Veta M, Villa ADM, et al. Deep-Learning-Based Preprocess-
Meeting of the International Society for Magnetic Resonance in Medicine. ing for Quantitative Myocardial Perfusion MRI. J Magn Reson Imaging
Berkeley, Calif: International Society for Magnetic Resonance in Medicine, 2020;51(6):1689–1696.
2019; 2019; 4837. 20. Lin TY, Goyal P, Girshick R, He K, Dollar P. Focal Loss for Dense Object
8. Knott KD, Seraphim A, Augusto JB, et al. The Prognostic Significance of Detection. IEEE Trans Pattern Anal Mach Intell 2020;42(2):318–327.
Quantitative Myocardial Perfusion: An Artificial Intelligence-Based Approach 21. Peng P, Lekadir K, Gooya A, Shao L, Petersen SE, Frangi AF. A review of
Using Perfusion Mapping. Circulation 2020;141(16):1282–1291. heart chamber segmentation for structural and functional analysis using
9. Xue H, Tseng E, Knott KD, et al. Automated detection of left ventricle cardiac magnetic resonance imaging. MAGMA 2016;29(2):155–195.
in arterial input function images for inline perfusion mapping using deep 22. Bernard O, Lalande A, Zotti C, et al. Deep Learning Techniques for Auto-
learning: A study of 15,000 patients. Magn Reson Med 2020. 10.1002/ matic MRI Cardiac Multi-Structures Segmentation and Diagnosis: Is the
mrm.28291. Published online May 7, 2020. Problem Solved? IEEE Trans Med Imaging 2018;37(11):2514–2525.
10. Bluemke DA, Moy L, Bredella MA, et al. Assessing radiology research on 23. Bai W, Sinclair M, Tarroni G, et al. Automated cardiovascular magnetic
artificial intelligence: A brief guide for authors, reviewers, and readers-from resonance image analysis with fully convolutional networks. J Cardiovasc
the Radiology Editorial Board. Radiology 2020;294(3):487–489. Magn Reson 2018;20(1):65.
11. Ronneberger O, Fischer P, Brox T. U-net: Convolutional networks for bio- 24. Moccia S, Banali R, Martini C, et al. Development and testing of a deep
medical image segmentation. Lect Notes Comput Sci 2015;9351:234–241. learning-based strategy for scar segmentation on CMR-LGE images. MAGMA
12. Zhang Z, Liu Q, Wang Y. Road Extraction by Deep Residual U-Net. IEEE 2019;32(2):187–195.
Geosci Remote Sens Lett 2018;15(5):749–753. 25. Camaioni C, Knott KD, Augusto JB, et al. Inline perfusion mapping provides
13. Xie S, Girshick R, Dollár P, Tu Z, He K. Aggregated residual transformations insights into the disease mechanism in hypertrophic cardiomyopathy. Heart
for deep neural networks. 2017 IEEE Conference on Computer Vision and 2020;106(11):824–829.
Pattern Recognition (CVPR), Honolulu, HI, July 21–26, 2017. Piscataway,
NJ: IEEE, 2017; 5987–5995.
14. Ioffe S, Szegedy C. Batch Normalization: Accelerating Deep Network
Training by Reducing Internal Covariate Shift. Proceedings of the 32nd