Pathomics, Clinical Studies, and Cancer Surveillance
0 likes77 views
The document discusses advancements in pathomics and computational pathology, focusing on the integration of AI in cancer diagnosis and treatment. Key topics include the characterization of tumor-infiltrating lymphocytes (TILs), cell detection and classification methods, and collaborations for real-world clinical research. It outlines the use of deep learning algorithms for large-scale analysis and emphasizes the importance of accurate data and validation processes in clinical applications.
Pathomics, Clinical Studies, and Cancer Surveillance
- 1. Pathomics, Clinical Studies, and Cancer Surveillance Joel Saltz MD, PhD Chair and Distinguished Professor Department of Biomedical Informatics Professor Department of Pathology Cherith Endowed Chair Stony Brook University HIMA Imaging Science Pathology Informatics Summit 2022
- 2. • No Conflicts to Disclose
- 3. Roadmap • Pathomics and Clinical Research • Cell Detection, Classification • Scalability • FDA • Modeling human attention
- 4. Ingredients of Computational Pathology Biomarkers • Every diagnostic WSI contains vast amounts of data – Tumor, lymphocyte infiltration – Cells – normal, dysplastic, lymphocytes, necrotic – Spatial distribution and morphology of glands, ducts – Collagen orientation Predict treatment response and outcome Patient stratification Treatment selection Cancer Epidemiology
- 5. Computational Pathology • Just as in flow cytometry cells are interrogated counted clustered and classified, using the newfound ability to segment and classify cells enables a spatially and formed type of highly granular highly quantitative characterization of tissue. • In this case the challenge is not to reproduce what a human pathologist chooses as a classification but instead to use a highly detailed set of labeled building blocks to build new biomarkers
- 6. Potential Clinical Applications of AI based TILs Methods TILs to predict prognosis (treatment response and survival) Spatial patterns of distribution of TILs maps to ascertain the functional immune status of the tumor microenvironment Combine diagnostic criteria and TILs to stratify patients, guide clinical management, and select therapy (e.g. immunotherapy)
- 7. Collaboration with ECOG-ACRIN to bring AI to Clinical Trials
- 8. Participating SEER Registries: New Jersey, Kentucky, Georgia, New York Collaboration with SEER Registries to Bring AI Pathology to Surveillance and to Create Real World Clinical Research Datasets
- 10. ● Deep learning based computational stain for TILS ●TIL patterns generated from 4,759 TCGA subjects (5,202 H&E slides), 13 cancer types ●Detected TILs correlate with pathologist eye and molecular estimates ●TIL patterns linked to tumor and immune molecular features, cancer type, and outcome ●Now refined and extended to roughly 20 cancer types
- 12. Pathomics tissue analytics for Precision Medicine (1) Tumor-TILs analyses (2) High-resolution detection and classification of tumor cells, lymphocytes, and stromal cells in the entirety of whole slide images (3) Support the scoring of the number of TILs in microscopic regions of interest chosen by pathologists (4) Analysis of cell composition and features in different tumor niches Legend red = lymphocytes blue = stromal cells green = tumor cells
- 13. TIL Pattern Descriptions Quantitative – Arvind Rao • Agglomerative clustering • Cluster indices representing cluster number, density, cluster size, distance between clusters • Traditional spatial statistics measures • R package clusterCrit by Bernard Desgraupes - Ball-Hall, Banfield- Raftery, C Index, and Determinant Ratio indices Qualitative (Alex Lazar, Raj Gupta) ‘‘Brisk, diffuse’’ diffusely infiltrative TILs scattered throughout at least 30% of the area of the tumor (1,856 cases); ‘‘Brisk, band-like’’ - band- like boundaries bordering the tumor at its periphery (1,185); ‘‘Nonbrisk, multi-focal’’ loosely scattered TILs present in less than 30% but more than 5% of the area of the tumor (1,083); ‘‘Non-brisk, focal’’ for TILs scattered throughout less than 5% but greater than 1% of the area of the tumor (874); ‘‘None’’ < 1% TILS - in 143 cases
- 14. TIL Pattern Characterization Cutaneous Malignant Melanoma,TIL map with “Brisk, Band-like” structural pattern Cutaneous Malignant Melanoma,TIL map with “Brisk, Band- like” structural pattern ” Squamous Cell Carcinoma of the Lung, TIL map with “Brisk, Diffuse” structural pattern
- 17. Breast Cancer TILS: TCGA and Carolina Breast Cancer Study
- 19. Carolina Breast Study – TIS and Subtype
- 20. Carolina Breast Cancer Study – “Risky Features” Recurrence in patients with 2 or more high risk features Ø Low intra-tumoral strength Ø TIL deserts Ø Absence of TIL forests Ø Low peritumoral strength Ø Lbsence of lymphoid aggregates Similar results for TCGA
- 21. Roadmap • Pathomics and Clinical Research • Cell Detection, Classification • Scalability • FDA • Modeling human attention
- 22. Spatial Contexts -- Cell Detection and Classification • Classification accuracy is frequently context sensitive • Training on new tissue types and new cell categories is time consuming • Method detects and classifies nuclei • Training requires “dotting” nuclei
- 23. Spatial Contexts -- Nuclear Detection and Classification International Conference on Computer Vision Github - https://github.com/Top oXLab/MCSpatNet
- 24. Pipeline encompasses cell detection, cell classifier and learning category specific spatial statistics
- 25. Spatial Statistical Learning -- Cells of a feather flock together
- 26. Roadmap • Pathomics and Clinical Research • Cell Detection, Classification • Scalability • FDA • Modeling human attention
- 27. A Framework for Automating I/O of Deep Learning Methods in Biomedical Imaging Applications Ana Gainaru, Scott Klasky, Dmitry Ganushin, Tahsin Kurc, Joel Saltz
- 28. Goals • Framework for automating AI workflows by separating the data and computational planes and offloading the I/O management to specialized processes • AI applications often involve complex workflows involving large datasets, multiple deep learning networks and multiple sets of hyper- parameters • Specific focus on applications that involve analysis of very large imagery or simulation output datasets • Immediate target is analysis of large collections of gigapixel Pathology whole slide images
- 29. Community Challenge – Breast Cancer Tumor/TIL AI Methods
- 30. Ensemble of 9 models Input Size UNet Encoder 512 x 512 pixels 640 x 640 768 x 768 EfficientNet-B2 ✓ ✓ ✓ ResNeXt-50-32x4d ✓ ✓ ✓ SEResNeXt-50-32x4d ✓ ✓ ✓ Jakub Kaczmarzyk
- 31. Sweep over hyperparameters (each line is one model)
- 32. Deep Learning Workflow Framework ●The framework is separated from AI applications ●Coordinates data reads and pre-processing associated with concurrently running applications ●The framework reuses the data loaded from storage for all the applications that require it and keeps track of the models generated ●I/O framework will be able to manage data transport and storage in a more efficient manner ●Layered on Oak Ridge National Laboratory ADIOS-2 framework Two algorithm ensemble - TIL analysis of 28,379 TCGA WSIs in 1 hour and 20 minutes on 6000 GPUs (500 nodes) on ORNL Summit
- 33. Roadmap • Pathomics and Clinical Research • Nuclear Detection, Classification • Scalability • FDA • Modeling human attention
- 34. AI Pathology Algorithm Validation FDA Led Collaborations Dataset of pathologist annotations for algorithms that process whole slide images • Researchers from FDA, academic colleagues in collaboration with the Alliance for Digital Pathology, are collecting pathologist annotations as data for AI/ML algorithm validation • Application - tumor infiltrating lymphocyte (TIL) detection and quantitation.. • Training materials and workflows to crowdsource pathologist image annotations on two modes: an optical microscope and two digital platforms. • The microscope platform allows the same ROIs to be evaluated in both modes. • Pursue an FDA Medical Device Development Tool Qualification
- 35. Validating Artificial Intelligence Methods for Clinical Use A Pathologist-Annotated Dataset for Validating Artificial Intelligence: A Description and Pilot Study
- 36. Roadmap • Pathomics and Clinical Research • Cell Detection, Classification • Scalability • FDA • Modeling human attention
- 37. Visual Attention Analysis of Pathologists Examining Whole Slide Images of Cancer • Multidisciplinary collaboration: Biomedical Informatics, Computer Science, Psychology, Pathology • Souradeep Chakraborty, Ke Ma, Rajarsi Gupta, Beatrice Knudsen, Gregory J. Zelinsky, Joel Saltz, Dimitris Samaras • Stony Brook, University of Utah ➢ Understand and model relationship between human attention and Pathologist classification ➢ Model the spatiotemporal distribution of human visual attention ➢ Employ attention models to improve precision of AI Pathology tasks
- 38. GOAL: Study pathologists’ attention as they examine whole slide images of prostate cancer tissue H&E image Tumor segmentation Visual attention heatmap
- 39. Figure: Processing of the captured attention data to obtain visual attention heatmaps and scanpaths Data processing
- 40. Figure: Processing of the captured attention data to obtain visual attention heatmaps and scanpaths Data processing
- 41. Figure: Processing of the captured attention data to obtain visual attention heatmaps and scanpaths Data processing
- 42. Proposed model for attention prediction Figure: Our model, ProstAttNet (Prostate Attention Net) for predicting visual attention on WSI patches ➢ We formulate attention prediction as a classification task ○ Goal: classify a WSI patch into one of the N attention intensity bins (N = 5 in our study). Ø We construct the final attention heatmap by assembling the predicted patch-wise heatmaps followed by gaussian smoothing and map normalization
- 43. Results: Qualitative evaluation Figure: Comparison of the predicted attention heatmap using our ProstAttNet model with the ground truth segmentation map and the ground truth attention heatmap Ø Test dataset à 17 whole slide images (from the TCGA-PRAD dataset) Ø The predicted attention heatmap correlates well with the ground truth attention heatmap and the ground truth tumor segmentations
- 44. Thanks! • Pathomics and Clinical Research • Cell Detection, Classification • Scalability • FDA • Modeling human attention
- 45. Stony Brook Deep Learning Pathology Faculty Dimitis Samaras, Chao Chen, Tahsin Kurc, Prateek Prasanna, Raj Gupta
- 46. Vu Nguyen– Graduate Student Computer Science Anne Zhao – Assistant Professor, Dept Pathology Stony Brook Raj Gupta – Department of BMI Stony Brook Deep Learning and Tissue Characterization Trainee Team circa 2018 Le Hou– software engineer, google core, Mountain View Han Le– Software Development Engineer II, Amazon, Seattle
- 47. The Stony Brook Multi-Scale Student Group: David Belinsky, Mahmudul Hasan Prantik Howlader
- 48. SEER UG3 Team • Stony Brook – Joel Saltz MD, PhD – Tahsin Kurc PhD – Dimitris Samara PhD – Erich Bremer – Le Hou – Shahira Abousamra – Raj Gupta – Han Le – Bridge Wang • Emory – Ashish Sharma PhD – Ryan Birmingham – Nan Li • Georgia SEER Registry – Kevin Ward MPH, PhD • Rutgers – David J. Foran PhD – Evita Sadimin, MD – Wenjin Chen, PhD – Doreen Loh M.S. – Jian Ren, Ph.D – Christine Minerowicz, MD • Rutgers SEER/ NJ State Cancer Registry – Antoinette Stroup, PhD – Adrian Botchway, CTR – Gerald Harris, PhD • University Kentucky; Kentucky SEER Registry – Eric B. Durbin, DrPH, MS – Isaac Hands, MPH – John Williams, MA – Justin Levens
- 49. QuIP ITCR Team Stony Brook University Joel Saltz Tahsin Kurc Raj Gupta Dimitris Samaras Erich Bremer Fusheng Wang Tammy DiPrima Le Hou NCI / DCEG Jonas S Almeida Emory University Ashish Sharma Ryan Birmingham Nan Li University of Tennessee Knoxville Jeremy Logan Scott Klasky Harvard University Rick Cummings ITCR/IMAT Supplement Richard Levinson Raj Gupta Tahsin Kurc Joel Saltz Han Le Maozheng Zhao Dimitris Samaras
- 50. 50 The PRISM Team • Fred Prior, PhD • Jonathan Bona, PhD • Kirk Smith • Lawrence Tarbox, PhD • Mathias Brochhausen, PhD • Roosevelt Dobbins • Tracy Nolan • William Bennett • Ashish Sharma, PhD • Annie Gu • Mohanapriya Narapareddy • Monjoy Saha, PhD • Pradeeban Kathiravelu, PhD • Joel Saltz, MD, PhD • Erich Bremer • Rajrisi Gupta MD • Tahsin Kurc, PhD • Tammy DiPrima • TJ Fitzgerald, MD • Fran Laurie
- 51. Thanks - Funding • U24CA180924, U24CA215109, NCIP/Leidos 14X138 and HHSN261200800001E, UG3CA225021-01 from the NCI; R01LM011119-01 and R01LM009239 from the NLM, Award 2123920 from the National Science Foundation Bob Beals and Betsy Barton, Stony Brook Mount Sinai Seed Funding, Pancreatic Cancer Action Network • This research used resources provided by the National Science Foundation XSEDE Science Gateways program and the Pittsburgh Supercomputer Center BRIDGES-AI facility