Embryo selection using AI

Feb 12, 2019Download as PPTX, PDF0 likes251 views

In vitro fertilization often results in multiple pregnancy, because doctors implant few embryos in mother to make higher chance of succeeding. Mothers carrying twins or triplets have an increased incidence of preeclampsia, maternal haemorrhage, operative delivery, uterine rupture, and preterm labor. Idea is to find single embryo that has the highest probability to implant and results in a live born baby. Data is presented as time lapse video of developing embryo (first 5 days). In this talk we would deep dive into several neural network architectures that are used in the project. Starting with U-net for embryo segmentation and extraction from raw videos, then unsupervised convolutional neural networks and recurrent neural networks for video processing. Also, we explore possibility of using external data source (patient and embryo records) alongside neural network in order to train end to end deep learning solution.

Segmentation and
Classification of Embryo
Videos using Deep
Learning

Stanko Kuveljić
Linkedin: https://www.linkedin.com/in/stanko-kuveljic/
Facebook: https://www.facebook.com/stanko.kuveljic
Email: stankokuveljic@gmail.com
SmartCat mail: stanko@smartcat.io
BattleNet: stankokuveljic@gmail.com

Quest Chain
● Problem and Motivation
● Deep Learning
● Full end-to-end Deep Learning Architecture
● Image Segmentation
○ Side Quest: U-Net
● Image Embeddings
○ Side Quests: Autoencoders
● Video Classification
○ Side Quest: LSTM
● Level Up: Results
● Summary and Future Work

Problem And Motivation
● In Vitro Fertilisation
● Multiple Embryo Transfer
○ Multiple Children
○ Health Issues
● Solution: Single Embryo Transfer
● The best Embryo
● Could AI help doctors to choose embryo?

Full End2End Architecture
● Deep Learning end2end solution
● Combines Video and Metadata information
● Steps:
1. Transforms Video into Frames
2. Detect and Crop Embryo in each frame
3. Deep Feature Extraction
4. Recurrent Neural Network
5. Combines RNN state and Metadata
6. Final Fully Connected layer → Score
● Output:
1. Liveborn
2. Miscarriage
3. No Implant

Embryo Crop and Preprocessing
● Embryo Segmentation/Crop
● Center + Resize
● Normalization

OpenCV
● Brightness Correction
● Adaptive Threshold
● Eroding/Dilating
● ROI selection

U-Net Training
● Input:
○ Random Crops (256x256)
○ Data Augmentation (Rotate, Translate, Flip, Brightness)
● Output:
○ Embryo Mask → Cropped Embryo
● Network:
○ Convolution/Deconvolution Filter size: 3x3
○ Activation: ReLU
○ Batch Normalization
○ Dropout
○ L2

Image Embeddings
● Unsupervised Feature Extraction
● Input: Image (Frame)
● Output: Feature Vector
● Autoencoders:
○ Variational Autoencoder
○ Stacked Denoising Autoencoder
● Generative Adversarial Networks

Side Quest: Stacked Denoising Autoencoder

Image Embeddings
Original
Original
SDAE
VAE
VAE needs more
training

Video Classification
● Input: Encoded Sequence of Frames
● Outputs:
○ State Vector
○ Probability - [Liveborn, Miscarriage, No implant]
○ Probability - [Implant, No Implant]
● Variable sequence length
○ Downsampling
○ Last N Frames

Long Short Term Memory (RNN)
● Layers ~ 2-3
● Units ~ 512-1024
● Dropout ~ 0.6
● Augmented
● TS: 64 - 512
● Fine Tuning

Level Up: Results
● Train/Test split: 90% / 10%
● SDAE + 3xLSTM(512) + Softmax
● Training:
○ SDAE - 2 days on GPU
○ LSTM - 12h on GPU
○ SDAE + LSTM + Fine Tuning - 5 days on GPU
● Liveborn vs Miscarriage vs No Implant
○ Accuracy: 0.68
● Implant vs No Implant
○ Accuracy: 0.8
● Shortcomings:
○ Not enough data
○ Need more diverse embryos (most of them are good looking)

Future Works
● Collect more data (embryo videos)
● Different LSTM approaches
● 3D CNN
● Different Feature Extraction Techniques
BONUS:
● Merge data sources into single model:
○ Patient data
○ Embryo data
○ Embryo time lapse video

Quest: Get [H|F]ired at
Do you wanna build an AI?
Quest Objective:
● Report to our Warchief
We are hiring:
● DATA SCIENTIST
● DATA ENGINEERS

The document describes KKBOX's efforts to develop a smarter monkey testing tool called APE using machine learning. APE uses the TensorFlow Object Detection API to train a model on screenshots labeled with UI elements. It then deploys the trained model on Android and iOS to automatically detect and interact with elements during testing. Some key findings included that training with GPU is faster, active learning helps labeling, F1 scores measure model performance, minimizing labels improves learning, and grayscale training works well. The vision is that APE can help with localization testing by detecting languages and aid design patterns like page objects through element detection. In summary, the document discusses how KKBOX is leveraging machine learning for more intelligent automated testing.

Introduction to Kinect - Update v 1.8Matteo Valoriani

Resolution Independent 2D Cartoon Video ConversionFaathima Fayaza

This document outlines a project to develop a resolution-independent 2D cartoon video conversion codec. The project aims to convert raster image frames in videos to vector image frames to overcome resolution dependency issues. Key aspects of the project include segmenting scenes, extracting and vectorizing backgrounds, extracting and tracking foreground objects, and reproducing the animation using the vectorized components. The project is implemented using OpenCV, C++, and vectorization tools like Potrace. Experiments demonstrate techniques for scene detection, background subtraction, object tracking, and animation reproduction. Further work is planned to automate color selection and optimize vectorization parameters.

3d Video Conferencing: Opportunities and PitfallsTrueConf

3d Video Conferencing: Opportunities and PitfallsTrueConf__

deep fake detection deep fake detection asadman190214

Emily Denton - Unsupervised Learning of Disentangled Representations from Vid...Luba Elliott

Deep Neural Networks for Multimodal LearningMarc Bolaños Solà

The document discusses deep neural networks for multimodal learning tasks. It presents models for video description, visual question answering (VQA), image description, and multimodal translation that use convolutional neural networks (CNNs), long short-term memory networks (LSTMs), and attention mechanisms. For video description, a bidirectional LSTM with attention is shown to effectively generate captions. For VQA, the VIBIKNet model uses a kernelized CNN to obtain image features and achieves state-of-the-art results. Future directions include using highway networks, compact bilinear pooling, and class activation maps to further improve performance across multimodal tasks.

Learning visual representation without human labelKai-Wen Zhao

Custom-Made Games with Machine Learning and Big DataAfrica Perianez

Abstract Video games define our era. Players generate extremely rich behavioral datasets that constitute an ideal playground to understand human behavior. With this unique source of information, machine learning can not only explore social and consumer dynamics, but also predict users' personalities. Yokozuna Data, a Keywords Studio headquartered in Tokyo, is leading the data science revolution in the video game industry, developing next-generation machine learning software to operationally predict the behavior of individual players. Models based on convolutional neural networks and conditional inference survival ensembles can reveal information such as the moment and game level at which a certain player will leave the game, how much money they will spend until then, when they will make their next battle or which virtual item they are likely to select. This allows game developers to take preventive actions aimed at maximizing player engagement and to create an optimal, personalized game experience. I will review how these techniques can be used to create customized game events and provide individual recommendations on in-game items and actions. These methods are versatile in that they can adapt to very different types of games and players. By using cloud computing capabilities and big data infrastructure as provided by Amazon Web Services, they can scale to be used with games of any size, as they are also able to deal with really large datasets (petabytes of data). It is the flexibility and scalability of these techniques what makes them particularly well suited for operational environments.

American sign language recognizerGarrett Broughton, Architect/Engineer

MIT's experience on OpenPOWER/POWER 9 platformGanesan Narayanasamy

The document discusses using temporal shift modules (TSM) for efficient video recognition, where TSM enables temporal modeling in 2D CNNs with no additional computation cost; TSM models achieve better performance than 3D CNNs and previous methods while using less computation, and can be used for applications like online video understanding, low-latency deployment on edge devices, and large-scale distributed training on supercomputers.

Scalawox deeplearningscalawox

This document introduces ScalaWox's deep learning technologies including GPU accelerated neural networks and video decoders. It describes ScalaWox's ScalaNET product which uses deep learning frameworks like Caffe and TensorFlow with GPU acceleration for real-time multi-channel object detection from video. ScalaWox also demonstrates ScalaNET's abilities like object classification with aerial images, scene recognition, and behavior recognition using example videos.

INTELLIGENT HELMET DETECTION USING OPENCV AND MACHINE LEARNINGIRJET Journal

This document describes a system for intelligent helmet detection using OpenCV and machine learning. The system uses a camera to capture video of a person's face in real-time. Each video frame is preprocessed using OpenCV and fed to a machine learning model trained on the YOLO algorithm to detect whether a helmet is present. If a helmet is detected, an Arduino board connected to the system will not activate a buzzer or turn on an LED, otherwise it will. The goal is to help enforce helmet usage and reduce fatal injuries from motorcycle accidents. Key components include the camera, Arduino, buzzer, LED, TensorFlow for the ML model, OpenCV for preprocessing, and Darkflow which implements YOLO in Python using TensorFlow

Deep Learning Architectures for Video - Xavier Giro - UPC Barcelona 2019Universitat Politècnica de Catalunya

Andrii Boichuk: Video-based action recognition - past, present and futureLviv Startup Club

Vdsg /Craftworks Industrial-AIVienna Data Science Group

The document discusses an AI company called craftworks that develops industrial AI solutions. It focuses on three main applications: predictive maintenance, visual inspection, and predictive quality. For predictive maintenance, craftworks uses sensor data and deep learning models to predict faults in machines like those in Vienna's district heating system weeks in advance. For visual inspection, it develops systems that can automatically detect and classify defects in industrial parts using computer vision. And for predictive quality, it predicts product quality at various stages of a manufacturing process through sensor data, images, and process metadata.

Finding the best solution for Image ProcessingTech Triveni

What is beyond using Tensorflow, GPU or TPU to process images seamlessly? Do we have a silver bullet for image processing? Over the years, image processing has picked up a different level of attraction. Everyone can think about its ease of usability because it has become a reality now. We have started seeing how Residual Neural Network architecture is being used for different cases and not only that, how Residual Neural network is being tweaked to solve different problems. Along with tweaking the ResNet, preprocessing is also being improved to support different architecture for this matter. Everyone has almost become cyborg already with mobile phones in our hands and apparently until human beings bring the AI/ML to the phones completely they are not taking any rest. We are going to see the development of different architecture and algorithms around running AI/ML on low configuration devices. In this session, we are going to talk about different research papers submitted for these matters and some implementations for the same as well.

Machine Learning approaches at video compression Roberto Iacoviello

Machine learning approaches are being explored for video compression. Conservative approaches replace individual MPEG blocks with deep learning blocks, while disruptive end-to-end approaches aim to replace the entire MPEG chain. Optical flow networks can exploit temporal redundancy by estimating motion between frames. Fully neural network-based video compression models jointly learn motion estimation, motion compression, and residual compression in an end-to-end optimized framework. However, performance gains must be balanced against increased complexity, and neural network approaches are not yet mature enough to be included in video compression standards.

DALL-E.pdfdsfajkh

DALL-E is a large AI model that can generate images from text descriptions. It was trained on a dataset of text-image pairs using a two-stage process: 1) A discrete variational autoencoder (dVAE) learned a visual codebook to represent images as discrete latent codes, and 2) A Transformer model learned the joint distribution between text captions and latent image codes to generate new images. The model achieved impressive zero-shot image generation capabilities, generalizing to new concepts and combining ideas in novel ways, as demonstrated through both quantitative and qualitative evaluation.

Salt Identification Challengekenluck2001

Kenneth presented a summary of his participation in the Kaggle TGS Salt Identification Challenge. He discussed the top solutions which utilized techniques like ResNeXt and ResNet backbones, feature pyramid attention, Lovasz loss, and test time augmentation. Kenneth's own solution ranked 1482nd, using a U-Net architecture with Lovasz loss and test time flipping. He analyzed techniques that did and did not work well for this salt segmentation task from radar images.

Video Captioning at TRECVID 2022George Awad

VIBE: Video Inference for Human Body Pose and Shape EstimationArithmer Inc.

The document describes the VIBE approach for 3D human pose and shape estimation from video. VIBE uses an adversarial learning framework with a temporal encoder network that incorporates self-attention. It regresses pose and shape parameters from video frames. A motion discriminator is trained to distinguish real from generated poses, enforcing kinematically plausible poses without 3D ground truth labels. Results show VIBE generates accurate 3D poses and shapes from in-the-wild videos.

Skin Lesion Detection from Dermoscopic Images using Convolutional Neural Netw...Universitat Politècnica de Catalunya

This document presents research on using convolutional neural networks (CNNs) to detect skin lesions from dermoscopic images. The researchers: 1. Developed a CNN (U-Net) to segment skin lesions from images, achieving a Dice coefficient of 0.8689. 2. Used a fine-tuned VGG-16 network to classify images as benign or malignant. They found that using their automatic segmentations as input improved sensitivity over using unaltered images. 3. Concluded that their deep learning approach can help dermatologists diagnose skin cancer, and that automatic segmentation improves classification sensitivity compared to using whole images, even without perfect segmentation. This verifies their hypothesis that segmentation enhances classification.

VacPresentationLindokuhle Biyase

This document summarizes Lindokuhle Biyase's vacation work report from December 2014 to January 2015. It discusses the projects worked on, including alignment of audio with text, corpus packaging with XML, and addition of metadata and gender identification to an annotated data set. Gender identification of audio clips was evaluated using Weka data mining tools, with the J48 algorithm performing best at 70.6% accuracy. Overall, it was a good learning experience in areas like Python programming, Linux, audio editing, and natural language processing tools.

Autom editor video blooper recognition and localization for automatic monolo...Carlos Toxtli

The Bleeding EdgejClarity

This document discusses the technology stack choices made by jClarity to build their performance monitoring solution. They chose AngularJS for the client, MongoDB for the database, Vert.x for the server framework, and Chef for configuration management. While these "bleeding edge" technologies provided advantages in certain areas, they also had immaturities and lack of tooling support. jClarity evaluated options using a weighted scoring system and prototyped the top choices. Their methodology focuses on an a la carte open source approach with commercial funding and strong community involvement rather than vendor lock-in.

Navigation in 3 d environment with reinforcement learning by Predrag Njegovan...SmartCat

Reinforcement learning in today's order of things when artificial intelligence is on the rise is a favorable field for new research. One of the problems that were tried to be solved in the last year or two is the problem of environmental control or navigation. This talk is going to present one form of solution to the problem of navigation and generalization in a three-dimensional environment while there are restrictions of rewards, by forming an autonomous agent with deep learning techniques.

Resilient event deduplication in Kafka by Vladimir VajdaSmartCat

Implementing even the simplest algorithms in a distributed system as highly available can be really challenging. There are a lot of scenarios where something can go wrong and produce incorrect results. This talk is about what it takes to have a resilient solution for event deduplication. I'll try to guide you through my thought process and try to present all the scenarios in which something can go wrong.

More Related Content

Similar to Embryo selection using AI (20)

Deep Neural Networks for Multimodal LearningMarc Bolaños Solà

Learning visual representation without human labelKai-Wen Zhao

Custom-Made Games with Machine Learning and Big DataAfrica Perianez

American sign language recognizerGarrett Broughton, Architect/Engineer

MIT's experience on OpenPOWER/POWER 9 platformGanesan Narayanasamy

Scalawox deeplearningscalawox

INTELLIGENT HELMET DETECTION USING OPENCV AND MACHINE LEARNINGIRJET Journal

Deep Learning Architectures for Video - Xavier Giro - UPC Barcelona 2019Universitat Politècnica de Catalunya

Andrii Boichuk: Video-based action recognition - past, present and futureLviv Startup Club

Vdsg /Craftworks Industrial-AIVienna Data Science Group

Finding the best solution for Image ProcessingTech Triveni

Machine Learning approaches at video compression Roberto Iacoviello

DALL-E.pdfdsfajkh

Salt Identification Challengekenluck2001

Video Captioning at TRECVID 2022George Awad

VIBE: Video Inference for Human Body Pose and Shape EstimationArithmer Inc.

Skin Lesion Detection from Dermoscopic Images using Convolutional Neural Netw...Universitat Politècnica de Catalunya

VacPresentationLindokuhle Biyase

Autom editor video blooper recognition and localization for automatic monolo...Carlos Toxtli

The Bleeding EdgejClarity

Deep Neural Networks for Multimodal LearningMarc Bolaños Solà

Learning visual representation without human labelKai-Wen Zhao

Custom-Made Games with Machine Learning and Big DataAfrica Perianez

American sign language recognizerGarrett Broughton, Architect/Engineer

MIT's experience on OpenPOWER/POWER 9 platformGanesan Narayanasamy

Scalawox deeplearningscalawox

INTELLIGENT HELMET DETECTION USING OPENCV AND MACHINE LEARNINGIRJET Journal

Deep Learning Architectures for Video - Xavier Giro - UPC Barcelona 2019Universitat Politècnica de Catalunya

Andrii Boichuk: Video-based action recognition - past, present and futureLviv Startup Club

Vdsg /Craftworks Industrial-AIVienna Data Science Group

Finding the best solution for Image ProcessingTech Triveni

Machine Learning approaches at video compression Roberto Iacoviello

DALL-E.pdfdsfajkh

Salt Identification Challengekenluck2001

Video Captioning at TRECVID 2022George Awad

VIBE: Video Inference for Human Body Pose and Shape EstimationArithmer Inc.

Skin Lesion Detection from Dermoscopic Images using Convolutional Neural Netw...Universitat Politècnica de Catalunya

VacPresentationLindokuhle Biyase

Autom editor video blooper recognition and localization for automatic monolo...Carlos Toxtli

The Bleeding EdgejClarity

More from SmartCat (7)

Navigation in 3 d environment with reinforcement learning by Predrag Njegovan...SmartCat

Resilient event deduplication in Kafka by Vladimir VajdaSmartCat

Journey through the ML model deployment to production @DSC5SmartCat

1. The document discusses various approaches for deploying machine learning models to production including deploying models on edge devices, performing batch inference in a database, using REST APIs with Flask and uWSGI, using TensorFlow Serving, and streaming with message queues. 2. It provides examples of deploying image classification and text models using different approaches and measuring their performance under load testing. TensorFlow Serving was able to achieve the highest throughput for GPU-based models while maintaining low response times. 3. The conclusion emphasizes that the best deployment approach depends on factors like business needs, accuracy requirements, latency constraints, data and model sizes, and whether the use case is for demonstration or production.

Journey through the ML model deployment to production by Stanko KuveljicSmartCat

In this talk, we are going straight into mud dirt and flames. It is not about how to train the ML model, nor how to win a Kaggle competition. It is not about ponies either. Instead, we are going to take a journey through the depths of deployment hell. We will talk about how to bring the ML model to life by exploring several architectures. The journey will start with a simple Flask application and it will finish with a scalable solution using TensorFlow serving. Brace yourselves for load tests and performance benchmarks.

Ai pitfalls through so you don't have toSmartCat

The goal of this presentation is addressing the most common pitfalls happening during AI projects. If your company is starting with AI or planning to hire an external consulting company such as ours, this information is a valuable resource which helps to evaluate your level of readiness for this kind of project. Additionally, this document serves as a guideline to better prepare for our successful cooperation, in case we address most of the issues on this list.

HVAC optimisation using RLSmartCat

Elasticsearch - under the hoodSmartCat

Elasticsearch is quite common tool nowadays. Usually as a part of ELK stack, but in some cases to support main feature of the system as search engine. Documentation on regular use cases and on usage in general is pretty good, but how it really works, how it behaves beneath the surface of the API? This talk is about that, we will look under the hood of Elasticsearch and dive deep in the largely unknown implementation details. Talk covers cluster behaviour, communication with Lucene and Lucene internals to literally bits and pieces. Come and see Elasticsearch dissected.

Navigation in 3 d environment with reinforcement learning by Predrag Njegovan...SmartCat

Resilient event deduplication in Kafka by Vladimir VajdaSmartCat

Journey through the ML model deployment to production @DSC5SmartCat

Journey through the ML model deployment to production by Stanko KuveljicSmartCat

Ai pitfalls through so you don't have toSmartCat

HVAC optimisation using RLSmartCat

Elasticsearch - under the hoodSmartCat

Recently uploaded (20)

Secure_File_Storage_Hybrid_Cryptography.pptx..yuvarajreddy2002

FPET_Implementation_2_MA to 360 Engage Direct.pptxssuser4ef83d

1. Briefing Session_SEED with Hon. Governor Assam - 27.10.pdfSimran112433

AI Competitor Analysis: How to Monitor and Outperform Your CompetitorsContify

GenAI for Quant Analytics: survey-analytics.aiInspirient

183409-christina-rossetti.pdfdsfsdasggsagfardin123rahman07

Classification_in_Machinee_Learning.pptxwencyjorda88

Safety Innovation in Mt. Vernon A Westchester County Model for New Rochelle a...James Francis Paradigm Asset Management

By James Francis, CEO of Paradigm Asset Management In the landscape of urban safety innovation, Mt. Vernon is emerging as a compelling case study for neighboring Westchester County cities. The municipality’s recently launched Public Safety Camera Program not only represents a significant advancement in community protection but also offers valuable insights for New Rochelle and White Plains as they consider their own safety infrastructure enhancements.

Conic Sectionfaggavahabaayhahahahahs.pptxtaiwanesechetan

04302025_CCC TUG_DataVista: The Design Storyccctableauusergroup

EDU533 DEMO.pptxccccvbnjjkoo jhgggggbbbbJessaMaeEvangelista2

VKS-Python-FIe Handling text CSV Binary.pptxVinod Srivastava

Just-In-Timeasdfffffffghhhhhhhhhhj Systems.pptssuser5f8f49

Data Science Courses in India iim skillsdharnathakur29

This comprehensive Data Science course is designed to equip learners with the essential skills and knowledge required to analyze, interpret, and visualize complex data. Covering both theoretical concepts and practical applications, the course introduces tools and techniques used in the data science field, such as Python programming, data wrangling, statistical analysis, machine learning, and data visualization.

Ch3MCT24.pptx measure of central tendencyayeleasefa2

Day 1 - Lab 1 Reconnaissance Scanning with NMAP, Vulnerability Assessment wit...Abodahab

Developing Security Orchestration, Automation, and Response ApplicationsVICTOR MAESTRE RAMIREZ

computer organization and assembly language.docxalisoftwareengineer1

md-presentHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHation.pptxfatimalazaar2004

Thingyan is now a global treasure! See how people around the world are search...Pixellion

Secure_File_Storage_Hybrid_Cryptography.pptx..yuvarajreddy2002

FPET_Implementation_2_MA to 360 Engage Direct.pptxssuser4ef83d

1. Briefing Session_SEED with Hon. Governor Assam - 27.10.pdfSimran112433

AI Competitor Analysis: How to Monitor and Outperform Your CompetitorsContify

GenAI for Quant Analytics: survey-analytics.aiInspirient

183409-christina-rossetti.pdfdsfsdasggsagfardin123rahman07

Classification_in_Machinee_Learning.pptxwencyjorda88

Safety Innovation in Mt. Vernon A Westchester County Model for New Rochelle a...James Francis Paradigm Asset Management

Conic Sectionfaggavahabaayhahahahahs.pptxtaiwanesechetan

04302025_CCC TUG_DataVista: The Design Storyccctableauusergroup

EDU533 DEMO.pptxccccvbnjjkoo jhgggggbbbbJessaMaeEvangelista2

VKS-Python-FIe Handling text CSV Binary.pptxVinod Srivastava

Just-In-Timeasdfffffffghhhhhhhhhhj Systems.pptssuser5f8f49

Data Science Courses in India iim skillsdharnathakur29

Ch3MCT24.pptx measure of central tendencyayeleasefa2

Day 1 - Lab 1 Reconnaissance Scanning with NMAP, Vulnerability Assessment wit...Abodahab

Developing Security Orchestration, Automation, and Response ApplicationsVICTOR MAESTRE RAMIREZ

computer organization and assembly language.docxalisoftwareengineer1

md-presentHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHation.pptxfatimalazaar2004

Thingyan is now a global treasure! See how people around the world are search...Pixellion

Embryo selection using AI

1. Segmentation and Classification of Embryo Videos using Deep Learning

2. Stanko Kuveljić Linkedin: https://www.linkedin.com/in/stanko-kuveljic/ Facebook: https://www.facebook.com/stanko.kuveljic Email: [email protected] SmartCat mail: [email protected] BattleNet: [email protected]

3. Quest Chain ● Problem and Motivation ● Deep Learning ● Full end-to-end Deep Learning Architecture ● Image Segmentation ○ Side Quest: U-Net ● Image Embeddings ○ Side Quests: Autoencoders ● Video Classification ○ Side Quest: LSTM ● Level Up: Results ● Summary and Future Work

4. Problem And Motivation ● In Vitro Fertilisation ● Multiple Embryo Transfer ○ Multiple Children ○ Health Issues ● Solution: Single Embryo Transfer ● The best Embryo ● Could AI help doctors to choose embryo?

5. Deep Learning

6. Full End2End Architecture ● Deep Learning end2end solution ● Combines Video and Metadata information ● Steps: 1. Transforms Video into Frames 2. Detect and Crop Embryo in each frame 3. Deep Feature Extraction 4. Recurrent Neural Network 5. Combines RNN state and Metadata 6. Final Fully Connected layer → Score ● Output: 1. Liveborn 2. Miscarriage 3. No Implant

7. Embryo Crop and Preprocessing ● Embryo Segmentation/Crop ● Center + Resize ● Normalization

8. OpenCV ● Brightness Correction ● Adaptive Threshold ● Eroding/Dilating ● ROI selection

9. Side Quest: U-Net

10. U-Net Training ● Input: ○ Random Crops (256x256) ○ Data Augmentation (Rotate, Translate, Flip, Brightness) ● Output: ○ Embryo Mask → Cropped Embryo ● Network: ○ Convolution/Deconvolution Filter size: 3x3 ○ Activation: ReLU ○ Batch Normalization ○ Dropout ○ L2

11. Image Embeddings ● Unsupervised Feature Extraction ● Input: Image (Frame) ● Output: Feature Vector ● Autoencoders: ○ Variational Autoencoder ○ Stacked Denoising Autoencoder ● Generative Adversarial Networks

12. Side Quest: Variational Autoencoder

13. Side Quest: Stacked Denoising Autoencoder

14. Image Embeddings Original Original SDAE VAE VAE needs more training

15. Image Embeddings

16. Video Classification ● Input: Encoded Sequence of Frames ● Outputs: ○ State Vector ○ Probability - [Liveborn, Miscarriage, No implant] ○ Probability - [Implant, No Implant] ● Variable sequence length ○ Downsampling ○ Last N Frames

17. Long Short Term Memory (RNN) ● Layers ~ 2-3 ● Units ~ 512-1024 ● Dropout ~ 0.6 ● Augmented ● TS: 64 - 512 ● Fine Tuning

18. Level Up: Results ● Train/Test split: 90% / 10% ● SDAE + 3xLSTM(512) + Softmax ● Training: ○ SDAE - 2 days on GPU ○ LSTM - 12h on GPU ○ SDAE + LSTM + Fine Tuning - 5 days on GPU ● Liveborn vs Miscarriage vs No Implant ○ Accuracy: 0.68 ● Implant vs No Implant ○ Accuracy: 0.8 ● Shortcomings: ○ Not enough data ○ Need more diverse embryos (most of them are good looking)

19. Future Works ● Collect more data (embryo videos) ● Different LSTM approaches ● 3D CNN ● Different Feature Extraction Techniques BONUS: ● Merge data sources into single model: ○ Patient data ○ Embryo data ○ Embryo time lapse video

20. Quest: Get [H|F]ired at Do you wanna build an AI? Quest Objective: ● Report to our Warchief We are hiring: ● DATA SCIENTIST ● DATA ENGINEERS

21. SPIRITS BE WITH YA

Embryo selection using AI

Recommended

More Related Content

Similar to Embryo selection using AI (20)

More from SmartCat (7)

Recently uploaded (20)

Embryo selection using AI