Deep Learning with TensorFlow: Understanding Tensors, Computations Graphs, Images, and Text
Download as PPTX, PDF8 likes23,826 views
1. The elements of Neural Networks: Weights, Biases, and Gating functions 2. MNIST (Hand writing recognition) using simple NN in TensorFlow (Introduce Tensors, Computation Graphs) 3. MNIST using Convolution NN in TensorFlow 4. Understanding words and sentences as Vectors 5. word2vec in TensorFlow
![Words as vectors (Embeddings)
list → [human, cow] → int list [0, 1] → embeddings [(2,2), (4,0)]](https://image.slidesharecdn.com/deeplearningwithtensorflowunderstandingtensorscomputationsgraphsimagesandtext-160614151136/85/Deep-Learning-with-TensorFlow-Understanding-Tensors-Computations-Graphs-Images-and-Text-28-320.jpg)
![[PR12] PR-036 Learning to Remember Rare Events](https://cdn.slidesharecdn.com/ss_thumbnails/pr12pr-036learningtoremeberrareevents-170917140144-thumbnail.jpg?width=560&fit=bounds)
Ad
More Related Content
What's hot (19)
Similar to Deep Learning with TensorFlow: Understanding Tensors, Computations Graphs, Images, and Text (20)
![[Revised] Intro to CNN](https://cdn.slidesharecdn.com/ss_thumbnails/googletechsprinttalkcnnintro-200730141604-thumbnail.jpg?width=560&fit=bounds)
Ad
More from Altoros (20)
Ad
Recently uploaded (20)
Deep Learning with TensorFlow: Understanding Tensors, Computations Graphs, Images, and Text
- 1. Deep Learning with TensorFlow Understanding Tensors, Computation Graphs, Images, Text Viswanath Puttagunta Technical Program Manager, Linaro
- 2. Vish Puttagunta Technical Program Manager, Linaro Statistics/Signal Processing in Images, Audio, RF Data Analysis and Machine Learning on ARM SoCs. Leading collaboration in ARMTM Ecosystem Enterprise, Mobile, Home, Networking, IoT Built on Open Source
- 3. Goals Basics behind Neural Networks Picturing Neural Networks. Understand the operations. Representing Neural Networks using Tensors and Computation Graphs Vision (MNIST), Text (Word2Vec) TensorFlow Gives you good tools to build a Neural Network Model. Some basic models… but Google's secret sauce still inside Google :)
- 4. The Basics Linear Regression Weight / Bias Cost Function Gradient Descent to optimize a cost function Logistic Regression Neuron Activation
- 5. Linear Regression Objective: Fit line/curve to minimize a cost function Line has "Weight / Slope" and "Bias / y-
- 6. Cost Function, Gradient Descent Source: https://www.youtube.com/watch?v=SqA6TujbmWw&list=PLE6Wd9FR--Ecf_5nCbnSQMHqORpiChfJf&index=16 https://youtu.be/WnqQrPNYz5Q?list=PLaXDtXvwY-oDvedS3f4HW0b4KxqpJ_imw&t=284 ● Cost Function, J(𝞡) ○ Weight / Slope: 𝞡1 ○ Bias / y-intercept: 𝞡2 ● Gradient Descent ○ Go down to the lowest point in the cost function in small steps BIG IDEA ● Starts with some Weights & Biases ● Define a Cost Function ● Optimize the Cost function by doing some sort of Gradient Descent ● Lots of Help from TensorFlow.
- 7. Weight, Bias in 0,1 Dimensions
- 8. Neuron and Activation Function BIG IDEAS ● Only one of the Neurons will fire at a time (value shoots greater than 0.5) ● Which neuron depends on "x", Weights, Biases ● Rest of neurons will not fire. (value much less than 0.5) ● Outcomes are between 0 and 1 → Probabilities (all outputs add to one) ○ → Good for classification
- 9. Neuron and Activation Function BIG IDEAS ● Only one of the Neurons will fire at a time (value shoots greater than 0.5) ● Which neuron depends on "x", Weights, Biases ● Rest of neurons will not fire. (value much less than 0.5) ● Outcomes are between 0 and 1 → Probabilities (all outputs add to one) ○ → Good for classification
- 10. Neuron and Activation Function BIG IDEAS ● Only one of the Neurons will fire at a time (value shoots greater than 0.5) ● Which neuron depends on "x", Weights, Biases ● Rest of neurons will not fire. (value much less than 0.5) ● Outcomes are between 0 and 1 → Probabilities (all outputs add to one) ○ → Good for classification
- 11. Same picture: Less mess, more classes
- 12. Simple Neural Network for MNIST Source: https://www.tensorflow.org/versions/r0.7/tutorials/mnist/beginners/index.html
- 13. Simple Neural Network for MNIST Weights actually Mean something!!
- 14. Weights/Bias → Evidence → Probability BIG IDEAS ● After proper Gradient Descent, Weights Generalize well. ● Correlation operation corresponds to "Evidence" ● Then do softmax(evidence) and you will get Probability(x 𝞊 specific class) Source: https://www.tensorflow.org/versions/r0.7/tutorials/mnist/beginners/index.html
- 15. MNIST Simple: Operations / Dimensions ● input image(x) ○ 28x28 matrix → 1x784 ○ each element type: 0 or 1 ● Weights (W0, W1…. W9) ○ 28x28 matrix → 1x784 ○ each element: float(-1, +1) ● Eg: W1Ⓧx (Correlation) ○ Multiply Accumulate! ■ matmul(W, xT) ○ single float (-1, +1) ● Biases (B0, B1…. B9) ○ single floating pt (-1, +1) ● Evidence: Eg: W1Ⓧx + B1 ○ single floating pt (- 784.0,+784.0) ● Probability ○ single float (0,1) ● Real output (Y), Predicted o/p(Y^) ○ 1x10 matrix ○ each is 0 or 1
- 16. MNIST Simple: In TensorFlow
- 17. MNIST Simple: In TensorFlow
- 19. MNIST using Convolution Neural Network
- 22. Weight, Bias in 0,1 Dimensions
- 23. MNIST: Using CNN
- 24. MNIST: Using CNN
- 25. MNIST: Using CNN
- 26. MNIST: Using CNN
- 27. Text Analytics Word2Vec Thinking of words and as vectors Sequence-to-Sequence models Recurrent Neural Networks (LSTM) Understanding sequence of words "Don't like" is closer to "hate"
- 28. Words as vectors (Embeddings) list → [human, cow] → int list [0, 1] → embeddings [(2,2), (4,0)]
- 29. Sequence of words as vectors!
- 33. word2vec: using Neural Network
- 34. word2vec: using Neural Network Note: Only trying to understand relationship between words and understand best way to vectorize words! Not really keen on sequence of words! (Later RNN)
- 35. word2vec: using NN (softmax classifier) Realize wv could easily be w50000 words in vocabulary!
- 36. word2vec: using NN (softmax classifier-pitfall) Too many computations for every set of inputs (Eg: Classifying across 50,000 classes if you vocabulary size is 50,000)
- 37. word2vec: using Noise Classifier Big Idea: Instead of trying to move every word(vector) by little bit in each step Move few random words (~16) at a time If you have lots of sentences, you should be good
- 39. word2vec Code Walk Through https://github.com/tensorflow/tensorflow/blob/master/tensorflow/exa mples/tutorials/word2vec/word2vec_basic.py
- 40. Summary TensorFlow was simple to install (Ubuntu 14.04) sudo apt-get install python-pip python-dev sudo pip install --upgrade https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow- 0.8.0rc0-cp27-none-linux_x86_64.whl Tutorials are 'relatively' easy to follow Good pointers to papers and blogs Would like to hear Google's perspective on
- 41. References Convolution Neural Network: https://youtu.be/n6hpQwq7Inw TensorFlow Tutorials: https://www.tensorflow.org/versions/r0.7/tutorials/index.html Khan Academy (Linear algebra): https://www.khanacademy.org/ LSTM, Text Analytics: Alec Ratford, Indico https://www.youtube.com/watch?v=VINCQghQRuM Current trends in Deep Learning (Andrew Ng, Baidu) https://www.youtube.com/watch?v=O0VN0pGgBZM