![# of Presentations
• Frequency
– Four in the semester
– One for the paper review and three for the project
• Expectations
– Present details
• Describe the problem: formulation, experiment, approaches, datasets
• Who has done what and compare your approach
• Final results; show results, demo code if possible
– Please clearly cite the source of each slide that is not your
own and other citations for all presentations
• Citation can be simple, small & bottom ex:
Last name 1st
author, ”Title maybe abbrev, Venue, year
[Giles, ”Face Recognition …,”IEEETNN, 97]](https://image.slidesharecdn.com/intro-597-250513131047-c5157cbf/85/Introduction-to-Deep-Learning-and-ML-pptx-25-320.jpg)
![Computational Resources
• 1] Google collab also has GPUs.
• 2] Kaggle [Nvida Tesla P100 ]
Both Kaggle and Colab offer free GPUs
– Jupyter Notebooks in the browser designed to foster collaboration.
• 3] CyberLamp -ACI
This has to be requested by the instructor. Please specify number of
GPUs needed.
Thanks to Drew Polasky for creating containers on cyberlamp
• 4] Gradient GPU[https://blog.paperspace.com/free-cloud-gpu/]
• 5] Amazon SageMaker studio lab
• 6] Microsoft Azure for students](https://image.slidesharecdn.com/intro-597-250513131047-c5157cbf/85/Introduction-to-Deep-Learning-and-ML-pptx-31-320.jpg)
Introduction to Deep Learning and ML.pptx
- 1. IST 597 Deep Learning Class Introduction Thanks to Dhruv Batra Georgia Tech http://clgiles.ist.psu.edu/IST597
- 2. IST 597 – OER Course • IST 597 is a mostly an Open Educational Resource class. • “Open educational resources (OERs) are freely accessible, openly licensed text, media, and other digital assets that are useful for teaching, learning, and assessing as well as for research purposes.” • All material in IST 597 that is not private is open, free, and online. wikipedia
- 3. IST 597 – Open to all at Penn State • Deep learning (AI) affects all of science, engineering, medicine, humanities, law, politics and warfare. • You need to know how it works and what it does and doesn’t do. • Our goal – make it one of the most useful courses you take at Penn State. • We will ask a lot of you.
- 4. Course goal • Prepare you for deep learning projects in –Academia –Industry –Government –Others
- 5. • David Reese Professor – IST; graduate Professor - CSE • Adjunct Professor – Princeton, Pennsylvania, Columbia, Pisa, Trento • Graduated over 30 PhDs • Fellow - ACM, IEEE, INNS. IEEE Pioneer Award in Neural Networks, INNS Gabor Award • Published over 600 papers with over 57,000 citations and h-index of 116, most use machine, deep learning and AI • Intelligent and specialty search engines; cyberinfrastructure for science, academia and government; big data; deep learning – Modular, scalable, robust, automatic science and technology focused cyberinfrastructure and search engine creation and maintenance – Large heterogeneous data and information systems – Specialty science and technology search engines for knowledge discovery & integration • CiteSeerx (all scholarly documents – initial focus on computer science) (NSF funded) • MathSeer (new math search engine) (Sloan funded) • BBookX, ( Book generation, Question generation) (TLT funded) • Scalable intelligent tools/agents/methods/algorithms – Information, knowledge and data integration – Information and metadata extraction; entity recognition – Pseudocode, tables, figure, chemical formulae, equations, & names extraction – Unique search, knowledge discovery, information integration, data mining algorithms – Text in wild – machine reading, deep learning • Strong collaboration record (over 400 collaborators). • Lockheed-Martin, FAST, Raytheon, IBM, Ford, Alcatel-Lucent, Smithsonian, Internet Archive, DARPA, Yahoo, Dow Chemical NSF, Sloan, Mellon
- 6. PhD students of C Lee Giles
- 9. Google Scholar • Useful resource for DL papers
- 10. Google Scholar - LLMs
- 11. Machine Learning • Who here has had a machine learning course? –Computer science –Statistics –Mathematics –Engineering –Science –Others
- 12. Class Design • Course outline and materials open and freely available at – https://clgiles.ist.psu.edu/IST597 • Project hands-on oriented course. • 3 exercises, all in PyTorch – 30% – Exercises are individual projects • One large research/application team project –50% – Initial class presentations on the project – One update – Final class presentation – Project paper due at end of the semester • Class presentations on selected paper(s) – 10% • Class participation – 10% • Grades on canvas.
- 13. Class protocol • Seminar • Code examples • Discussion of projects • Related paper presentations • Questions to presenters by students
- 14. What’s Covered • Textbook for most of the material: • Deep Learning - https://www.deeplearningbook.org • Papers for newer areas – Transformer – Large Language Models – Pretrained Foundation models – Prompting – XAI for deep learning • Material changes from year to year – fast moving field • Deep learning foundations
- 15. What’s Covered • Intro to AI and computational complexity • Machine learning basics & computational complexity • Backpropagation & multilayer neural networks • Regularization and optimization • Practical methodologies • Convolution and recurrent networks • Autoencoders • Adversarial networks • Transformers • Large language models - foundation models • Science informed neural networks • Neural networks with memory
- 16. What’s Not Covered Yet • Reinforcement learning • Explainable and interpretable deep learning • Verifiable deep learning • Reproduceable deep learning • Ethical and fair deep learning
- 17. Coding • This is not a coding course. We will help you get started in PyTorch. • For this course you must already know how to code. We do not teach you how to code. • This is a hands-on course. You will build an application using PyTorch for your project. • A GitHub link to your project code must be built.
- 18. Frameworks – paperswithcode Why PyTorch
- 20. 2022
- 21. Prerequisite Knowledge Should know • Linear Algebra • Calculus • Probability / information theory • Numerical computation - Will give an introduction to Machine Learning • Programming! – Exercises will require PyTorch – Your language of choice for project
- 22. Prerequisites • Intro Machine Learning – Classifiers, regressors, loss functions, MLE, MAP • Linear Algebra – Matrix multiplication, eigenvalues, positive semi-definiteness… • Calculus – Multi-variate gradients, hessians, jacobians…
- 23. Paper Reviews by Teams • Written summary of paper (do not use an LLM) – Length 200-400 words. • Due: Midnight before class • Organization of summary and presentation – Summary: • What is this paper about? What is the main contribution? Describe the main approach & results. Just facts, no opinions yet. – List of positive points / Strengths: • Is there a new theoretical insight? Or a significant empirical advance? Did they solve a standing open problem? Or is a good formulation for a new problem? Or a faster/better solution for an existing problem? Any good practical outcome (code, algorithm, etc)? Are the experiments well executed? Useful for the community in general? – List of negative points / Weaknesses: • What would you do differently? Any missing baselines? missing datasets? any odd design choices in the algorithm not explained well? quality of writing? Is there sufficient novelty in what they propose? Has it already been done? Minor variation of previous work? Why should anyone care? Is the problem interesting and significant? – Reflections • How does this relate to other papers we have read? What are the next research directions in this line of work? • Powerpoint presentation in class
- 24. Questions by Teams • Each team presents 2 questions to the presenter after the presentation. • Questions should be about the paper presented. • Questions are written on canvas during class
- 25. # of Presentations • Frequency – Four in the semester – One for the paper review and three for the project • Expectations – Present details • Describe the problem: formulation, experiment, approaches, datasets • Who has done what and compare your approach • Final results; show results, demo code if possible – Please clearly cite the source of each slide that is not your own and other citations for all presentations • Citation can be simple, small & bottom ex: Last name 1st author, ”Title maybe abbrev, Venue, year [Giles, ”Face Recognition …,”IEEETNN, 97]
- 26. Team Project • Goal – Chance to try Deep Learning – Encouraged to apply to your research (computer vision, NLP, robotics,…) – Must be done this semester. – Can combine with other classes • get permission from both instructors; delineate different parts – Extra credit for shooting for a publication – Teams constructed by instructor – All students will fill out our an experience form
- 27. Team Project • Main categories – Competitions • Compete in a machine learning competition (ongoing or completed) using a different deep learning – Formulation/Development • Formulate a new model or algorithm for a new or old problem – Theory • Theoretically analyze an existing algorithm – Application • Compare a bunch of existing algorithms on a new application domain of your interest • Instructor/TA have projects if needed
- 28. Team Project Presentation • 15 minutes. Send your ppt before the presentation • Cover in order (no more than 15 slides) – Introduce your team – What you are going to do – What has been done so far – How will you do it - a time line – Why should we care • Practice your presentation – keep to the time restrictions • Instructor/TA have projects if needed
- 29. Collaboration Policy • Collaboration – Only on project – You may discuss the assignments – Each student writes their own answers – Write on your homework anyone with whom you collaborate – Each student must write their own code for the programming part • No plagiarism – Neither ethical nor in your best interest – Always credit your sources – Don’t cheat. We will find out.
- 30. Some Data sets for Deep Learning Open data sets – https://archive.ics.uci.edu – https://en.wikipedia.org/wiki/Wikipedia:Database_download – https://www.kaggle.com/datasets – http://primo.ai/index.php?title=Datasets – https://github.com/huggingface/datasets
- 31. Computational Resources • 1] Google collab also has GPUs. • 2] Kaggle [Nvida Tesla P100 ] Both Kaggle and Colab offer free GPUs – Jupyter Notebooks in the browser designed to foster collaboration. • 3] CyberLamp -ACI This has to be requested by the instructor. Please specify number of GPUs needed. Thanks to Drew Polasky for creating containers on cyberlamp • 4] Gradient GPU[https://blog.paperspace.com/free-cloud-gpu/] • 5] Amazon SageMaker studio lab • 6] Microsoft Azure for students
- 32. Most of what you will use can be accessed from this page http://clgiles.ist.psu.edu/IST597 Questions? Grades on canvas; also some assignments (exercises)