![“[Self] Attention is all you need”
• 2017 breakthrough paper titled “Attention is
all you need” introduced the Transformer
model architecture.
• Key idea: Focus on “positional encodings”,
“attention” and “self attention” for the tokens
(integers) representing the words/sub-words.
https://arxiv.org/abs/1706.03762](https://image.slidesharecdn.com/chatgpt-and-generative-ai-landscape-240322065627-7361844f/85/ChatGPT-and-Generative-AI-Landscape-Working-of-generative-ai-search-13-320.jpg)
ChatGPT-and-Generative-AI-Landscape Working of generative ai search
- 1. ChatGPT and Generative AI Landscape (UW Guest Lecture) Sanjeev Jagtap
- 2. Contents
- 3. Demo Harry Potter and the Goblet of Fire Chapter Seven
- 4. Harry Potter and The Goblet of Fire: Chapter Seven
- 5. When asked, ChatGPT gets imaginative, incorrectly making up the book and school names aka “hallucinates”.
- 6. We help by providing the page text and it correctly picks out the correct answers. (Can also be search results, documents, web pages, …)
- 7. Another example, similar mistake to begin with… but this time, we ask it to “think it through”….
- 8. Still stubbornly incorrect, but we persist in nudging and instructing…
- 10. Takeaways • Semantically, ChatGPT has no idea. It’s just doing Math. • Factually, it can occasionally make errors. • It has awesome natural language understanding and generation skills. • Prompting and grounding with missing knowledge closes the gaps. • Generative AI models are “jack of all trades”. • “Collaborate” with them via prompting, finetuning, humans, models. • Think “Copilots”, “Assistants”, “Wizards”, “Suggestions”…
- 11. The Tech
- 12. The Problem of Language modeling Pre-2017, while CNNs (convolutional neural networks) worked great for images, RNNs (recurrent neural networks) for language did not. RNNs were sequential, error-prone, and did not capture the language model insights and were therefore did not output natural looking text.
- 13. “[Self] Attention is all you need” • 2017 breakthrough paper titled “Attention is all you need” introduced the Transformer model architecture. • Key idea: Focus on “positional encodings”, “attention” and “self attention” for the tokens (integers) representing the words/sub-words. https://arxiv.org/abs/1706.03762
- 14. In plain English Capture a sense of “nearness” by similarity of use, numerical distance, etc. and relate to other words (aka tokens) for ordering and deep understanding of the language model to compute the probability of the next token in sequence. (Its just Math)
- 15. Over-simplified How-it-works • Assign to each unique word a unique identifier, a number that will serve as a token to represent that word. • Note the location of every token relative to every other token. • Using just token and location—determine the probability of it being adjacent to, or in the vicinity of, every other word. • Feed these probabilities into a neural network to build a map of relationships. • Given any string of words as a prompt, use the neural network to predict the next word (just like AutoCorrect). • Based on feedback, adjust the internal parameters of the neural network to improve its performance. • Extend the prediction to the next phrase, the next clause, the next sentence, the next paragraph, and so on, by using feedback to further adjust its internal parameters. • Based on the above, generate text responses to user questions and prompts that reviewers agree are appropriate and useful.
- 16. Summarizing • Generative AI language models fundamentally predict the next words (represented as numerical tokens) in sequence like auto-complete. • Words are tokenized into numbers and the model processes these “tokens” while dealing with context, nearness, and weights in the neural network and iterating on them as the model is trained.
- 17. Adoption landscape and trade-offs Prompted: Zero or One-shot Prompted: Multiple-shot Finetuned DIY General/Broad task Narrow task LLM LLM Specialized Specialized + Synthetic data generation AI Quality Evaluation Inferencing in production Usage Scope Composition Lifecycle Iterative benchmarking and regression testing become critical.
- 18. The Race
- 19. AI models were getting larger quickly… AS AI MODELS HAVE GOTTEN PROGRESSIVELY LARGER THEY HAVE BEGUN TO SURPASS MAJOR HUMAN PERFORMANCE BENCHMARKS. SOURCES: © THE ECONOMIST NEWSPAPER LIMITED, LONDON, JUNE 11TH 2022. ALL RIGHTS RESERVED; SCIENCE.ORG/CONTENT/ARTICLE /COMPUTERS-ACE-IQ-TESTS- STILL-MAKE-DUMB-MISTAKES- CAN-DIFFERENT-TESTS-HELP https://www.sequoiacap.com/article/ generative-ai-a-creative-new-world/
- 20. Recall the Tech Adoption‘S’Curves Transitioning from one S-curve to another is the tricky part for companies. The bridges go over choppy waters. The transitions between innovations can span months and years. Therefore, you need bridges in form of hybrid systems, handovers of technology and customers, and managing of resources and investment as the landscape shifts.
- 21. The race in a nutshell Technological discontinuity Era of ferment Era of incremental change Dominant design emerges Cost decreasing with usage Chosen dominant design Learn (fast) Adopt (fast) Dominate (fast) Performance increasing with usage Size of installed base Availability of complimentary goods
- 22. First mover(s) pros & cons 1 2 3 Winner(s) take(s) all Brand amplification and loyalty Technological leadership Preemption of scarce assets Exploitation of buyer switching costs • Research and development expenses • Undeveloped supply and distribution channels • Immature enabling technologies and complements • Uncertainty of customer requirements Winner(s) must deal with: There is no free lunch. Just strategic tradeoffs.
- 23. Which CEO said what (paraphrasing) “The one that I think is going to have the fastest direct business loop is going to be around helping people interact with businesses. I mean, you can imagine a world where over time every business has an AI agent that basically people can message and interact with them. … It’s quite human labor intensive for a person to be on the other side of that interaction.” “Some of our most successful products were not first to market. They gained momentum because they solved important user needs and were built on deep technical insights.” “The most important thing we are doing in AI is trusted and responsible AI. Customers, particularly large enterprises, are equal parts intrigued and concerned by the technology’s potential. We’ve all seen the movies and where this can go. We have all these crazy ideas in our head of what can happen.” “Many of today’s A.I. chatbots and other generative A.I. tools are part of the “hype cycle,” while we are focused on the “substance cycle…This is a 10K race while everyone is just 3 steps in.” “We are making the first moves and if our friends (competition) respond and they will, we want everyone to know we made them move.” Amazon Microsoft Facebook Google Salesforce
- 24. Customers
- 25. Most practical customer scenarios • Written content augmentation and creation: Producing a “draft” output of text in a desired style and length • Question answering and discovery: Enabling users to locate answers to input, based on data and prompt information • Tone: Text manipulation, to soften language or professionalize text • Summarization: Offering shortened versions of conversations, articles, emails and webpages • Simplification: Breaking down titles, creating outlines and extracting key content • Classification of content for specific use cases: Sorting by sentiment, topic, etc. • Software coding: Code generation, translation, explanation and verification https://www.gartner.com/en/topics/generative-ai
- 28. https://www.sequoiacap.com/article/generative-ai-a-creative-new-world/ Generative AI projected timeline view
- 29. Advanced Use Cases by Industry https://www.gartner.com/en/articles/beyond-chatgpt-the-future-of-generative-ai-for-enterprises “Generative AI has already been used to design drugs for various uses within months, offering pharma significant opportunities to reduce both the costs and timeline of drug discovery.”
- 30. In a recent Gartner webinar poll of more than 2,500 executives, 38% indicated that customer experience and retention is the primary purpose of their generative AI investments. This was followed by revenue growth (26%), cost optimization (17%) and business continuity (7%). https://www.gartner.com/en/articles/beyond-chatgpt-the-future-of-generative-ai-for-enterprises
- 31. Costs: From negligible to many millions • Free versions of public, openly hosted applications, such as ChatGPT, or by paying low subscription fees. However, free and low-cost options come with minimal protection of enterprise data and associated output risks. • Larger enterprises and those that desire greater analysis or use of their own enterprise data with higher levels of security and IP and privacy. In this instance, costs can be in the millions of dollars. • Generative AI capabilities will increasingly be built into the software products you likely use everyday, like Bing, Office 365, Microsoft 365 Copilot and Google Workspace. Vendors will pass on costs to customers as part of bundled incremental price increases. Microsoft recently announced their per-user pricing for Office 365 CoPilot pricing.
- 32. Careers
- 33. Product (Managers) • Learn up the what/how(AI level) to update product (what/why) • More OOB model power => emphasis on user value and experience • User generated vs. AI generated content trade-offs • More collaborative with higher accountability bar • Career opportunity to get into AI (Tech)
- 34. Engineering professionals • Vocabulary – Prompting, Finetuning, Turns, … • Engineering – Systems, frameworks, tooling, metrics • Frameworks- LangChain, Semantic Kernel • Prompting – knowledge, tooling (PromptFlow) • Research – Augment, enhance, build on top • Open source – Alternatives, crowd-sourced
- 35. Business planning • New underlying economics • Willingness to pay (WTP) questions • Pricing models – new units (e.g., tokens) • Stand-alone vs. bundling with existing • New and more stakeholders to manage • Business projections vs. realization
- 36. User experience design impact* Collaborative not command AI notices ala disclaimers Reverse- prompt users Citations and references Feedback for improvement Address costs and speed https://www.linkedin.com/learning/ux-for-ai-design-practices-for-ai-developers/designing-for-ai
- 37. Software Testing/QA • Learn about AI quality metrics and lifecycle • Word Error Rate (WER), FactX, ROUGE-L, Human evaluations • Precision, Recall, F1-Score, Ground truth (reference) data • Learn about development and evaluation data set creation • Learn about AI benchmarking and regression best practices • Create a GitHub repo, download a public dataset as your ground truth (reference), create a test dataset from the internet, GPT, or your own and publish your findings
- 38. Recap career tips Generative AI levels the playing field between tech and non-tech Opportunity for entry into the tech and AI space New disciplines like Prompt Engineering are hot Responsible AI is an example of new careers to learn up and enter the AI industry Use experience design principles are evolving to meet challenges For newbies, learn it up while prioritizing your target niche For experienced ones, decide how and where you want to play
- 39. Responsible AI
- 40. Why it is more important today With Generative AI power also come significantly increased challenges related to harmful content, manipulation, human-like behavior, privacy, and more. The industry is unanimous and for good reasons that responsible use and AI need to be tied to the hip.
- 41. Example planning framework • Policy • Research • Engineering • Fairness • Reliability and safety • Privacy and Security • Inclusiveness • Transparency • Accountability • Operationalization • Advocacy • Alliances • Positioning Pillars Principles Practices https://www.microsoft.com/en-us/ai/responsible-ai
- 42. Responsible Use Product Guidance (example) • Harm mitigation • Identify • Measure • Mitigate • Operation • Transparency note • Introduction • Concepts • Intended use cases • Considerations when choosing a use case • Capabilities • Limitations • Limited access aka “gating” • Application • Review of use case • Other criteria • Approvals • Code of Conduct • Access requirements • Content requirements • Mitigations requirements Azure OpenAI Responsible Use Guidance (example)
- 43. Data, privacy, and security • What data is processed? • How does the service process data? • Cover over the wire and in the cloud infrastructure • Which secure networking options are available • Regulatory compliance certifications • Government clouds, Public clouds Microsoft Azure Compliance Offerings (example)
- 44. Content safety as a service https://azure.microsoft.com/en-us/products/ai-services/ai-content-safety
- 45. Takeaways • Learn up on the latest and what companies and agencies are up to • Excellent way to get into AI as a career • Critical regulatory and compliance space that’s poised to grow • Solving hard problems while working with AI researchers
- 46. • https://www.deeplearning.ai/short-courses/chatgpt-prompt-engineering-for-developers/ • https://www.deeplearning.ai/short-courses/building-systems-with-chatgpt/ • https://build.microsoft.com/en-US/sessions/db3f4859-cd30-4445-a0cd-553c3304f8e2 • https://karpathy.medium.com/software-2-0-a64152b37c35 • https://www.linkedin.com/pulse/understanding-chatgpt-triumph-rhetoric-geoffrey-moore • https://developer.nvidia.com/blog/deep-learning-nutshell-core-concepts/ • https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work/ • https://daleonai.com/transformers-explained • https://a16z.com/2023/05/25/ai-canon/ More references
- 47. Thank You!