Rsqrd AI: Making Conversational AI Work for Everybody
- 1. Making Conversational AI Work for Everybody: An Intro to Conversation Driven Development Dr. Rachael Tatman
- 3. How I got here About Me ● 2012- 2017: PhD in Linguistics ○ Modelling demographic disparities in Automatic Speech Recognition ○ FAT* ML ● 2017- 2020: Developer Advocate @ Kaggle (Google) ○ "The home of data science" ○ Machine learning education ○ Live Streaming: Livecoding & Paper Reading ● 2020 - now: Developer Advocate @ Rasa ○ Open source conversational AI framework & free software for conversation driven development ○ Developer education ○ Video, blogs, tutorials, live streaming, talks :)
- 4. Sooo.... why do I choose to work on chatbots/ conversational AI?
- 5. Some false assumptions we tend to make - Everyone has access to a computer - Everyone has access to the internet - Everyone knows how to Google stuff - Everyone likes visual interfaces - Everyone has a good/affordable enough internet connection to load graphics - Everyone knows how to use a website - Everyone knows how to use YOUR website - Everyone can make calls during business hours https://answers.microsoft.com/en-us/ie/forum/ie11-iewindows8_1/how-much-data-being-us ed-when-visiting-a-website/5028b804-3adf-4093-b10a-a058d020b19a
- 6. Why conversational AI? - Most people already know how to have a conversation - NLP methods are able to handle a lot more variation in input now - A focused Conversational AI can help people do tasks much faster - Chalmers, a conversational assistant for people experiencing homelessness, helped people find information 60 times faster than searching (https://www.amplelabs.co/chalmers/)
- 7. Why conversational AI? - Most people already know how to have a conversation - NLP methods are able to handle a lot more variation in input now - A focused Conversational AI can help people do tasks much faster - Chalmers, an converational assistant for people experiencing homelessness, helped people find information 60 times faster than searching Your assistant has to hold up their end of the bargain: have a good, helpful conversation that meets your users’ expectations and genuinely helps them. Good design is important, but you probably can’t predict everything your users will need/want/do.
- 9. CDD captures the lessons we’ve learned as a community WHY CDD If you’ve built conversational AI before, you know that: ● It is very hard. ● Building a prototype is not the hard part. ● The hard parts all show up when you want to go from a prototype to something you’d want to ship. Conversation-Driven Development should: ● Help all of us build better conversational AI. ● Save newcomers from having to learn this the hard way.
- 10. Conversation-Driven Development is made up of six actions ABOUT CDD share fixtrackreview annotate test
- 11. Users will always surprise you. So get some test users to try your prototype as early as possible. Shipping without having a bunch of test users has never worked. Your project won’t be the exception. Share
- 12. Review At every stage of a project, it is worth reading what users are saying. Avoid getting caught up in metrics right away. Conversations are valuable data.
- 13. Annotate Using a script to generate synthetic training data Turning real messages into training examples
- 14. Test Professional teams don’t ship applications without tests. Use whole conversations as end-to-end tests Run them on a continuous integration (CI) server.
- 15. PrototypeTrack Use proxy measures to track which conversations are successful and which ones failed. ‘Negative’ signals are useful too, e.g. users not getting back in touch with support.
- 16. PrototypeFix Study conversations that went smoothly and ones that failed. Successful conversations can become new tests 🎉 Fix issues by annotating more data and/or fixing your code 🔧
- 17. CDD in practice
- 18. It’s not a linear process: you’ll find yourself jumping between these actions PROCESS share fixtrackreview annotate test
- 19. Some actions require software skills, others a deep understanding of the user TEAM Image credit: Freepik
- 20. How does this help?
- 21. Why conversational AI? - Most people already know how to have a conversation - NLP methods are able to handle a lot more variation in input now - A focused Conversational AI can help people do tasks much faster - Chalmers, an converational assistant for people experiencing homelessness, helped people find information 60 times faster than searching Your assistant has to hold up their end of the bargain: have a good, helpful conversation that meets your users’ expectations and genuinely helps them. Good design is important, but you probably can’t predict everything your users will need/want/do.
- 22. Why does CDD help? - CDD lets you meet users where they are now and help them right away - You don’t have to predict or guess what your users want or need - You can make quick, incremental improvements - You can meet shifting needs or expectations
- 23. Rachael Tatman @rctatman [email protected] Rasa Technologies GmbH, Schönhauser Allee 175, 10119 Berlin, Germany Questions?