Six Steps to Conversation Driven Development
0 likes396 views
This document discusses Conversation-Driven Development (CDD) as a process for building conversational AI assistants. It describes CDD as involving six key actions: sharing prototypes with test users, reviewing user conversations, annotating conversations as training data, testing the assistant, tracking the success of conversations, and fixing issues found. CDD captures lessons learned from building conversational AI to help teams iteratively improve assistants based on user feedback. The document also promotes Rasa X as a tool that can help teams implement CDD through features like reviewing conversations, annotating data, and continuous integration testing.
Six Steps to Conversation Driven Development
- 1. Conversation-Driven Development Alan Nichol | co-founder, CTO @ Rasa July 2020
- 2. Agenda CONVERSATION DRIVEN DEVELOPMENT ● About Rasa ● Conversation Driven Development ● CDD in Practice ● Q&A Housekeeping ● Please keep yourself muted and your camera turned off ● Use the Zoom chat to ask questions. We’ll answer questions at the end, but you can ask your questions at any point during the broadcast ● We’ll post the recorded webinar on YouTube in a few days. We’ll email you the link along with a short survey.
- 4. About Rasa
- 5. We’re building the standard infrastructure for conversational AI HOW WE ACHIEVE OUR MISSION Champion Open Source Tools Nurture a Community of Makers Ship Applied Research to Solve Real Problems
- 6. Developers all over the world build with Rasa: 2m+ downloads, 9,500+ forum members, 400+ contributors RASA COMMUNITY Healthcare Insurance Banking Telecommunications Manufacturing Technology
- 8. The 5 levels of conversational AI got a 2020 upgrade
- 9. All of you are already gathering the data you need to climb from level 3 to 4 and 5. You just need to listen.
- 10. Conversation-Driven Development: the process of listening to your users and using those insights to improve your AI assistant
- 12. CDD captures the lessons we’ve learned as a community WHY CDD If you’ve built conversational AI before, you know that: ● It is very hard. ● Building a prototype is not the hard part. ● The hard parts all show up when you want to go from a prototype to something you’d want to ship. Conversation-Driven Development should: ● Help all of us build better conversational AI. ● Save newcomers from having to learn this the hard way.
- 13. Conversation-Driven Development is made up of six actions ABOUT CDD share fix track review annotate test
- 14. Users will always surprise you. So get some test users to try your prototype as early as possible. Shipping without having a bunch of test users has never worked. Your project won’t be the exception. Share
- 15. Review At every stage of a project, it is worth reading what users are saying. Avoid getting caught up in metrics right away. Conversations are valuable data.
- 16. Annotate Using a script to generate synthetic training data Turning real messages into training examples
- 17. Test Professional teams don’t ship applications without tests. Use whole conversations as end-to-end tests Run them on a continuous integration (CI) server.
- 18. Prototype Track Use proxy measures to track which conversations are successful and which ones failed. ‘Negative’ signals are useful too, e.g. users not getting back in touch with support.
- 19. Prototype Fix Study conversations that went smoothly and ones that failed. Successful conversations can become new tests 🎉 Fix issues by annotating more data and/or fixing your code 🔧
- 20. CDD in practice
- 21. It’s not a linear process: you’ll find yourself jumping between these actions PROCESS share fix track review annotate test
- 22. Some actions require software skills, others a deep understanding of the user TEAM Image credit: Freepik
- 23. Rasa X is a free tool to help teams do CDD
- 24. ≈≈ Rasa X layers on top of Rasa Open Source PRODUCT Rasa X: Tool for Conversation-Driven Development Rasa Open Source: Framework for building AI assistants
- 25. Rasa X local mode
- 26. Rasa X server mode
- 27. Share your bot with testers using just a link
- 28. Review conversations coming in from every channel
- 29. Annotate the messages coming in
- 30. #232a2e Push new training data to git and trigger your CI pipeline
- 31. Track failures and successes Use the API to automatically tag conversations, or add tags manually as you read Turn successful conversations into new end-to-end tests
- 32. Q&A
- 33. Recap
- 34. CDD captures the lessons we’ve learned as a community WHY CDD Conversational AI is hard - CDD helps us make our assistants better over time. As experienced conversational AI teams, we owe it to newcomers to help them avoid mistakes. That’s the only way this field can continue to progress.
- 36. Share your knowledge and help other teams move faster rasa.com/cdd-quiz
- 37. Alan Nichol Co-founder & CTO [email protected] Email me!