Building Conversational Experiences with Actions on Google
3 likes547 views
Since we first started using computers, we have dreamt of interacting with them in a more natural, human-like way using spoken language. Advances in machine learning and artificial intelligence have spiked an interest in natural language processing, and building conversational interfaces no longer is a dream, but a reality every developer can use in their own applications. The problem of recognising spoken input has been largely solved, and the next challenge is how to build compelling user experiences that are modelled after natural human conversation. In this talk, I will give an introduction to the Google Assistant and how to use Actions on Google to build conversational interfaces for devices with and without screens. We will take a look at how API.AI makes implementing natural language understanding feasible even for non-linguists, and discuss how you can integrate agents with existing backends. In conversation, it's all about being natural, so we will also look at how to create agents that feel like real characters.
![if (app.hasSurfaceCapability(app.SurfaceCapabilities.SCREEN_OUTPUT)) {
app.ask(app.buildRichResponse()
.addSimpleResponse(`Here's a fact for you. ${fact} Which one ` +
`do you want to hear about next, Google's history or headquarters?`)
.addBasicCard(
app.buildBasicCard('Google is an amazing company.')
.setImage(GOOGLE_LOGO_SRC))
.addSuggestions(['History', 'Headquarters']));
} else {
app.ask(`Here's a fact for you. ${fact} Which one ` +
`do you want to hear about next, Google's history or headquarters?`);
}](https://image.slidesharecdn.com/actionsongoogle-buildingconversationalexperienceswithactionsongoogle-170713155850/85/Building-Conversational-Experiences-with-Actions-on-Google-32-320.jpg)
![<speak> <!-- Must be at the start of the string -->
<say-as interpret-as="characters">SSML</say-as>
<break time="3s"/>.
<audio src="https://example.com/file.mp3"></audio>
<say-as interpret-as="cardinal">10</say-as>.
<say-as interpret-as="ordinal">10</say-as>
<say-as interpret-as="characters">10</say-as>.
<sub alias="World Wide Web">WWW</sub>.
<p><s>This is one.</s><s>This is two.</s></p>
</speak> <!-- Must be at the end of the string -->
“S S M L”
[3 second pause]
[audio file plays]
“Ten”
“Tenth”
“One Oh”
World Wide Web
[two sentences]](https://image.slidesharecdn.com/actionsongoogle-buildingconversationalexperienceswithactionsongoogle-170713155850/85/Building-Conversational-Experiences-with-Actions-on-Google-41-320.jpg)
![function suggestionChips (app) {
app.ask(app.buildRichResponse()
.addSimpleResponse({
speech: 'Howdy! I can tell you fun facts about ' +
'almost any number like 0, 42, or 100. What number do you have ' +
'in mind?',
displayText: 'Howdy! I can tell you fun facts about almost any ' +
'number. What number do you have in mind?'
})
.addSuggestions(['0', '42', '100', 'Never mind'])
);
}](https://image.slidesharecdn.com/actionsongoogle-buildingconversationalexperienceswithactionsongoogle-170713155850/85/Building-Conversational-Experiences-with-Actions-on-Google-46-320.jpg)
Ad
More Related Content
Similar to Building Conversational Experiences with Actions on Google (20)
More from Peter Friese (20)
Ad
Recently uploaded (20)
Ad
Building Conversational Experiences with Actions on Google
- 2. To organize the world's information and make it universally accessible and useful.
- 4. Google Home/Mobile device The Google Assistant — Actions on Google —
- 7. Well, it’s kind of cold outside, so I'd like something to warm me up, like a hot soup, and I want it fast. I have some chicken, and also canned tomatoes.
- 9. Assistant app { conversation API request } { conversation API response } user input app response
- 11. Well, it’s kind of cold outside, so I'd like something to warm me up, like a hot soup, and I want it fast. I have some chicken, and also canned tomatoes.
- 12. “ Intent Matching — Entity Extraction — @
- 29. const App = require('actions-on-google').ApiAiApp; exports.yourApp = (request, response) => { const app = new App({request, response}); console.log('Request headers: ' + JSON.stringify(request.headers)); console.log('Request body: ' + JSON.stringify(request.body)); // Fulfill action business logic function responseHandler (app) { // Complete your fulfillment logic and send a response app.ask('Hello, World!'); } const actionMap = new Map(); actionMap.set('<API.AI_action_name>', responseHandler); app.handleRequest(actionMap); };
- 30. const app = new ApiAiApp({request: request, response: response}); const WELCOME_INTENT = 'input.welcome'; // the action name from the API.AI intent const NUMBER_INTENT = 'input.number'; // the action name from the API.AI intent const NUMBER_ARGUMENT = 'input.mynum'; // the action name from the API.AI intent function welcomeIntent (app) { app.ask('Welcome to action snippets! Say a number.'); } function numberIntent (app) { let number = app.getArgument(NUMBER_ARGUMENT); app.tell('You said ' + number); } let actionMap = new Map(); actionMap.set(WELCOME_INTENT, welcomeIntent); actionMap.set(NUMBER_INTENT, numberIntent); app.handleRequest(actionMap);
- 32. if (app.hasSurfaceCapability(app.SurfaceCapabilities.SCREEN_OUTPUT)) { app.ask(app.buildRichResponse() .addSimpleResponse(`Here's a fact for you. ${fact} Which one ` + `do you want to hear about next, Google's history or headquarters?`) .addBasicCard( app.buildBasicCard('Google is an amazing company.') .setImage(GOOGLE_LOGO_SRC)) .addSuggestions(['History', 'Headquarters'])); } else { app.ask(`Here's a fact for you. ${fact} Which one ` + `do you want to hear about next, Google's history or headquarters?`); }
- 40. Hey! This is ___ Welcome to ___ Ready to play ____ Hi! ___ here. Hello. I’m ___ Greetings, human. Welcome back to ___ Hey again. ___ Let’s play ___ Here’s your ___ Brought to you by ___ Hi there, ___ Let’s get started. Ready for your ___ ___, here to… Live from ___ This is ___ What’s up, ___ and more...
- 41. <speak> <!-- Must be at the start of the string --> <say-as interpret-as="characters">SSML</say-as> <break time="3s"/>. <audio src="https://example.com/file.mp3"></audio> <say-as interpret-as="cardinal">10</say-as>. <say-as interpret-as="ordinal">10</say-as> <say-as interpret-as="characters">10</say-as>. <sub alias="World Wide Web">WWW</sub>. <p><s>This is one.</s><s>This is two.</s></p> </speak> <!-- Must be at the end of the string --> “S S M L” [3 second pause] [audio file plays] “Ten” “Tenth” “One Oh” World Wide Web [two sentences]
- 42. function welcome (app) { return isPreviousUser(app.getUser().userId).then((userHasVisited) => { if (userHasVisited) { app.ask(`Welcome to Number Genie!...`, NO_INPUT_PROMPTS); } else { app.ask(`Hey you're back...`, NO_INPUT_PROMPTS); } }); }
- 45. function simpleResponse (app) { app.ask({ speech: 'Howdy! I can tell you fun facts about ' + 'almost any number, like 42. What do you have in mind?', displayText: 'Howdy! I can tell you fun facts about ' + 'almost any number. What do you have in mind?' }); }
- 46. function suggestionChips (app) { app.ask(app.buildRichResponse() .addSimpleResponse({ speech: 'Howdy! I can tell you fun facts about ' + 'almost any number like 0, 42, or 100. What number do you have ' + 'in mind?', displayText: 'Howdy! I can tell you fun facts about almost any ' + 'number. What number do you have in mind?' }) .addSuggestions(['0', '42', '100', 'Never mind']) ); }
- 47. function basicCard (app) { app.ask(app.buildRichResponse() .addSimpleResponse('Math and prime numbers it is!') .addBasicCard( app.buildBasicCard(`42 is an even composite number. It ` + `is composed of three distinct prime numbers multiplied together. It ` + `has a total of eight divisors. 42 is an abundant number, because the ` + `sum of its proper divisors 54 is greater than itself. To count from ` + `1 to 42 would take you about twenty-one…`) .setTitle('Math & prime numbers') .addButton('Read more') .setImage('https://example.google.com/42.png', 'Image alternate text') ) ); }
- 50. let permission = app.SupportedPermissions.DEVICE_COARSE_LOCATION; app.askForPermission('To find bookstores near you', permission);
- 55. Ok Google, ask Personal Chef for a hot soup recipe Trigger Phrase App Name Developer Specified Action Preposition Action Phrase Developer Specified Ok Google, let’s talk to Personal Chef Trigger Phrase App Name Developer Specified
- 61. GUI :: VUI What’s the difference?
- 62. Top 3 Design Tips
- 64. Persona is conveyed through: Tone Word and phrase choices Functional design Style Technique Voice ...and based on: Your user population and their needs The imagery & qualities associated with your brand
- 65. Think outside the box. Literally.2
- 66. Confidential and proprietary code code code code code code code code code code code “ dialog string” code code code code code code code code code code “dialog string” code code code code code code code code code code code code code code code code code code code code code code code “dialog string” code code code code code code code code code code code code code code code code code code code code code code code code code STRUCTURE + CODE
- 67. Confidential and proprietary code code code code code code code code code code code “ dialog string” code code code code code code code code code code “dialog string” code code code code code code code code code code code code code code code code code code code code code code code “dialog string” code code code code code code code code code code code code code code code code code code code code code code code code code STRUCTURE + CODE
- 68. Confidential and proprietary DIALOG + STRUCTURE
- 69. In conversations, there are no “errors”.3
- 71. “I don’t understand you”
- 72. When a so-called “error” occurs in a conversation, it should be treated simply as a new turn in the dialog, only with different conditions.
- 77. goo.gl/PMdr7N