Raiyan Sayeed

another immigrant

Making a Google Action app

2018-11-15programming

The Google Assistant is powering more devices than there are people in the US and Canada combined. Actions are at the heart and soul of this revolutionary AI service, letting ordinary developers extend it vast array of functionalities. With some free time on my hands, I decided to play around with Google Actions and the hot new chatbot framework Dialogflow. Like most of my projects, I wanted to make something that appealed to everybody. It was the beginning of …

Inspiration Nation

Corny, I know. But it was a good way of testing the different features of Dialogflow. I looked around the dashboard but eventually had to watch a Google I/O video. The graphical interface wasn’t very helpful as it was hard to make the app have complex logic to make the dynamic features I wanted. This meant that I had to resort to… programming (dun dun dun).

The app would work as such:

  • The user says “inspire me” and the action would play an inspirational clip
  • The user says “repeat” and it will repeat the last clip played
  • The user says “help” and all the available commands would be said.

Before I could start anything, I needed some data. I checked their Knowledge tab (which was in beta at the time of writing) but it felt like more work when compared to making a simple object. It didn’t seem like Dialogflow could hold very big data structures so I had to go over to the fulfillment tab to work some object-magic. I created a JSON object to store my clip properties like ‘url’ and ‘author’

const functions = require('firebase-functions');
const {dialogflow} = require('actions-on-google');
const {
  Image,
  SimpleResponse,
  BasicCard,
  Suggestions,
  Button,
  Permission,
} = require('actions-on-google');

const list = [
    {
        name: "Rocky Balboa",
        url: "<audio src='[URL]' />",
        type: 'Life',
        author: 'sylvester stallone'

    },
    {
        name: "Pursuit of Happiness",
        url: "<audio src='[URL]'",
        type: "inspiration",
        author: 'will smith'
    },
    {
      name: "his Stay Hungry Talk",
      url: "audio src='[URL]'",
      type: "persistence",
      author: "tony robbins"
    }
  ]

Next I had to create the intents which map out user inputs to specified outputs. The main ones were ‘Ask’, ‘Help’, ‘Repeat’, and ‘Specific’ (the cool features were added later). The intents with dashes were follow-up intents which are intents that build off the info of the original intent (like an inherited class or Props from ReactJS). In this case, they were asking if the user wanted to listen to the same clip again.

const ASK = "Ask";
const SPECIFIC = "Specific";

app.intent(ASK, (conv) => {
    let Awe = awe[Math.floor(Math.random() * awe.length)];
let Random = list[Math.floor(Math.random() * list.length)];
    let Clip = list[Math.floor(Math.random() * list.length)];
    //used SSML (Speech Synthesis Markup Language) instead of normal string for more flexibility if I needed it.
    //Assistant automatically plays <audio> tags
    Output = "<speak> Here's a clip by " + Clip.author + " from " + Clip.name + Clip.url + ". Let's get inspired again! Say inspire me. </speak>";
    conv.close(Output);
});

app.intent(SPECIFIC, (conv) => {
    //Using entities. More on that below.
    const ClipType = conv.parameters[GENRE].toLowerCase();
    let Random = list[Math.floor(Math.random() * list.length)];

        let Clip = list.find(function(element) {
            return element.type == ClipType;
        });

        Output = "<speak> Here's a " + ClipType + " clip from " + Clip.name + Clip.url + "Try another type of clip like " + Random.type + "</speak>"

        conv.close(Output);
});

app.intent(ASK_REPEAT, (conv) => {
  //I could access this variable because variables from the parent intent are passed into the followup
    conv.ask(Output);
});

Next, I had to update the intents on the GUI interface to make sure they used the fulfillment code instead of the default Dialogflow handlers.

drawing

I also needed to remind the user of how to use the app but that seemed like a menial task…something perfect for the Dialogflow GUI!

drawing drawing

Since I was using dynamic data within the same slot (a spot in the sentence where different instances of the same type of data could occur). In my case, it was the tag name. Therefore I needed to make use of entities.

drawing

And there’s the barebones structure of the app! Now to test it out on the Dialogflow simulator…

Dialogflow testing image

Help command image

Repeat command

Seems like it’s working. But to make sure it’s working on our end client, I head over to the Actions simulator.

Actions simulator image

Hurrah! Everything works! And another inspirational app done! I hope this inspires you to test out Dialogflow and voice apps for yourself

You can read more articles like this over at my blog. From philosophical musings to programming projects, it has your regular dose of variety.