By Vikas Bajpayee and Lokesh Kalal

Our aim behind building a Super App was to help our users get rid of the daily frictions in their lives. But the journey doesn’t stop there. Even after building all these convenient products, we still try and reduce frictions that may be encountered even within our app — such as booking a ride or ordering food.

So we thought, why don’t we integrate with Google Assistant, and let our users communicate with our products in a new way?

In this post, we explain how we integrated our food and transport products with the Google Assistant.

The why

We’ve already addressed one part of the why — convenience. The other is user re-engagement. According to this survey conducted by Silicon Valley researcher Andrew Chen 77% of users stop using an app three days after they install it. Within a month, that number goes up to 90%. ?

The how

To help integrate Android apps with the Google Assistant, Google launched App Actions:

What are app actions?

App Actions are a new way to make your android app content available in places like Google Search, Google Assistant etc. From a user’s perspective, App Actions behave like shortcuts to parts of your Android app. When users invoke an App Action, the Assistant either launches a screen in the Android app that the user has already installed or shows an embedded visual card (Android Slice) that users can interact with.

App actions provide a faster way for users to access Android apps. It can be achieved in two ways:- either user can be directed to a specific activity(screen) using a deep link or user may see relevant content on the Assistant itself called Slices.

More info on this can be found here: https://developers.google.com/assistant/app/overview

What are Slices?

Slices are super-powered app actions. They provide a way to interact with apps without moving away from Assistant by showing a small piece of UI within Assistant. Slices are UI templates that can display rich, dynamic, and interactive content from the app in Google Assistant.

Slices can help users perform tasks faster by enabling engagement outside of the fullscreen app experience.

You can build Slices as enhancements to App Actions.

Here’s an example:

We choose to use slices to show food order status, as the order status will have limited information (eg. Driver is on the way to Restaurant, with his location on Map ) which can be easily shown in a small piece of UI segment.

In fact, this functionality was recently demoed on stage at a Google event in Indonesia. You can check it out below:

How does it work?

Whenever a user says or types something in Assistant, assistant parses the query. If the query matches the built-in intent grammar then Google Assistant extracts the query parameter in schema.org and generates a deep link URL using the mapping we provide in an actions.xml file.

Google then attaches those extracted parameters to the deep link URL and creates a final deep link which allows a user to launch specific content or screens in the app.

Let’s dig a little deeper:

Google applies Machine Learning and Artificial Intelligence (Natural Language Processing) to understand all the sentences we type in Assistant.

While users can type anything they want in Assistant and the system gets better at interpreting them over time, but there are some sentences for specific intent that are fixed by Google. More info here.

How did we integrate our services with Assistant?

The core of integrating Gojek with Assistant involved creating actions on Google Assistant with actions.xml file.

Let’s explore how to do this:

1. Create actions.xml file in your XML folder. This contains all the actions with built-in intents which define what actions our app can support in the Assistant. Each action contains built-in intents supported by Google as a <intent-name> tag.

2. Define a fulfilment mode for each app action — which can be deep links or slices. You have to pass a fulfilment mode in each action. This is compulsory.

3. Pass the URL template which will allow a user to launch your app — this can be the deep link to your app or URL to Slice provider.

4. Define entity-set — the Gojek app is available in multiple locales, so to provide locale for all the regions, we added the entity-set, which allows Assistant to understand more than one version of category. ex:- taxi can be pronounced as taksi in Bahasa.

So, If you want locale for your specific parameter in deep-link URL, you can simply attach an entity-set with that parameter, see how you can achieve this in below example

<parameter name=”taxiReservation.category”><entity-set-reference entitySetId=”TransportCategoryEntitySet” /></parameter>

5. Add a reference to actions.xml file using the <meta> tag in your app’s manifest.xml file inside the application tag.

In order to integrate Slices:

1. Implement an Android Slice by following the steps described in the Slices Getting Started guide.
2. In the actions.xml file, in the <fulfillment> element, specify fulfillmentMode=“actions.fulfillment.SLICE” to indicate that a user intent can be fulfilled using a Slice in your app.

A few quick notes on what we learnt

1. Testing this feature is pretty hard and things become weird when you have multiple build types of your app. For testing, you need a build type on the Play Store, (which means the package name of the testing build should match the package name of the published app). You can use a plugin (developed by Google) called App Actions Test Tool for testing this feature.

Please make sure you are following points while testing with App Actions Test Tool:
a) Log in to Android Studio and Google Assistant with the account which has access to your published app on Play console.
b) ApplicationId of the app should be the same as your published app.
c) Your Gradle should be built successfully.
d) Use App Action Tool to create and update preview and test the app actions and slice.

2. If your app is used in different locales, you can create locales using entity-set in the actions.xml file.

3. In order to give early builds to QA or other users (if you have multiple build types in your app), you can create an internal test track on Play console and add relevant people to it. This allows you to share the build to production for only some listed users.

(See this for how you can create an internal test track for your app.)

4. Actions.xml shouldn’t be obfuscated in release apk — it means if you are using any obfuscated tools like proguard then actions.xml shouldn’t be obfuscated there.

5. Slices can be requested before the onCreate of your application is completed. You can run into issues if you are using something which is initialised in onCreate while creating slices.

6. Don't refresh slices from onBindSlice() method, it will end up in an infinite loop where Google Assistant will call onBindSlice() infinitely.

That’s all from us folks. Hope this post gave you a better understanding of how App Actions and Slices work with Google Assistant. Working on this feature was an amazing experience for us. If you use the Gojek app, make sure to give them a try and let us know what you think. ?

Liked what you read? Sign up for our newsletter and we’ll send you updates from the blog straight to your inbox! ?