Building A Messaging Highway For Our iOS Apps

By Alfian Losari

Here is a summary of the previous posts so far in the series:

• Introduction of Courier — An information superhighway between Mobile and Server.
• Courier Message Broker — Benchmark of various brokers and reasoning behind choosing VerneMQ.
• Courier Android Library — Discussed challenges in Android and background to provide clean API, multi message adapters support, adaptive keep alive, and many more!

We’ve already rolled out Courier for our Android Merchant app to send order updates with high reliability and efficiency. We also intend to utilise Courier in the Gojek consumer app to improve user experience.

Introduction

Every time an order is made on GoFood, customers see an Active Order Screen, which looks like this:

The screen shows information such as driver location on the map, a timeline update of the order status, etc. The timeline starts from restaurant order acceptance until the food is delivered by the driver to the customer’s location.

We believe this is the best starting point for us to utilise Courier in Gojek consumer app as we were using HTTP pooling to get the latest food order status update. This was not efficient in terms of power and data consumption. We’re passionate about delivering the best experience, while also ensuring our app’s energy and data consumption is as efficient as possible.

As we started rolling out Courier to Gojek consumer App, we also needed to build the Courier library for iOS which provided the same set of features and API to the equivalent Courier Android library.

Background of Courier iOS Library

The Courier iOS library high level architecture is designed with several goals in minds:

1. Provide an abstract and simple interface for the client to subscribe to topics, receive stream of messages based on the topics subscribed, and publish messages to topics with any QoS.
2. The communication protocol should be agnostic and hidden from the client. With this, we should be able to switch to any communication protocol without much change from the client. (MQTT, WebSocket, etc).
3. The library used should be agnostic, so we can switch to any library without much effort (CocoaMQTT, MQTTClientFramework, etc).

With these goals in mind, we decided to build Courier iOS library instead of just using a MQTT library directly.

MQTT Libraries on iOS

There are 2 most used open source libraries that we can use to integrate MQTT:

1. MQTT-Client-Framework is a native Objective-C iOS library. It uses CFNetwork for networking and CoreDatafor persistence. It is a complete implementation of MQTT 3.1.1 and supports TLS.
2. CocoaMQTT is a MQTT v3.1.1 client library for iOS written with Swift 5. It uses CocoaAsyncSocket for networking and currently has no disk persistence support as of v1.2.5.

We have explored both libraries and created the comparative analysis table for both libraries below.

On our initial experiment, we decided to use CocoaMQTT on Courier; since it is written on Swift, most of the APIs seems pretty modern and Swifty compared to MQTTClientFramework which uses native Objective-C.

It served us pretty well without any issues for receiving messages from broker. But, as our future use cases will require the app to also publish message to broker, the requirement to support disk persistence and offline message bufferingbecame super important. This will ensure us to persist message sent while the user has poor or no internet connection and send it when the user is reconnected.

As of version 1.25, CocoaMQTT doesn’t have this feature, so instead of reinventing the wheel and wasting resources, we decided to switch to use MQTTClientFramework which provide full QoS 1 & 2 disk persistence and offline message buffering. Based on our testing, it worked really well for both publishing and receiving messages.

Courier iOS Library Features

Courier is not just a simple wrapper on top of MQTTClientFramework. It also provides several main features to increase robustness and reliability for the iOS client app to communicate with the broker. Let me walk you through of all the main features of iOS Courier library.

Clean and Reactive API

Courier provides clean and simple API to connect, subscribe, and receive stream of messages from subscribed topic. With the nature of persistent realtime long run connection that process messages asynchronously, we designed Courier API to be reactive using Combine Publisher and Subscriber pattern.

// Method to get publisher for a subscribed topic
func messagePublisher<D>(topic: String) -> AnyPublisher<D, Never>
// Usage (type of model is generic using type inference)
courier.subscribe(topic: topic, qos: qos)
courier.messagePublisher(topic: topic)
.sink { [weak self] (message: Message) in
   self?.handleMessageReceiveEvent(.success(message))
}
.store(in: &cancellables)

Multi Message Adapter Support

Courier provides multi message adapters that supports various serialisation formats such as JSON, Protobuf, XML, or anything else that can be serialised into data and deserialised into model.

import SwiftProtobuf
class ProtobufMessageAdapter: MessageAdapter {
   
    func fromMessage<T>(_ message: Data) throws -> T {
        // Deserialize to Protobuf Message
    }
    func toMessage<T>(data: T) throws -> Data {
        //Serialize Protobuf Message to Data
    }
}

Subscription Store

Courier provides subscription store to manage currently subscribed topics and pending unsubscribe topics. This is super useful to avoid stale subscriptions in broker when the user connects with clean session flag set to false.

Fallback Policies

Courier provides several built in fallback policies for various Courier lifecycle system events (connection loss, connection failure, subscribe failure, and many more). User can react and provide their fallback logic in case of failure (for example fallback to HTTP polling). The user can also implements their own custom fallback policy and react to various Courier system events accordingly.

Database Persistence and Offline Message Buffer

MQTTClientFramework provides this feature to persist QoS 1 and QoS 2 messages in case the user is offline when sending those messages. It will retry to send the messages maintaining the order when the internet connection is back.

Automatic Reconnect

Courier provides the reconnect mechanism to handle the case where the internet connection is lost and whether it should retry to connect when the internet connection is reachable.

Custom Authentication Provider

In case, the user needs to be authenticated when connecting to the broker using username and password. Courier provides API to delegate the authentication handling from the user side. The user should be able to implement their own custom authentication to get the username and password used for making connection to the broker.

Event Provider

The user can optionally listen to Courier system events such as connection success, connection lost, message received from a topic, and many more). This is pretty useful in case they want to add analytics and metrics for their use case.

Conclusion

Courier is our solution to provide a realtime, lightweight, and high efficient messaging highway between mobile apps and server using MQTT.

With Courier at Gojek, we have just started scratching the surface of many possible future use cases that we are planning to implement such as:

1. Chatting.
2. Realtime app metrics & analytics.
3. Any realtime bi-directional data sharing (e.g. collaborative cart sharing, etc.)

Thanks for reading and stay tuned for our upcoming articles!

Click here to read more stories about how we do what we do.

And we’re hiring! Check out the link below: