How to Secure a SuperApp

By Amol Naik

Building a SuperApp, with the equivalent of multiple apps in one, is a challenge in many ways. Managing security for a SuperApp? That’s a whole other ballgame. Add a few acquisitions along the way, and the complexity increases even further.

This is the story of Gojek’s Product Security team, and our efforts to maintain a world-class security program for a SuperApp.

Flashback time

The Gojek app was launched in 2015. In those early years, the app was rather weak from a security standpoint — to the extent that many blog posts were published highlighting security loopholes in the app. You can read some of them here (requires translation from Bahasa Indonesia) and here. You can read our Group CTO Ajey Gore’s response to some of these issues in this post. 👇

Security — Vulnerabilities lifecycle at GoJek - 3 min read

null

Gojek3 min read | Mar 31, 2017Share:

Needless to say, things weren’t in great shape.

To counter these problems, a three-member Security team was put together in March 2016. The objective of this team was to secure the mobile app and to provide security services for upcoming products/services.

Test ! Test ! Test !

The first step was to secure the existing app. All hands on deck. We spent most of our time testing the application, finding technical and logical security bugs, and getting them patched. Some major bug classes were:

• Authentication Issues — Ex: APIs can be accessed without login. Such APIs were targets for scripted attacks
• Authorisation issues — Ex: One customer can view other customers’ order history (which includes sensitive details like address, phone number, etc.)
• Race Conditions — Ex: While transferring money via GoPay, if multiple requests get sent to the server at the same time, the sender will be debited a lower amount and the receiver gains extra credit (magic money 😬)

This was also a time when Gojek was scaling rapidly, and just like the other engineering teams, this took a toll on us too. As we kept adding more resources daily, the need for Infrastructure Security arose in parallel.

Setting up vulnerability scanning capabilities with manual validation and patching them with the help of the DevOps team was a challenge. We started with a static list of public assets to be used by scanners and planned to automate the solution for discovery of assets to be targeted by scanner for vulnerability management.

Becoming the Blocker

Gojek was adding new products and features at a breakneck pace, with more in the pipeline — like GoBills, GoPoints and new GoPay services.

Along with exponential growth, the need to provide security services for upcoming products/services grew as well. We observed a need to have a manual testing phase in the release cycle for the products/services. We managed this by controlling the API gateway.

An API gateway is a network component responsible for allowing Mobile API (endpoint) to be accessed by the consumer from the app.

We established the security policy for API clearance, which required every API to obtain approval from the Security team before it could go live. As we were managing the API Gateway, it was a mandate for product team to fulfil the security requirements. The Security team would then test the API manually with business context, report bugs, retest when bugs were patched, and publish the API at gateway only when everything checked out.

This process ensured that every API which went live was free of security bugs and worked. However this process had some disadvantages:

• As the security testing was being carried out at the end of product development, the cost of patching bugs & changing product architecture at the last moment was very high.
• Security team was becoming a blocker for product releases.

The resultant cost to the product is something we have experienced first hand, especially instances where we have to block a product/feature release.

One example of this was implementing ‘Login with Facebook’ integration in the Gojek app.

During secure design review, we missed an architectural flaw which could result in hijacking of customer accounts. It was only when we tested the product at the end that we found this flow. At a time when we were still fighting Login OTP scams, this flaw could have cost more customer account hijacks.

Keeping the severity of the issue in mind, we decided to stop the production release of this feature. We talked to the product team, demonstrated the attack, and discussed the impact. After multiple discussions, product teams were in agreement about the architecture flaw and agreed to stop the production release. Even with the combined efforts of the security team and product team, it took close to two months to redesign the product, develop as per requirements and complete security testing for the production release.

We had learned our lesson.

More Agile, More Automation

At the end of 2017, we started with a new security process based on the SDLC approach. We call it PSDR (Product Security Design Review). With PSDR, we started introducing security at early stages of product engineering, mainly the design stage. Here are a few things we ask from Product teams for this review:

• Product design documents
• API contract documents
• Data flow diagram

With this, the Security team gets better visibility on the product and recommends security practices. This eliminates the need for architecture changes at the end, and identifies a list of probable security issues, which product teams are encouraged to mitigate while developing the product. This also solved a lot of complications in 3rd party integrations, where the Security team plays an important role in suggesting secure ways for communication, data sharing models, etc.

In the near future, we will work towards enhancing the above model, where security guidelines will be prepared as per the tech & nature of the application. The idea is to have a pre-compiled list of recommendations for each application type — based on our learnings over a long period of time.

We also started building a Security Pipeline which helps in solving the following problems:

• Hardcoded credentials in code
• Security bugs in code
• Vulnerable Code dependency
• Automated API/Web scanning
• Automated API whitelisting

Just to be clear, this pipeline does not take away manual testing. Manual testing will always be needed to identify bugs which are hard to detect using automation, like authorisation bugs. However, this pipeline raises the security confidence of a product which can be used in production without testing. This reduces time to production for services. Manual testing can be scheduled later in time.

This is also where our Bug Bounty program comes in.

An Extended Security Team

In early 2017, we started a private bounty program with BugCrowd. The bug bounty program has been helpful for us in maintaining the security of our public assets i.e business app, infrastructure, 3rd party integrations, etc.

In this program, BugCrowd invited top researchers from their platform to test our app and report the bugs in it. This was an eye-opening experience where we learned about new bugs and their testing methodologies. One such bug was reported in a Swagger UI. We identified XSS in Swagger and asked the product team to update the gem (if it was being used) or remove it. The Product team decided not to use the Swagger UI. As a result, the affected code was deleted, but the gem was still there. The security researcher found the Local file read using this vulnerable version of Swagger UI, and reported via BugCrowd. This was a critical issue for us and we rewarded the highest bounty for this.

Being an internal Product Security team, it’s not always possible to address all sorts of issues within limited testing time, and this program helped us uncover those issues.

After a year of running private bug bounty, we made it public in February 2018. The response to the program was very good and researchers from around the world participated in it.

Here are a few stats from our bounty program so far:

Private Bug Bounty:

Total Reward: USD 8,000

Public Bug Bounty:

Total Reward: USD 122,500

The Way Ahead

Being responsible for all security-related concerns in Gojek, we are building capabilities in Corporate Security to serve those needs. Things like Endpoint protections, Mobile Device Managements, centralised employee onboarding and offboarding, and Incident Response policies have shaped our enterprise security program.

On product security, our goal is to develop a self-serve security pipeline which will automate most elements of manual testing. Secure infrastructure provisioning is another major focus area for us.

As Gojek has grown, our initial 3 member team has expanded to 10. We now assign multiple products to each security engineer so that they get detailed context of the product and start getting involved with dev teams more. This works as security evangelism and promotes transparent security processes.

One of the major skills a product security engineer must have is building relationships with dev teams. This fosters mutual understanding and helps establish realistic timelines. The win for us is when Product Managers and devs discuss abuse scenarios during product inception.

It’s been a long journey, but today we have much more confidence in the security protocols watching over the Gojek SuperApp. If there’s one takeaway we have from years of doing this job, it’ll be this:

“Experiment, and do not fear failure. If you fail, learn, and move on.”