How We Solved Real-Time Image Processing at Gojek
By Ajat Prabha
Here’s a fun fact. The GoFood ecosystem is made up of over 300,000 merchant partners, who list more than 16 million menu items between them. That’s a lot of content to display in an app. 😅
Helping us achieve this goal is our in-house real-time image processing application — Darkroom. Thanks to Gojek’s scale, Darkroom is used to serve 500,000 images/minute with real-time manipulation of images. You can read more about what it does and how it was built in this post.
As mentioned in that post, we moved GoFood away from using a third-party service for its image requirements, but other services inside Gojek were still using that third-party service for their requirements. The next task was to make Darkroom available as another service to cater to image needs across Gojek.
Now that Darkroom was mature enough to handle GoFood’s scale, we wanted to onboard other teams as well onto the service and deprecate the third-party service completely, once and for all.
That didn’t come without challenges for us, we had to make sure that Darkroom was able to handle ever-increasing load and also ensure the service is able to produce essential logs and metrics based on different applications using Darkroom as a service.
Why Darkroom as a SaaS?
By getting rid of dependancies and switching to Darkroom for GoFood, we saved thousands of dollars per month. However, as Darkroom was built keeping GoFood’s needs in mind, other teams were still going back to the aforementioned service. We needed a way to allow other services to specify their configurations and rely on Darkroom.
We added a feature in Darkroom to register different image sources and provide specific routes for those image sources. Now, other services can start using Darkroom without being constrained by the GoFood flow.
Architecture
The flow is pretty simple, you register your image source with Darkroom and in return get a host which can be used to access your images along with all the image processing capabilities that Darkroom can perform.
Challenges
We faced quite a few challenges but one of the prominent ones was:
“How will we monitor if everything is working fine for them?”
Metrics analysis, proper logging, and reliable alerting is something that you just can’t afford to ignore in a service. The end result was a proper Grafana dashboard with metrics coming through on a per application and per request basis. This approach worked for us.
We love open source 💚
Darkroom turned out great and now we wanted to give back to the community. We pulled out the essential parts of the service, made them modular enough to be used quite flexibly and pushed them to a public repository on GitHub. You can find it here:
The project has first-class support for Docker images and the service itself is stateless, it is as easy as running the docker image with a configuration file to get Darkroom up and running.
Users can also implement their own Storage Backends and Image Processors to gain custom functionality. Examples can be adding support for Google Cloud Storage bucket as the image source and use a GPU accelerated image processor to achieve even shorter response times. The following interfaces can be implemented to gain custom functionality:
- Storage interface to interact with your image source
2. Image Manipulator interface holds the logic to process the image according to the ProcessSpec.
Darkroom was a labour of love for us, and we hope you like it. Head over to the GitHub and contribute to the project if you find this interesting. Feedback is also welcome.
BTW, if you’d like weekly updates on our stories, sign up for our newsletter.