Anatomy and mindset of the data army at GO-JEK
Stories from the land of data at GO-JEK
By Ravi Suhag
At GO-JEK, we build products that help millions of Indonesians commute, shop, eat and pay, daily. The Data Engineering team is responsible to create a reliable data infrastructure across all of GO-JEK’s 18+ products.
Mission
We aim to provide disproportionately large advantages to GO-JEK over its competitors by making data available, accessible, reliable and actionable at scale.
The GO-DATA platform allows internal teams to build on our products to develop innovative solutions; creating new opportunities for our customers and expanding on what’s possible.
Approach
Some informal philosophies and approaches we follow during our time working on data infrastructure:
Scale
Data at GO-JEK doesn’t grow linearly with the business, but exponentially, as people start building new products and logging new activities on top of the growth of the business. We currently see 6+ Billion events daily and rising.
Automation
Working at such a large scale makes it really important to automate everything from deployment to infrastructure. This way we can push features faster without causing chaos and disruption to the production environment.
Product mindset
We operate as an internal product organization. We measure success with business metrics like user adoption, retention, and the revenue, or cost savings generated per feature. Our customers are Product Managers, Developers, Data Scientists, and Analysts.
4 min read | Apr 30, 2018Share:
5 min read | Mar 12, 2018Share:
Anatomy
We are a team of full-stack, opinionated, polyglot engineers and passionate human beings who believe the core of a developer’s job is not just writing code.
We own the full product cycle and are equally responsible for adoption and experience as our product manager. Having product ownership gives everyone a responsibility to put their best and gives meaning to our daily work.
3 min read | Oct 01, 2017Share:
3 min read | Dec 26, 2017Share:
Work
As data engineers, we solve problems with data to improve the product that we offer to our users.
We build tools, infrastructure, frameworks, and services and on the way develop new skills, new ways of doing things, new tools, and turn our backs to traditional methods. At GO-JEK, data engineering is much closer to software engineering and we understand the tradeoffs between different open source Big Data technologies and aren’t afraid to build it ourselves if needed.
4 min read | Feb 19, 2018Share:
4 min read | Apr 02, 2018Share:
As a member of the data team, you’ll have immense impact — shaping the data with which the company makes innumerable business decisions.
Tools
Given the real-time nature of the business at GO-JEK, the entire data infrastructure from the ground up focuses on real-time data processing, unlike traditional batch processing architectures. Follow the below post for a simplified view of our data architecture and tools.
5 min read | Mar 12, 2018Share:
4 min read | Jul 09, 2018Share:
6 min read | Oct 03, 2018Share:
We also rely on open-source and are thankful to the collaborative data engineering community for inventing new data technologies and open-source it. Our work wouldn’t be possible without the foundational work of so many engineering teams before us.
If you like what you’re reading and interested in building large-scale infrastructure that excites you, do check out our engineering openings at gojek.jobs. As always, I would love to hear what you guys think.
