Dev Box — Where The Magic Happens

How we implement Dev Box testing and the process that follows.

Dev Box — Where The Magic Happens

By Chinmay Karadkar

Consider this (hypothetical?) scenario:

You’ve just reached home after a long day, and your driver partner is waiting to get paid. At that moment, you decide to go scouring inside your bag.

First you reach the notepad of musings (the one that knows your good and bad days). 📒
Right next to that, you meet the pen that’s used to write the said musings (which is running out of ink, btw). 🖋
To its right is an umbrella (why is it even there? It hasn’t rained in 2 months!). 🌂
A little ahead, is the hand-sanitiser bottle (which runs out quicker than you expected it to). 🧴
Mask #2 is also there (because #1 is on your face and it’s better to have a spare). 😷
Finally, you reach your wallet (where, legend has it, money would be stored). 💰

Now the driver partner is wondering what’s taking you so long to pay them.

This. This is what GoPay effortlessly avoids. More than 50% of the Gojek Super App’s transactions are processed through GoPay, and needless to say, it’s a pretty big deal — for us, and for everyone in our ecosystem. Moreover, when a product falls in the finance domain, the stakes are high.

So, how does Gojek keep up with customer expectations around product reliability, experience, and more?

The straightforward answer would be: Build a quality product.

This is obviously true. But there are multiple layers to a seemingly simple answer. We collaborate with a quality-first approach in the Gojek Financial Services team. The teams follow the Test Driven Development (TDD) and an integral part of this process is Dev Box Testing.

What is Dev Box Testing? 🧐

Dev Box is an abridged version of ‘Developer Machine’. In Dev Box testing, a quality engineer validates, tests, and verifies a feature in scope on the developer’s machine. It’s as simple as that. A basic approach through which one ensures the quality of a feature.

The process of successful Dev Box testing starts early in the development cycle, when the feature is being discussed between the Product, Dev, and QA teams. While a story is being groomed, a quality engineer ensures the following are in place:

  • Acceptance criteria (acts as verification points for the test)
  • In scope and out of scope features (sets the scope of testing)
  • Pre-conditions, if any

Once the above parameters are concluded, the dev team can start working on it. After the development is finished, the SDE will call out for dev box testing.

How to carry out Dev Box Testing? 🛠

The primary goal of the process is to verify the acceptance criteria and ask questions. Lots of them.

There are no official guidelines or a defined process on how to proceed with dev box testing.

The steps that we follow are:

  1. The quality engineer reviews the story and checks the preconditions (databases, user states, service calls, etc)
  2. Ask the developer to show the workflow to ensure the required functionality is working
  3. Review the unit, functional, integration tests (written as part of the TDD approach) and execute them
  4. Check the boundary conditions and test the functionality against various data sets
  5. Ask questions to the developer
    - Why do they think this would work in production?
    - On a scale of 1–10 how confident are they about this working in production?
    - If it did, when would this fail in production? What would be possible scenarios?
    - Would this perform as expected under performance-stress-load tests?
    - Are there any known issues with the code, module, or integrations?
    - What edge cases have they handled in the implementation?
    - Is there a better way to implement the same code?
    - Are there any platform or hardware constraints?
    - Questions around scenarios of timezone, localisation, etc.

The questions would depend on what type of story you’re testing, for example: front-end or back-end stories.

While going through the above points, note the details such as discussion points, issues found, suggested changes, action items, and share them as minutes of meeting.

This would ensure the required updates are made.

If no issues are found, the developer can then merge the code to the staging branch where the quality engineer would execute functional tests.

What is the return on investment? ♻

The following are primary benefits if Dev Box is integrated as a part of the core development process:

  1. Technical understanding of the product by the quality engineer
  2. Upskilling quality engineer
  3. Failing fast provides faster feedback to the team
  4. Improve vertical testing
  5. Ensure trivial issues are not passed on, later to be found by the integration or end to end tests
  6. Improve cross-team communication and build trust
And with great powers comes great responsibilities.

The following are some trade-offs while implementing Dev Box in a team:

  1. It may take time to get each team member in sync
  2. If the team is not following the TDD approach, it may prove difficult or futile
  3. The team should consist of at least two quality engineers (Dev Box is quality-centric, hence, would require enough QEs to accommodate all tasks)

Dev Box, when implemented properly, could yield a quality, reliable product. Hoping these steps would come in handy when you’re planning to execute Dev Box testing. ✌️️

Want to read more stories from the vault? Check out our blogs!