Flip is a series B startup enabling free, instant inter-bank transfer for Indonesian consumers and businesses in South East Asia. Below, we spoke to Harith Fanani, the Staff Test Engineer at Flip. We asked him how he had come to work with Global App Testing, and how he'd come to save a week and a half in Flip's troublesome regression testing suite.
Interested in learning about how we could help you? Talk to a member of our team.
The company
Flip is an Indonesian Fintech company. Its main product is free and seamless interbank transfer for consumers and businesses.
Founded in 2015, Flip raised a $48m in a series B in 2021. That sets Flip on a course to continue their formidable growth into 2024 and beyond.
The challenges
- Financial transactions is a regulated space which makes for long regression cycles.
- The lack of in-house device management meant that the Flip team was unable to simulate the real experiences of their users
Month-long continuous regression cycles
"Before we had a partnership with GAT, the regression cycle time was slowing down everything in the business. It was a real bottleneck," said Harith.
"We did regression and regulatory testing for days and sometimes weeks…. The sanity tests alone took days. It took so long because [as a fintech provider] we have to execute more than 2,000 test cases and we have resource constraints. We do back-to-back regulatory testing which takes a long time. We also don't have time to undertake projects we'd like to, like automating more of our test suite."
"A regression cycle this long meant we couldn't get the features we needed released fast enough. We're a startup and we need to grow every year and onboard new clients. In order to do that, we need a high feature output which was ultimately being slowed down by our regression cycles."High quality and compatibility requirements
We asked Harith why it's essential to have a high-quality product. "We're in a highly regulated industry, so a significant bug could ultimately put us in breach of regulations. Also, consumers need to have the very highest confidence in financial applications because money is something they have worked hard for and want to protect."
"Most of our users are on mobile, and I personally am responsible for our mobile app. But that brings with it a high degree of compatibility testing – our app is available on multiple operating systems and our users use a wide range of devices."
Global App Testing delivered a bespoke solution to booking dot com which was tailored to their challenges and goals.
Regression tests in two days or less
"When we first signed up with GAT, we wanted to see how long your testers could take.... it was amazing that you could get through our test cycles in days."
The Flip team jettisoned as much of their regression cycles as they could to Global App Testing in order to radically improve the speed of releases. "The main reason I'd recommend Global App Testing is that you can complete the test cases much faster than our internal testers.
Broader and best-practice device mix in the staging environment
"We base our targeting in Global App Testing on the device mix which represents most of our users. This is really useful because many of the most critical bugs we've found with Global App Testing have been on devices we haven't always had the facility to test in-house. For example with Oppo or Xiaomi devices."
The results
1.5 weeks saved per sprint cycle
The main result is that Harith is able to get regression tests done much faster. "Since we can release more, faster, to users, that impacts our revenue. The marketing teams can roll out the campaign earlier to users. That’s why we wanted a testing service... to get more efficient and more effective. We can also get features to users faster."
20% of total test scenarios automated
Harith says he and his team conduct just 30% of the test case scenarios that they did before. Global App Testing have taken on 50% of the test case scenarios, resulting in a time and labour saving more than the value of the GAT contract. Additionally, that free time has allowed the Flip team to automate the final 20% of test cases resulting in a direct saving.
