We were building the industry’s first end-to-end digital payroll system, which was about to be used by major US studios for the first time.
We had kicked off development work and our MVP was due for release in a few months. Previous product launches had experienced a rocky start - just weeks after the release of our timesheets product, we uncovered a bug that had led to numerous people being paid incorrectly.
Problem
There are more than a 5²⁰⁰ combinations of fields entered in the contracts and timesheets needed to calculate payroll.
Our QA team was small, but even if every person on earth could help test - they’d need to test more than 2 trillion combinations each, every time we pushed a release. This is simply impossible without automation.
We needed to increase the bandwidth of our QA function while minimising risk of error and maximising confidence in our software.
The development team already had unit tests for the code they had written - but, aside from the obvious downsides of marking your own homework, they didn’t reflect the end-to-end capability that we needed to test. Our product was complicated, with objects being passed between different people in order to progress towards payroll.
I was the sole designer working across three squads, one of which was dedicated to building this payroll product.
As the Product Designer, I was responsible for:
Discovery
Interaction design
User testing & design QA
I also wrote the queries and built the Domo dashboards to measure the impact of the design.
My hypothesis was that we could reduce the time it would take to start running automated tests by basing our automated tests on real life scenarios.
I had previously coordinated the largest ever internal end-to-end test for our nascent timesheets product. Because of this, I was invited to join our SVP of Global Engineering and VP of Delivery in India to train our recently acquired external QA team about our users and the way they coalesce across our products and services.
I flew to Hyderabad to deliver hands-on training about our users and our software, including:
Why people used the Production Portal
How the different personas interact with one another and how that’s represented on the portal
The detailed workflows and each persona’s involvement in them
Given the complex nature of the product, was prepared to start with the basics.
However, the team, comprised of a number of former software developers, had reverse-engineered the code and done quite a lot of research into the industry already. Srikanth, the team lead, had even prepared a pocketbook that summarised almost everything I was going to present using just two sides of A3 paper. This meant I had to adapt my approach, diving into far more detail than I had initially prepared for.
By the end of a challenging and lively day of training, I’d answered all of the team’s questions and, in discussion with my colleagues, we were highly confident in the team’s knowledge and abilities.
Results
The training I delivered reduced the time it took to start writing tests from 3 weeks to 3 days.
We laid the foundations for automated testing and maximised the teams’ confidence by answering 100% of their questions.
Since my trip, I am in regular communications with the team.
I’ve sent Srikanth coins to add to his collection
I’ve (virtually) celebrated the birth of Ram’s child
I’ve tried (and failed) to answer some of Kowsalya’s most detailed follow-up questions