‍

‍

The CX leadership at Etsy takes a very scientific approach to managing the team. They keep themselves up to date with the latest books and studies in management psychology, and conduct focus groups and regular surveys of their agents in order to serve them better.

During one of their surveys, the Etsy team realized that they could do a lot better in the way they gave feedback, and that they needed the help of a robust QA program in order to do this.

‍

The challenge: Their QA program did not sufficiently recognize and reward agent’s efforts

Etsy introduced QA with the intent of scaling the team rapidly to 300 agents, while providing them with regular and thoughtful feedback.

Before embarking on a hiring exercise that would triple their agent count, the CX leadership at Etsy wanted to engage their agents and identify areas of improvement for their QA program. For QA, the leadership team organized focus group discussions for agents - and found that a majority of agents surveyed chose this sentence to describe their ideal outcome of a QA program: QA recognizes and rewards my best work.

Further interviews confirmed that the current QA program was not delivering on the above statement. As with many other QA programs, their current system felt punitive and designed to point out mistakes instead of celebrating wins - instead of recognizing and rewarding agent’s best work.

It was clear that Etsy’s agents wanted a better QA program to be put in place, and Etsy had to act on it before thinking about scaling further.

‍

The solution: Overhauled QA program and QA scorecard to build in positivity and qualitative feedback

The QA team set out to do some research before committing to changes.

They learned about a study that discovered that the ideal praise-to-criticism ratio in high-performing teams was 5:1, and suggested that CX teams performed better in positive environments. With positivity in mind, the team started to make specific changes to include gratitude in their systems. 

Etsy decided to do away with the 5-point scale on which they graded, choosing instead to implement a binary “meets expectations/does not meet expectations” model, and shifted the focus to providing more qualitative feedback from the grader.

There was still one piece left in the puzzle. Etsy brought MaestroQA on board to help tie all of these new initiatives together into a single platform that scaled with their team while providing trusted data on the changes that were being implemented.

‍

The positive impact: Data-driven business decisions, and happier agents

Implementing MaestroQA gave Etsy’s CX leaders a wealth of QA data that they could trust to make decisions.

The team could now easily quantify their QA praise-to-criticism ratio - something that they previously were unable to do. This ratio currently stands at 3.7:1, and the team is constantly working to bring it closer to the ideal 5:1 ratio.

With more than 30,000 tickets graded since implementing their new QA program, the team can easily quantify the effects of changing their grading model: agents meet expectations in 76% of scored criteria, while receiving thoughtful, detailed feedback from graders for the other 24%.

By removing their 5-point grading system, Etsy has placed great stock in the collective institutional knowledge in the CX team. This has helped to build trust, and allowed agents and managers to have deeper conversations about whether a customer interaction had hit or missed the mark.

Agents and managers alike have reported feeling a lot happier with the new QA program, and it’s easy to see why. Etsy’s QA program now combines CX management best practices and best-in-class QA software to celebrate the wins of each agent and provide timely feedback where necessary.