%201.svg)
Increase in CSAT
30%
Decrease in agent ramp time
120%
Increase in monthly coaching sessions
300%
Online fitness company ClassPass motivates people to live inspired, healthier lives by connecting them with impactful workout classes. Members can browse ClassPass’ library of 4,000 on-demand workouts, find live classes nearby, and choose from more than 40,000 virtual classes that are held online each week.
Seeking to surface data-driven insights and ensure an ever-improving customer experience, ClassPass launched its first-ever QA program in 2019. And, with the unexpected pandemic that changed every aspect of life in early 2020, ClassPass’ decision came at exactly the right moment.
Keep reading to learn more, or watch Sydney's session from The Future of Quality below!
Challenge: Elevating data integrity with objective QA grading and accurate tagging
Soon after its formation, ClassPass’ QA team realized that it needed a better approach for grading support agent performance. Prior rubrics were too generic, which yielded QA data that was neither informative nor actionable. For example, communication skills were assessed based on subjective criteria—such as “meets expectations” or “needs work”—which varied widely based on the grader’s perspective.
Inconsistent support ticket tagging in Zendesk further eroded confidence in ClassPass’ QA data.
“Only 58% of our email tickets were being accurately tagged by agents, which is extremely low,” said Sydney McDowell, CX Enablement Lead at ClassPass. “This was especially problematic, because we use tagging to understand inquiry volume by type and measure the Voice of the Customer.”
Unreliable QA insights lessened the team’s ability to support complex decisions about staffing and chatbot automation, accurately scope out “bugs,” and measure customer sentiment.
Solution: Used QA to improve the accuracy and reliability of support ticket tagging data by 30%
Realizing the need for more reliable QA data and richer CX insights, the team doubled down on the company’s adoption of QA best practices, processes, and systems—including fully utilizing MaestroQA and its suite of integrations.
Consistent QA auditing of support ticket tagging created a new sense of accountability for agents, resulting in dramatic improvements in data accuracy. In fact, email tagging accuracy increased to 88%—a 30% improvement. Chat ticket accuracy also increased to 87%, up by nearly 20%.
Hiring one full-time grader in tandem with implementing a binary grading rubric provided meaningful insights for accelerating change.
“Our new grading system enabled a more granular view of each agent’s performance in the use of proper spelling, grammar, brand voice, personalization, and other key criteria,” McDowell said. “This type of actionable data made it easier to identify opportunities for the team’s improvement.”
Result: Leveraging QA data to save 6250 days' worth of chat time annually and maintain 83% retention during COVID-19
Reliable and actionable QA data empowers the team to dig deeper and confront ClassPass’ biggest CX challenges.
For example, analyzing support ticket tagging data in MaestroQA allowed the team to identify and fix ClassPass’ largest driver of negative customer sentiment: cancellation chats.
“In 2019 alone, we spent the equivalent of 6,250 days—that’s more than 17 years—chatting with 1.5 million contacts for cancellations,” McDowell said. “COVID-19 compounded this situation, so we decided to fully automate this process. Now we have zero cancellation chats handled by agents.”
Speaking of COVID-19, data-driven QA insights helped the team stay one step ahead of account cancellations while maintaining high levels of customer satisfaction—despite shifting to a support model that is now comprised of 75% outsourced agents (up from only 25% prior to the pandemic).
“During COVID-19, we took the most customer-centric approach of anything we’ve ever done,” McDowell said. “By analyzing a combination of tagging data, CSAT data, and churn data, we learned that requiring customers to reach out and pause their accounts was causing unnecessary friction.”
To reduce friction and ensure a positive CX, ClassPass implemented an outreach strategy for COVID-19 account pauses. As COVID-19 infection rates climb in a geographic area, ClassPass proactively engages customers to ensure they’re not billed for services that they cannot actually use. Customers appreciate this, as evidenced by the rising CSAT scores and stable retention rates at ClassPass.
“Our COVID-19 outreach has a 96% CSAT rate compared to 87% CSAT for other inquiry types,” McDowell said. “We believe this has made an impact on our ability to maintain 83% customer retention throughout COVID-19, compared to our expected retention of 61%.”
In-depth analysis of QA data surfaced a major flaw in ClassPass’ workflow for reenabling paused accounts. Members who were supposed to be paused for 30 days were actually paused indefinitely, leading to unrealized revenue and inaccurate business projections.
“Discovering this issue led to a cross-functional initiative with product engineering. We changed how the unpause structure is configured so that we avoided missed income as markets recovered,” McDowell said. “We would have never known about this issue without QA data.”
Unlock Meaningful CX Insights with MaestroQA
Looking to elevate the productivity and effectiveness of your QA program? Start with a better approach by collecting and analyzing CX insights in MaestroQA.
