Sentiment analysis has emerged as a pivotal topic in the realm of customer support interactions, sparking intense conversations among industry professionals. As CX (Customer Experience) and QA (Quality Assurance) practitioners, it's crucial for us to approach this technology with a discerning eye, understanding both its potential benefits and challenges. We recently hosted a webinar discussion with CX leaders aiming to shed light on these important questions, fostering an environment of critical thinking and innovative problem-solving. In this blog, we delve into the key takeaways from this discussion, offering practical insights to effectively integrate sentiment analysis into QA practices.

"Sentiment analysis is something that's coming up so much in our conversations with customers & potential customers. It's the type of topic that is being thrown around so much on the internet and by different folks and it's a very sexy idea." - Vasu Prathipati, CEO & Co-Founder, MaestroQA

‍

‍

Wide Adoption and Implicit Detection 

Sentiment analysis has gained popularity due to its ability to analyze sentiment across a high volume of conversations. Unlike traditional customer satisfaction (CSAT) surveys, sentiment analysis offers implicit detection of customer sentiment without the need for explicit surveys. This wide adoption provides valuable insights into customer interactions.

"Sentiment analysis could be on 100 percent of conversations and it's implicitly detecting what is the customer experience sentiment versus asking for an explicit survey." - Vasu Prathipati, CEO & Co-Founder, MaestroQA

Enhancing Quality Assurance Efforts 

One of the main motivations for exploring sentiment analysis is to augment QA programs without significant increases in individual grader headcount. By leveraging sentiment analysis, companies can gain deeper insights into customer support interactions, even when QA resources are limited. It offers the potential to scale QA efforts and derive valuable analytics from a larger volume of data.

‍

With MaestroQA, teams can comprehensively analyze 100% of interactions for negative sentiment using an omni-channel transcription tool. Through transcribing all voice interactions, teams can construct AI classifiers and Smart attributes based on keywords. These conversation analytics are completely customizable to accommodate the specific needs of the business.

"The biggest pushback that I hear from our customers is, 'Hey, we love Maestro. We plan on scaling the team, but we can't scale the QA program in terms of headcount proportionately. What kind of technology and AI and stuff can you use to help us get more in analytics without having to scale the QA teams headcount?'" - Vasu Prathipati, CEO & Co-Founder, MaestroQA

‍

Third-Party Tools and Open Source Solutions 

Many companies, (including MaestroQA, at the moment) utilize third-party tools to power their sentiment analysis capabilities. Industry giants like Amazon and Google offer pricey sentiment analysis APIs that leverage machine learning models. Additionally, open-source technologies have emerged as cost-effective and potentially improved options for sentiment analysis.

"Most people are using third party technologies to power their sentiment. We're evolving our sentiment analysis at MaestroQA. We use a hybrid of multiple tools to provide our sentiment analysis and we're working on transitioning towards an open-source model. The benefit of that is we can do it at a tenth of the cost, so we can make it a lot more affordable for our customers." - Vasu Prathipati, CEO & Co-Founder, MaestroQA

‍

Use Cases and Nuances 

Sentiment analysis can be applied to various use cases, such as measuring positive or negative sentiment in agent and customer emails, chats, or phone calls. Each use case has its own nuances, and sentiment analysis should be tailored accordingly. As every business is different, "positive sentiment" can mean one thing for one company and something completely different for another. Once you've determined what type of sentiment you are looking for, positive sentiment may be more relevant in evaluating agent performance, while negative sentiment can help identify areas for product improvement or potential process & policy adjustments.

‍

For example, using MaestroQA, Novo, a leading mobile banking platform for small businesses, was able to uncover an issue that was occurring with mobile check deposits, where specific checks from the California State Government were being repeatedly rejected by their banking software. With the use of AI Classifiers, Marcus will be able to build reports using keywords associated with the "check rejection" issue to determine its frequency, identify which agents are skilled at handling these situations, and assess the negative sentiment resulting from these customer interactions. MaestroQA's highly accurate transcriptions will even enable the Classifier to listen for key phrases such as "check rejected, California, or deposit error" to identify phone calls discussing this particular issue. Novo’s QA team can also pull in other metadata from his HelpDesk to identify tickets where a customer started the conversation upset but was satisfied with the agent's response or conversations where a customer initially felt fine but became upset due to the agent's service or policy.

"You have to think about your actual conversation flow and where does it make sense to actually try and detect it. When applied critically, sentiment analysis can uncover hidden trends you wouldn't see." - Vasu Prathipati, CEO & Co-Founder, MaestroQA

Accuracy and Actionability 

Ensuring the accuracy of sentiment detection is essential. If your sentiment analysis is yielding high overall positive sentiment scores, its actionable value should be questioned. Metrics that provide actionable insights should be prioritized, enabling targeted coaching and informed decision-making. 

‍

Our customers are harnessing the power of our AI-assisted sentiment analysis to develop more refined and actionable metrics, which in turn provide a detailed overview through a Targeted QA heatmap. For instance, one company identifies tickets with a "low empathy response," a customized Key Performance Indicator (KPI) established using MaestroQA's sentiment analysis and AI classification tool known as AI Classifiers. The concept of "low empathy response" involves detecting instances where a customer expresses negative sentiment and receives a poorly tailored macro-like response from the agent. When agents fail to provide personalized or appropriate reactions to customers' negative sentiments, they miss the opportunity to connect effectively with the customer, potentially exacerbating the negative experience.

‍

By unearthing this particular metric, the team managed to delve deep into targeted QA, uncovering valuable learning and coaching prospects for their agents. This proactive approach aims to mitigate such interactions in the future and enhance overall customer experiences.

"Did you get a new perspective that you can consider internally to have smarter discussions internally?" - Vasu Prathipati, CEO & Co-Founder, MaestroQA

‍

Machine Learning and Agent KPIs 

During the discussion, opinions were divided on using machine learning-based sentiment analysis as a key performance indicator (KPI) for measuring agent performance. While some participants (47% according to a poll) saw potential in leveraging machine learning for identifying lower-performing agents or areas of concern, others expressed concerns about the explainability and transparency of such metrics (41% said no and 11% were undecided).

“I think a really important consideration when we're talking about, ‘Hey, do we wanna use this as a way to measure agent performance?’ Well, it's machine learning. If someone on your team says, ‘Hey, why did I get this score? Why is this my score?’ How comfortable do you feel explaining this?” - Vasu Prathipati, CEO & Co-Founder, MaestroQA

‍

The Black Box vs the Transparent Box

‍

At MaestroQA, we adopt a cautious approach to Sentiment Analysis and AI implementation to maximize accuracy and reliability of results. Amidst the numerous claims of software offering 100% automated QA for customer interactions, a crucial detail often remains unaddressed. The source of this data remains ambiguous: Is it derived from an undisclosed black box of sentiment analysis or an internally developed Auto QA system that's uniformly applied across all QA clients?

‍

At MaestroQA, we are committed to dispelling this opacity. Through our MaestroQA AI Classifiers, QA teams gain the ability to establish tailored customer Auto QA KPI metrics (a transparent box). This empowers you and your team to precisely define positive and negative customer sentiment within the context of your unique business and outline benchmarks for superior agent performance that align with your customer expectations.

‍

The development of these AI classifiers occurs collaboratively with your designated MaestroQA Customer Success Manager. By deconstructing the specific customer pain points you intend to address, you and your representative collaboratively construct a logical framework within the MaestroQA tool. This framework facilitates the comprehensive scanning of 100% of support tickets to identify qualifying attributes. The outcome is then presented to you on a Performance Dashboard, displaying the percentage or total count of tickets that adhere to this established criteria.

‍

What sets our sentiment analysis and auto QA apart is its genuine relevance to your business. Through customization, it becomes a tool that resonates meaningfully within your unique operational context.

‍

Bridging the Divides 

In conclusion, sentiment analysis shows promise in enhancing customer support quality assurance efforts by providing deeper insights and analytics. However, it is crucial to carefully evaluate use cases, choose appropriate tools, and ensure the accuracy and actionability of sentiment analysis results. Additionally, using sentiment analysis as a key performance indicator (KPI) for agent performance should be approached cautiously, taking into consideration the transparency and explainability of machine learning-based metrics or the value of sentiment analysis as a tool for targeted manual quality assurance.

‍

The community discussion on sentiment analysis, as part of our bi-monthly CEO/CX: Strategy Chats, sparked valuable insights and discussions among participants, enabling them to consider new perspectives and challenges. As the field continues to evolve, staying informed about the latest developments, evaluating the suitability of sentiment analysis for individual businesses, and making informed decisions based on specific use cases and requirements are essential.

Want to learn more?

For a deeper dive into these topics, we recommend watching the full recording of our Strategy Chat with MaestroQA CEO & Co-Founder, Vasu Prathipati, or joining our upcoming webinars on the future of customer experience. Your participation is greatly appreciated, and we look forward to engaging in more insightful discussions in the future. If you're not already familiar with MaestroQA, don't hesitate to request a demo today!