This guest blog is one in a series by sponsors of the 2023 PQA Leadership Summit on elevating clinical quality assurance. The views shared in this series are those of the author and do not reflect any PQA positions.

In the heart of communities, pharmacists stand as the lifelines of accessible healthcare education. Through Medication Therapy Management (MTM), they establish unique connections with patients on a personal level while offering profound insights into medication therapy. At MedWatchers, quality is at the top of our priorities. Delivering the highest quality of clinical services is paramount in ensuring healthcare resources don’t go to waste. As we continuously try to innovate and think outside the box to optimize healthcare solutions and patient engagement, we embrace cutting-edge technologies such as Artificial Intelligence (AI) and automation. The integration of AI and automation into our operations bolsters our efforts to enhance health outcomes and elevate the standard of patient care in an ever-evolving healthcare landscape.

Introducing a New Perspective: Scaling QA with AI

Venturing into the integration of AI into our quality assurance (QA) program not only brought about a new, insightful strategy for providing continuous and swift feedback to our team, but also demanded persistence and dedication to achieve usable results and functionality. For MedWatchers, AI-assisted validation and evaluation streamlined call processing, enabling our QA team to evaluate twice as many calls as before. This, coupled with AI-leveraged process changes, introduced a methodical layer of feedback for new hire pharmacist training, enhancing scalability and efficiency in our QA efforts.

In the initial stages, our team engaged in manual data validation sessions to enhance the accuracy of transcriptions and refine our automation model. The result was a remarkable 90% validation rate, indicating that 90% of the items automatically evaluated for a call no longer required manual evaluation by QA. During validation, the model's predictions were compared to actual, known outcomes in the validation set. These collaborative efforts were instrumental in overcoming obstacles, optimizing AI-driven transcription generation and automation processes, and laying the foundation for successful integration into our QA workflow.

Despite significant progress in the capabilities of AI, it's essential to acknowledge its inherent limitations. The most significant benefit was the complementary effect of AI on improving the human QA experience. Rather than replacing people, AI enabled the same individuals to evaluate the quality of more calls, making evaluations more streamlined and consistent. This idea of AI enhancing human skills, particularly in pharmacy and MTM, is a crucial lesson we gained from this experience. The combined capabilities of AI and human expertise lead to synergistic quality improvements across the board.

A Glimpse into Behavioral Trends and Future Potential

These new innovations have ushered in a new era for quality assurance at MedWatchers. Beyond optimizing QA team efficiency, it has unveiled valuable behavioral trends on both individual and team levels. Applied initially to comprehensive medication reviews (CMRs), this innovative step holds promise for scalability across various quality and performance metrics, aligning with our commitment to continuous improvement and reinforcing our mission to elevate the standard of patient care at every encounter. In embracing this innovative approach to QA, we are not just redefining quality assurance but reassuring that the best clinical services are delivered at MedWatchers.

Improvement Summary:

• 40% increase in overall time efficiency with use of transcriptions
• Ability to detect errors in quality focus areas 7x as fast, allowing for faster and more continuous feedback
• Persistent training of the AI-model helped to improve accuracy of automated transcription grading to 90%

MedWatchers Authors and Collaborators:

Alexandra Basmadjian, PharmD
Clinical Program Lead, Quality Assurance Pharmacist

Christopher Armstrong
Chief Information Officer