Developing PQA Measures Through Public Comment and Endorsement
PQA follows a systematic, transparent and consensus-based development process to ensure that measures are important, scientifically acceptable, feasible and usable. This approach produces strong, consistent results, but it also takes time.
From the specification phase of convening a technical expert panel to achieving PQA endorsement, measure development can last 12-24 months or longer, depending on the complexity of the measure. As a result, it’s critical that PQA selects the right measures for development. In November 2021, we posted a blog that walked through the first step of the measure lifecycle, measure conceptualization, and in the second of this series, posted April 2022, we detailed the next steps of the process, measure specification and measure testing. This blog will describe the measure endorsement process, including panel reviews, public comment period, and PQA membership voting.
Measure Endorsement (Part One): Vetting Measures Through PQA Panels
Once a measure completes the measure testing phase, it undergoes review by a pair of critical PQA panels, beginning with the Measure Validity Panel (MVP). This group of 10-15 expert members is composed of a subset of the Measure Update Panel selected by PQA staff to evaluate measure validity. After reviewing measure specifications and testing results and discussing the measure during an online meeting, members of the MVP assess face validity by voting on whether the performance scores resulting from the measure can be used to distinguish good from poor quality care. This builds on the face validity assessment conducted by the TEP during the specification phase by offering an additional evaluation from experts who were not involved in the development process.
Following the MVP’s vote, the measure proceeds to the final Quality Metrics Expert Panel (QMEP) evaluation. As discussed in the previous blog, the QMEP is a standing panel composed of 20-25 members with expertise in measurement science, clinical practice, data analytics, quality improvement and measure use. QMEP members have extensive experience serving on PQA-convened measurement groups and are selected by PQA staff through an annual self-nomination process.
The QMEP reviews measure materials produced during development, including specifications, testing results and MVP voting results. They evaluate the measure according to PQA’s key measure criteria, listed below, and vote on whether or not they recommend the measure to proceed to PQA’s membership for an endorsement vote.
Measure Endorsement (Part Two): Soliciting Feedback Via Public Comment
Following the QMEP’s recommendation to advance to an endorsement vote, the measure enters a public comment period. The comment period represents an important part of the PQA measure development process to maximize transparency and allow all stakeholders, including those outside PQA’s membership, to make their voice heard. This follows best practices laid out in important measure development resources such as the CMS Measures Management System Blueprint.
To kick off the comment period, PQA staff distribute a Call for Comments on the measure being considered for endorsement via email communication and other channels to membership and relevant external stakeholders, including patient representatives and patient advocacy groups. The Call for Comments includes the measure specifications, evidence-based rationale, and a summary of key considerations about the measure including the development process, intended use, and other information needed to support informed and thoughtful comments, and for members to make an educated endorsement decision.
The public comment period lasts 15 business days, and PQA staff are available throughout the comment period to answer questions and provide as much clarity as possible to support member decision making. Following the comment period, PQA staff carefully review all comments received and note emergent themes and key areas of feedback.
The QMEP reviews measure materials produced during development, including specifications, testing results and MVP voting results. They evaluate the measure according to PQA’s key measure criteria, listed below, and vote on whether or not they recommend the measure to proceed to PQA’s membership for an endorsement vote.
Measure Endorsement (Part Three): PQA Membership Voting
After the close of the comment period, PQA hosts an all-member webinar to summarize and address feedback received during the comment period. This webinar provides PQA members with critical information regarding the measure and provides an opportunity for membership to ask additional questions prior to casting their vote.
Immediately following the all-member webinar, the measure moves to the PQA membership endorsement vote. By this time, the measure has been
- In development for anywhere from 12 to 24 months or longer;
- Approved by its Technical Expert Panel;
- Tested using representative datasets;
- Approved by the MVP; and
- Found to meet PQA measure criteria by the QMEP.
Measures that advance to membership endorsement vote are products of countless hours of work from PQA staff and experts from across PQA membership and represent a product that has passed through PQA’s rigorous, consensus-based development process.
During the PQA member endorsement voting period, each PQA member organization casts its vote for Measure Endorsement noting their support (yes), opposition (no) or abstention. The voting process takes place electronically and lasts for 10 business days. Each organization has one vote, and a greater than 60% vote in support (of member organizations voting yes or no) is required for a measure to receive PQA endorsement.
When a measure is endorsed by PQA’s membership, it is added to the PQA Measure Manual and associated value sets will be distributed to licensees during the next February release. The PQA-endorsed measure is then ready to be used in programs to help improve medication use quality!