Measure Specification: Engaging PQA’s Members through Technical Expert Panels 

Measure Specification is the stage of development where a concept is refined for testing, complete with a detailed definition of the numerator, denominator, exclusions and other elements of the measure.  

In the early stages of development, PQA staff create preliminary measure specifications that serve as a starting point. However, refinement of initial specifications requires diverse viewpoints from methodology experts and the stakeholders involved in the medication use process, who would be held accountable for a measure or play a role in its implementation. To that end, PQA leverages our members to gather input through Technical Expert Panels composed of individuals with expertise across clinical, measurement, and programmatic domains. 

Technical Expert Panels (TEPs) are small groups (10-15 members) composed of relevant subject matter experts from PQA members to address key questions identified during initial measure concept specification and feasibility assessment. In certain situations, external non-member experts may also be invited to participate. 

Prior to convening a TEP, PQA issues a call for self-nominations. Interested individuals complete a self-nomination form to highlight their relevant experience and qualifications to participate in the TEP. PQA staff review all self-nominations and assemble a panel that brings the necessary expertise while also representing the diverse stakeholder viewpoints of PQA members. 

TEPs consider key questions and potential measure design options during the specification process. TEP members engage in consensus-based decision making, including participating in votes to establish agreement on critical decision points. A good example of TEP input is in the discussion and evaluation of the lookback period in the Initial Opioid Prescribing for Long Duration (IOP-LD) health plan measure, which evaluates the initial opioid prescriptions with greater than seven cumulative days’ supply. 

The lookback period in the IOP-LD measure determines the length of time preceding an opioid prescription claim, during which there were no other prescription claims for any opioids, such that the claim could be considered an initial prescription. Lookback periods were variably defined in the literature, and the TEP carefully discussed the benefits and drawbacks of various lengths of time and associated tradeoffs between sensitivity and specificity. To further inform discussions, sensitivity analyses were completed and shared with the TEP to add a data-driven perspective to the evidence and clinical rationale. 

Calling on their diverse sets of experience, sensitivity analyses, and existing literature, the TEP came to consensus on a lookback period of 90-days. The Initial Opioid Prescribing measures have since been endorsed by PQA members, and the IOP-LD measure is additionally endorsed by the National Quality Forum and implemented in the Medicare Part D Patient Safety Reports. 

When specification is complete, the TEP also completes a face validity assessment to evaluate the extent to which the measure captures the intended aspect of quality, and votes on the measure concept’s readiness to move forward to testing. 

Engaging PQA’s Expert Members: The Quality Metrics Expert Panel  

After specification is complete, PQA creates a testing plan and presents it to the Quality Metrics Expert Panel (QMEP). The QMEP is a standing panel composed of 20-25 members with expertise in measurement science, clinical practice, data analytics, quality improvement and measure use. QMEP members have extensive experience serving on PQA-convened measurement groups and are selected by PQA staff through an annual self-nomination process. 

The testing plan is a comprehensive document that describes the activities that will be completed by testing organizations and includes detailed results templates that these testing partners use to summarize their findings. The QMEP reviews the testing plan to ensure the measure testing process will answer key questions needed to evaluate the measure and provides feedback to hone the testing approach.  

Measure Testing: Informing Measure Development with Comprehensive Data 

The goal of measure testing is to apply the measure specifications to data representative of the intended measure population to evaluate critical characteristics like reliability, opportunity for improvement, distribution of measure rates, frequency of exclusions and other properties.  

PQA frequently tests measures using internal data assets, such as samples of Medicare or Medicaid data. However, a core aspect of PQA’s testing process is partnering with external organizations with access to data sources needed to calculate the measure rates. Collaborating with external testers broadens the test population, validates that the measure specifications are clear, and ensures the calculation steps are reproducible.  

PQA prioritizes member organizations when testing opportunities are available, and testers gain early insights into how their organization might perform if the measures were ultimately adopted into an accountability program. Knowledge gained form testing informs refinements to measure specifications and gives PQA the confidence that measures will work as intended when implemented in quality programs.  

For example, during the development of the recently endorsed Migraine Preventive Therapy (MPT) health plan measure, the TEP discussed potential exclusion of individuals with cluster headaches and tension-type headaches from the measure. These populations have different recommendations and clinical considerations than individuals with migraine headaches, and thus are not appropriate for inclusion in a measure capturing appropriate use of migraine preventive therapies. However, the TEP was unsure of the frequency of these populations being included in the eligible population of the measure as specified, and thus whether an exclusion was necessary. 

To help answer this question, PQA and testing partners explored the frequency of these potential exclusions during the testing process. Analyses revealed that exclusions of individuals with cluster headaches and tension-type headaches affected as much as 15% and 11% of the eligible population, respectively, which confirmed the importance of having these exclusions in the final measure specifications. 

Get Involved: Future TEP Opportunities 

If you belong to a PQA member organization, you can participate in the specification process! Later this year, PQA will launch a technical expert panel to support development of a Chronic Obstructive Pulmonary Disease Treatment Ratio health plan measure, so watch for communications with the timeline and process. We look forward to reviewing qualified candidates’ self-nominations.