Important Information about the 2020 Patient Advocate Certification Exam Cycle

by Heidi Kummer, MD, MPH, BCPA, Exam Chair, on behalf of the Patient Advocate Certification Board of Directors

To say that 2020 has been an unusual year would be an understatement. The impact of the Covid-19 pandemic has been devastating and far reaching, including causing major disruptions to the normal processes of offering and taking certification examinations.

All organizations issuing credentials like the BCPA must engage in ongoing review and analysis of each item on the exam as well as overall performance of each exam cycle. PACB, in collaboration with a panel of Subject Matter Experts (SMEs) and psychometricians perform these tedious but important tasks each year. In fact, the exam committee and SMEs worked tirelessly last fall and into the spring writing new items and applying the Angoff*/Hofstee** process in order to statistically establish a cut score on the new version of the exam. In addition two sets of 25 brand new questions that did not contribute to the score were added for pilot testing.

Results of the revised exam given to the first cohort scheduled to test in March 2020 were eagerly awaited, but then the pandemic hit. With COVID-19 becoming a public health emergency, the entire network of Prometric test centers abruptly closed just a few days into the testing cycle, which meant only a tiny fraction of the registered candidates completed the test in the spring. Great effort was expended to enable the option for remote proctoring and although we managed to squeeze in an interim summer cycle, it too was quite small with most of the 2020 candidates moving to the fall cycle.

Adequate sample size is a significant factor for any effective and reliable analysis. To use one of the recent examples from Covid-19, after a small sample of patients seemed to get better after receiving Hydroxychloroquine, it was touted as the game changer in the fight against the virus. Once it was given to larger numbers of patients and could be more thoroughly evaluated, however, it turned out not to have much benefit at all and possibly even be harmful.

In terms of the BCPA exam, the amount of data collected from the spring and summer cycles was simply too small to yield any meaningful information. It was not until the data from the March and July cycles could be combined with the fall cohort data that a large enough sample size allowed more reliable and meaningful analysis.  After careful review of the collective data, the exam committee in collaboration with psychometricians determined a few of the new items had not performed as expected.  Thanks to the due diligence of having completed the *Angoff/Hofstee** process in which a cut score range is determined, the cut score was adjusted to reflect the impact of those outlier items.

It is not at all unusual to re-evaluate or rescore an exam based on poor item performance; in fact it is common practice. What made this process so unusual is that it affected previously tested cohorts. Most exams forms get switched out and are thus never identical; in this case all candidates took the same exam form, so any changes to the fall cycle had to also be applied to the spring and summer cycles. All candidates received letters explaining this process.

The PACB is sorry for any confusion this may have caused, but trusts this will allay concerns over the fairness or validity of the Patient Advocacy Certification Exam. In closing, we would like to wish all a safe and Happy Holiday and once again congratulate and welcome the entire 2020 cohort of newly credentialed Patient Advocates!

Please contact our Executive Director, Danielle Marshall  with any questions.

You may also be interested in this post by Vita Greco, MA, Director of Psychometric Services, Professional Testing Corporation – How a Certification Exam is Developed

*Click here for more information on the Angoff method
**Click here for more information about the Hofstee process