The short answer to this question is 350. A score of 350 certifies you are knowledgeable and able to safely and effectively practice as a Physician Assistant. But what does this score actually mean? Where does this number come from? What is a passing PANCE ™ score in practical terms? We’ll lay it out for you today.
Standardized test development
The PANCE is a standardized test. Standardized tests use statistical analysis in both test development and scoring. The PANCE is no exception.
For a standardized test, strict specifications ensure each test is similar in difficulty. But, unless all tests are identical, it’s impossible to exclude some variability in the difficulty of individual questions.
For this reason, a scaled-score method is used. This limits the variability in difficulty across different tests. For example, someone scoring 50% on a difficult exam may score 80% on an easy exam. A process must be used to account for this variability.
A scaled score creates a standard range for test-takers and allows a fair comparison of results between different tests. It creates fairness among test takers given different questions on different days. No system is perfect, but most standardized tests use a standard-setting method.
The Angoff Method
The NCCPA uses the Angoff Method as their standard-setting approach. The Angoff Method is a system where field experts, called subject-matter experts (SMEs), examine each test question. These SMEs predict how many questions a minimally-qualified candidate, or in our case a new PA graduate, should be able to answer correctly. Each question is given a score based on these results.
For example, a question rated 0.50 means 50% of minimally-qualified candidates would answer it correctly. A question rated 0.90 means 90% would be able to, implying a much easier question.
After the SMEs individually rate the question, they review them a second time. Often this part of the process is done together as a group. A consensus is made about the rating for each question. This rating is called its predicted difficulty.
The predicted difficulty for all the questions is totaled, then divided by the number of questions. This value is called a cut score, or score required to pass.
Example
Here’s an example of how this likely plays out in developing a passing score for the PANCE.
A panel of physician assistant content experts with experience and knowledge about what a new grad should know is formed. This might include PA faculty or practicing PAs. Non-PAs with expertise in our field may participate as well. All we know is they are content experts on the knowledge, skills, and abilities of entry-level physician assistants.
These panelists individually review the test questions and determine whether a new PA graduate should be able to answer the question correctly. Once this is finished, everyone on the panel reviews the ratings of the other members and compares their ratings. They come to a conclusion and assign a predicted difficulty for each question.
The ratings are then totaled, and divided by the number of questions to get the passing score. For the PANCE, it’s 350.
Other scores you may see on your score report
Reference Groups
Reference groups compare a first-test taker’s scores to others taking the exam for the first time. This gives you a sense of how you compare to your peers. The sample size includes all first-time test takers over the previous two years.
Standard error of measurement
The standard error of measurement, or SEM, gives you a range you should be able to score if you took the test again without studying. A smaller SEM indicates you’re likely to achieve a similar score if you took the test again.
Subscores
Subscores tell you how well you performed in each area of content. Bigger content areas such as cardiovascular or pulmonary have more questions. This makes their SEM scores typically smaller.
If there are 40 cardiology questions and you miss 3, you likely know the content very well. If there are 5 hematology questions and you miss 3, your subscore will indicate you don’t know the content well. But, it may not be accurate. What if the difficulty level of those questions was high? More questions decrease the variability of scores.
The bottom line is the subscores can be helpful or interesting to look at, but may not accurately reflect your knowledge.
Passing the PANCE
When it comes down to it, passing the PANCE requires you to know the blueprint content at a level expected of a new graduate. You are not expected to score perfectly. The test takers aren’t out to make you fail. Understanding major systems, concepts, diagnosis, and common disease management will help you achieve a passing score.
