Computer-adaptive testing (CAT) has become a well-accepted testing modality today. Despite its acceptance and broad-based use, there remain many questions about CAT. The most frequently asked questions from candidates taking the National Council Licensure Examination (NCLEX(R)) with answers follow.
If I had just been administered more questions, I know I would have passed the test. My paper-and-pencil test was a lot longer. This test is unfair.
Perhaps an explanation of how CAT works for many testing programs would be helpful. The goal of CAT is to determine candidate ability, based on the difficulty of questions answered correctly, NOT on how many questions the candidate answers correctly. This is a fundamentally different approach than is used on paper-and-pencil tests, where everyone receives the same questions. CAT examinations are individualized.
The difficulty of each of the questions in the item pool is known, because each has been taken as a pretest (or unscored) question by hundreds of candidates and then statistically analyzed. Imagine the questions all lined up, from easiest to hardest. If candidates were asked the easiest questions, they would get most of them right. If candidates were asked the hardest questions, they would probably answer most of the questions wrong. As a candidate moves from easy to hard questions, there will come a point where the candidate goes from giving more right answers to giving more wrong answers. This is when the candidate is answering 50% of the questions correctly. For harder questions, the candidate would probably answer them incorrectly (some would be correct, but more would be wrong); questions easier than that would probably be answered correctly. That point is different for everyone.
Subject matter experts could probably answer correctly at least half of the hardest questions, whereas beginners would need to be asked the easiest questions before they could answer even half of them correctly. Most candidates for the nursing licensure examination probably fall somewhere between those two points. The goal of CAT is to find that point for the individual candidate. Candidate ability level is related to the difficulty level of the questions at the point where the person can answer half of the questions correctly. (Some testing programs use a different percentage, such as 60%.)
When a candidate begins the NCLEX, the computer administers a relatively easy question, and if the candidate answers it correctly, the computer administers a somewhat harder question. As this candidate continues answering questions correctly, the questions get harder and harder. If the candidate answers a question incorrectly, the items administered are sequentially easier until that candidate answers an item correctly again. The next question administered after that correctly answered item is classified as more difficult. Each time the candidate answers an item correctly, the next question given will be more difficult. (Test questions on the NCLEX examination are referred to as "items.")
Each time the candidate answers a question incorrectly, the next item the candidate receives will be easier. This process continues as it zigzags, narrowing in on the point where the candidate has answered 50% of the questions correctly (eg, 1 right, then 1 wrong). That point represents the candidate's ability level. This is why everyone ends up correctly answering about 50% of the questions they are asked.
After candidates have answered the minimum number of questions, the computer compares their ability levels to the NCLEX passing standard and makes 1 of 3 decisions:
1. If candidates are clearly above the passing standard, they pass and the examination ends.
2. If candidates are clearly below the passing standard, they fail and the examination ends.
3. If candidates have ability levels that are close to the passing standard and it is still not clear whether they should pass, then the computer continues to ask questions until the individual candidate has provided correct answers to 50% of the questions.
"Clearly" passing or failing is defined as when the "gray zone" around candidate ability level falls entirely above or below the passing standard. The gray zone is the region within which the ability level of the candidate might vary if administered more questions. The gray zone shrinks after each question, because the candidate's ability level is based on more information.
When the candidate answers the difficult questions correctly, there is no point wasting candidate time by administering easy questions. If the candidate cannot answer the easy questions correctly, that candidate will not be able to answer the difficult questions. In fact, the computer often could make a decision after less than the minimum of 60 questions, but 60 is necessary to ensure coverage of the NCLEX-RN(R) test plan. Different test plans and testing programs specify different minimum and maximum numbers of questions. It is important for candidates to have the opportunity to answer several questions in each of the test plan content areas because this allows for demonstration of particular strengths or weaknesses.
After each question, the candidate ability level and the gray zone are recomputed, adding the latest response to the total of the candidate's previous answers. When the gray zone is entirely on one side or the other of the passing standard, the candidate will have clearly passed or failed and the examination ends.
Of course, some people have ability levels that are close to the passing standard. For some (but not all) of these individuals, all of the questions in the item pool still might not be enough to make it "clear" whether they should pass or fail. These are the people who are administered the maximum number of questions as illustrated in Figure 1. At that point, the computer disregards the gray zone and simply looks at whether the final ability level, based on every question answered, is above or below passing. If the candidate is above the passing standard (illustrated by the straight line), the candidate passes. If not, the candidate fails.
Is it true that some candidates are randomly selected to receive a long test?
It is NOT true that some candidates randomly receive a maximum-length examination. The length of an examination is based totally on the performance of the candidate on the examination. As a candidate takes the examination, questions are selected based on the individual candidate response to previous questions. Once the minimum number of questions has been answered, testing will stop when candidate ability is estimated to be either above or below the passing standard with a predetermined level of certainty, regardless of the number of questions taken or the amount of testing time elapsed. If the candidate's ability is close to the passing standard and it is still not clear whether the candidate should pass or fail, then the computer continues to ask questions until a clear pass or fail decision can be made, the maximum number of questions is reached, or time runs out. Thus, depending upon candidates' patterns of correct and incorrect responses, different candidates will take varying numbers of questions and use varying amounts of time.
Several questions on my examination asked the same information. Did I receive similar questions because I answered incorrectly the first time?
Each examination is designed to meet all NCLEX test plan requirements. The examination is constructed to recognize that each candidate has strengths and weaknesses in particular subject areas and is NOT designed to administer a similar rephrased question for questions answered incorrectly by candidates. There are several reasons candidates may receive questions that appear to cover similar content areas. Candidates may have received pretest questions (questions that are embedded in the examination that do not count but are used in future examinations as real or operational questions) or questions in which the content was similar but the question covered a different area of the test plan.
How can I find out more Information about the NCLEX examination?
The National Council of State Boards of Nursing (NCSBN), the organization that develops the NCLEX examination, provides helpful information about the test. See the NCSBN Web site (http://www.ncsbn.org) for general information about the test plan, registering for the NCLEX examination, and a listing of available resources.