How to Use IRT Scoring

About: You can configure a test in Surpass to generate an IRT theta value as a candidate’s score, which can be used to generate a grade, scale score, and pass/fail information. This article will explain how to set up your test for IRT scoring.

Step 1: Create Tag Groups

First, you need to create IRT tag groups for your subject, so you can apply IRT values to your items.
In the ‘Setup’ section of Surpass, go to the ‘Subjects’ tab.
Select your subject from the list to open the subject information on the right. Select the ‘Tag Groups’ tab to view a list of the tag groups associated with the subject.

Select the ‘Create Tag Group’ button to create new tag groups.
The tag groups you can create to support IRT scoring are:
  • IRT_A
  • IRT_B
  • IRT_C
  • IRT_Domain
IRT_B is the only mandatory tag group you must create to support IRT scoring and this must be created as a numeric tag group.
You can also create the IRT_Domain tag group containing tag values for which you want to generate an IRT theta value for statistical analysis.
For more information about each tag group, please see the IRT Tag Groups section of the ‘Subject Tags and Tag Groups’ help page in the in-built help.

Step 2: Add tags to items

When editing items in ‘Item Authoring’, you must manually add IRT theta values as tags to items you want to use to generate an IRT score for a candidate. 

Note: You must generate IRT theta values for items before you can add IRT tag values. You must add the tag values manually – these will not be updated automatically in Surpass. 

To do this, go to the ‘Edit item settings and tags’ section of the item editing screen. Here you can see the tag groups you have created for your subject.
Select a tag group to add a tag value.
To set up IRT scoring for an item, you must ensure you apply at least an IRT theta value for the IRT_B tag group. The value can be positive or negative.

Step 3: Apply IRT grade boundaries

In ‘Test Creation’, you can apply grade boundaries at a test form level based on a candidate’s IRT score.
When editing your test form, select the ‘Test Form Details’ tab.
Here, you can ‘Edit’ grade boundaries for your test form. Selecting this button will open a new window where you can add or remove grade boundaries.
Select the ‘Grade Boundary Type’ drop-down menu and select ‘IRT Score’. You can specify an IRT score for each grade boundary. You can also edit the values for the ‘IRT Range’, which will apply if the candidate answers everything either correctly or incorrectly, or if they do not attempt any questions.

All fields support positive and negative values up to 4 d.p.

Step 4: Apply IRT section pass mark

To set an IRT score as a pass mark for a section, select your chosen section and then select the ‘Edit’ button to open the ‘Section Properties’.
Change the ‘Pass Mark’ type to ‘IRT Score’ using the drop-down menu and enter the IRT value you want to set as the pass mark for the section.
In delivery, the candidate must achieve the pass mark for all sections in order to pass the test.

Step 5: Apply IRT scale score

In ‘Test Form Details’, you can also apply a scale score mapping file to map IRT scores on to a scale. If necessary, this scale score can be edited in the ‘Rescoring’ tab of ‘Reporting’ once the test has been sat.
Read this article to find out more about applying a scale score mapping file.

Step 6: Retrieve a candidate's IRT score

Once a candidate sits a test, you can retrieve their IRT score from the following resources in the API:
  • Result
  • AnalyticsResult
  • HistoricalResult
For more information about the API resources, please read this section of the Knowledge Base.

Feedback and Knowledge Base