How to Use Mapping Rules and Criteria-based Marking

About: SecureMarker allows users to define test-level scaling rules, item-based criteria, and weightings against items. Once applied, this opens a marking panel to show this additional information and applied scaling when an examiner marks responses. In Quality Control, you set up your item-level criteria and weightings, then upload the Mapping Rules to the form.

Below are some advice and guidance on using Criteria-based Marking in SecureMarker.

Mapping Rules

The Mapping Rules is a CSV spreadsheet that is uploaded to exam. This spreadsheet determines what values the Examiner will place against the individual criteria and the relative scale for applying the raw marks. 

To upload, select the 'Mapping Rules' button and then upload the file from your files. When uploading, there will be validation against the following:
  • The file is in the CSV format
  • The file contains data only in the first two columns
  • The file contains only numerical data outside the first row
  • The file contains no negative numbers
  • The file contains a mapping of 0 to 0
The file must be uploaded before you release the exam or items for marking and must be uploaded if you wish to use Marking Criteria. Once released, you are unable to change the mapping rules.

See below for an example of Mapping Rules.
Value entered by user Mapped value
5 100
4 80
3 60
2 40
1 20
0 0
The highest value in left column is value or mark that the examiner will input against the critera. In this example, the examiner will be able to award a maximum mark of 5 against each. 

The scale in the left hand column defines a proportion of the total item mark (as defined in Item Authoring) awarded, according to the marker's entered value, before any weighting is applied. 

SecureMarker identifies the highest value in the right column - in this case 100 - and considers that value to be 100% of the item total mark (prior to any weightings). All other numbers in the scale are then considered in relation to, and as a proportion/percentage of this number.

Assigning Criteria to Items

You can assign criteria against manually marked items in a test; it is against these criteria that examiners will then grade responses when marking. You must setup any required marking criteria on items before you release them for marking. 

Once items in a test have been released for marking you won’t be able to add criteria to these items or to edit any existing criteria on released items.

To add criteria, select your desired item from the list in Quality Control, and select the 'Marking Criteria' button.

Enter your criteria in the text entry field and then select 'Add Criteria'. Then assign your weightings in the Weight column.

About Weightings

Next to each criteria on an item, you can add the relative weighting. This is the weighting applied to mark assigned to a response. Weightings must be entered as an integer and every criteria on an item must have a weighting applied.

An example of achieving the same weighting setup in different ways is (using the above example):
Criteria 1 // Weighting: 10
Criteria 2 // Weighting: 10
Criteria 3 // Weighting: 20

This is treated in the same way as:
Criteria 1 // Weighting: 1
Criteria 2 // Weighting: 1
Criteria 3 // Weighting: 2

As the weightings are relative to one another for the items, the examples above behave the same; they are effectively treated as a percentage of the marks assigned.

Each criterion will take a proportion of the marks awarded - Criterion Weighting / Total of all Criteria Weightings

So, in the first example:
Criterion 1 weighting / Total Criteria weights (10 / 40) = 0.25, i.e. 25%

In the second example:
Criterion 1 weighting / Total Criteria weights (1 / 4) = 0.25, i.e. 25%

Marking

During Marking, a 'Marking Panel' will be available to the Examiner to input marks against.

Next to each criteria displayed for the item, the marker has available to them a box to grade the response. The maximum they are allowed to assign to each criteria is the same (defined by the maximum value in column one of the Mapping Rules for the test) and is shown to the right of each entry box.

As the Marker enters marks against criteria, the overall total mark for the response is automatically calculated from the total of all the raw marks (for all criteria marked). The total mark for an item response cannot be separately edited if an item uses criteria-based marking.


For further information on Criteria-based Marking, please consult the full SecureMarker reference guide.

Feedback and Knowledge Base