DRS marker reliability research
Our work used empirical analysis to enhance a tool designed by DRS to ensure quality marking within high stakes examinations. Our analysis has enhanced the potential for the tool to efficiently identify poor markers and remove them from the marking process.
The research was concerned with improving the processes associated with percentage double marking; that is where a marker’s work is periodically checked be peers. Analysis quantified consistency between markers as well as identifying those factors that were associated with erratic marking. From this starting point an optimal process for percentage double marking was designed dealing with issues such as how often checks should be performed and what level of discrepancy between markers should be considered acceptable.