Expectations, observations, and the cognitive processes that bind them: expert assessment of examinee performance

Christina St-Onge*, Martine Chamberland, Annie Lévesque, Lara Varpio

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

15 Scopus citations


Performance-based assessment (PBA) is a valued assessment approach in medical education, be it in a clerkship, residency, or practice context. Raters are intrinsic to PBA and the increased use of PBA has lead to an increased interest in rater cognition. Although several researchers have tackled factors that may influence the variability in rater judgment, the critical examination of rater observation of performance and the translation of that data into judgements are being investigated. The purpose of this study was to qualitatively investigate the cognitive processes of raters, and to create a framework that conceptualizes those processes when raters assess a complex performance. We conducted semi-structured interviews with 11 faculty members (nominated as excellent assessors) from a Department of Medicine to investigate how raters observe, interpret, and translate performance into judgments. The transcribed verbal protocols were analyzed using Constructivist Grounded Theory in order to develop a theoretical model of raters’ assessment processes. Several themes emerged from the data and were grouped according to three macro-level themes describing how the raters balance two sources of data [(1) external sources of information and (2) internal/personal sources of information] by relying on specific cognitive processes to assess an examinee performance. The results from our study demonstrate that assessment is a difficult cognitive task that involves nuance using specific cognitive processes to weigh external and internal data against each other. Our data clearly draws attention to the constant struggle between objectivity and subjectivity that is observed in assessment as illustrated by the importance given to nuancing the examinee’s observed performance.

Original languageEnglish
Pages (from-to)627-642
Number of pages16
JournalAdvances in Health Sciences Education
Issue number3
StatePublished - 1 Aug 2016


  • Cognitive processes
  • Dual-processing theories
  • Rater cognition
  • Rater-based assessment
  • Theory of expertise


Dive into the research topics of 'Expectations, observations, and the cognitive processes that bind them: expert assessment of examinee performance'. Together they form a unique fingerprint.

Cite this