Using the Critical Incident Questionnaire as a Formative Evaluation Tool to Inform Online Course Design: A Qualitative Study

Anita Samuel*, Simone C.O. Conceição

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

2 Scopus citations


The online instructor plays a prominent role in influencing how students respond to an online course, from designing the course structure, course activities, and assignments to encouraging interaction. Therefore, to develop effective online courses, instructors need robust feedback on their design strategies. Student evaluation of teaching (SET) functions as a summative evaluation of the course design and delivery. Yet, the feedback from SETs can only be integrated into the next iteration of the course, thereby failing to benefit the students who provide the feedback. One suggestion is to use midsemester formative evaluation to inform course design in real time. A qualitative research study was conducted to explore whether the Critical Incident Questionnaire (CIQ) could be an effective formative evaluative tool to inform real-time online course design and delivery. Thematic analysis was conducted on the midcourse evaluations obtained from 70 students in six fully online master's level courses. There are three key findings from this study. First, CIQ use can provide opportunities for real-time adjustments to online course design and inform future redesign of online courses. Second, responses received via the CIQ prioritize the student voice and experience by focusing on factors that are critical to them. Finally, this deep-dive analysis reinforces the enduring factors that contribute to effective online course design and delivery. A recommendation for practice is to use the CIQ as an effective tool to gather formative feedback from students. This feedback can then be used to adjust course design as needed.

Original languageEnglish
Pages (from-to)151-169
Number of pages19
JournalInternational Review of Research in Open and Distance Learning
Issue number2
StatePublished - May 2022
Externally publishedYes


  • Critical Incident Questionnaire
  • formative assessment
  • online course design
  • student evaluation of teaching


Dive into the research topics of 'Using the Critical Incident Questionnaire as a Formative Evaluation Tool to Inform Online Course Design: A Qualitative Study'. Together they form a unique fingerprint.

Cite this