Non-human Questions are Underutilized Tools in Assessing Higher-Order Cognitive Skills in Human Physiology

Since most physiology core concepts are equally applicable to humans and other multicellular organisms, human physiology instructors may reasonably wonder whether consistent use of non-human assessment questions would enhance students' understanding of human biology, perhaps by offering novel contexts that demand application of human physiology concepts. In this study, we explore this by comparing two types of assessments: test banks from human physiology textbooks, and vetted assessments prioritizing physiology core concept knowledge. We checked for non-human questions and further categorized each non-human example as having No Analogy or Perfect Analogy or Uncertain Analogy to humans, with the latter having potential for assessment of higher-order cognitive skills (HOCS). We found that of nearly 14,000 test bank questions from six different human physiology textbooks, only 53 concerned non-humans and only 6 of those were judged to be Uncertain Analogy. Practice MCAT exams and physiology concept inventories included non-human questions at rates ranging from 11% to 80%. We further classified the non-human questions (n=164) from all sources as assessing lower-order cognitive skills (LOCS) or HOCS and found that on average the Uncertain Analogy bin (n=67) had the highest average percentage of HOCS questions at 61.6%. Our data suggests a mismatch between the potentially high value of non-human questions for HOCS development and their rarity in test banks and we recommend greater inclusion of non-human questions in test banks. For instructors interested in using non-human examples in assessments, Test Question Templates (TQTs), which help generate questions based on lesson learning objectives, may be useful.