Using Rates of Low Scores to Assess Agreement between Brief Computerized Neuropsychological Assessment Batteries: A Clinically-based Approach for Psychometric Comparisons

Objective: To assess agreement between four brief computerized neurocognitive assessment tools (CNTs), ANAM, CogState, CNS Vital Signs, and ImPACT, by comparing rates of low scores. Methods: Four hundred and six US Army service members (SMs) with and without acute mild traumatic brain injury completed two randomly assigned CNTs with order of administration also randomly assigned. We performed a base rate analysis for each CNT to determine the proportions of SMs in the control and mTBI groups who had various numbers of scores that were 1.0+, 1.5+, and 2.0+ standard deviations below the normative mean. We used these results to identify a hierarchy of low score levels ranging from poorest to least poor performance. We then compared the agreement between every low score level from each CNT pair administered to the SMs. Results: More SMs in the mTBI group had low scores on all CNTs than SMs in the control group. As performance worsened, the association with mTBI became stronger for all CNTs. Most if not all SMs who performed at the worst level on any given CNT also had low scores on the other CNTs they completed but not necessarily at an equally low level. Conclusion: These results suggest that all of the CNTs we examined are broadly similar but still retain some psychometric differences that need to be better understood. Furthermore, the base rates of low scores we present could themselves be useful to clinicians and researchers as a guide for interpreting results from the CNTs.