OBJECTIVE: Evidence suggests that Immediate Post-Concussion Assessment and Cognitive Testing (ImPACT) validity thresholds may not be adequately sensitive to baseline performances that are not representative of an athlete’s full, uninjured cognitive abilities. The true prevalence of this occurrence is unknown. This study used improvement on post-injury testing (i.e., better performance after the athlete has been removed from play due to suspected concussion than at baseline) to assess the frequency of unrepresentative baseline ImPACT assessments. METHOD: Post-injury ImPACT assessments by NCAA athletes with preceding baseline performance that was considered valid using traditional indices were included. Published reliable change indices (RCI) identified acute post-injury composite scores that improved from baseline. RESULTS: Of 155 post-injury assessments, 68 (43.9%) exhibited reliable improvement from baseline on at least 1 composite score, even after excluding persons with invalid protocols. CONCLUSIONS: Prevalence of ImPACT unrepresentative baseline ImPACT performances may be higher than previously estimated, and many individuals may not be detected by current validity indices. Further research is needed to refine assessment and promote player safety.