The aim of this large-scale research was to determine the frequency of valid, invalid, and sandbagging results in ImPACT baseline testing of high school athletes. This retrospective study identified valid, invalid (identified by five embedded Invalidity Indicators), and sandbagging (identified by three “red flags”) results in the ImPACT baseline test scores of 6,346 high school athletes. In addition, the ImPACT postconcussion scores of 266 athletes who sustained a concussion during the school year were evaluated to compare the baseline-to-postconcussion changes of valid versus a combined group of invalid and sandbagging scorers. There were 3,299 (51.99%) athletes who had valid baseline scores, 269 (4.24%) had invalid scores, and 3,009 (47.42%) had sandbagging scores. (There were 231 who obtained both invalidity and sandbagging scores.) The overall difference in baseline-to-postconcussion changes between the valid scorers and the combined group of invalid and sandbagging scorers was statistically significant. The high rate of athletes who had invalid and sandbagging scores raised concern that the underperformance of baseline testing occurs more commonly than is probably realized by those who utilize computerized neuropsychological testing with high school athletes. Accordingly, efforts are needed to improve test administration procedures so that maximal attention and effort can be maintained among the test takers. In the meantime, increased caution is called for in employing the baseline-to-postconcussion paradigm when return-to-play decisions are made.