Objective: With the widespread use of computer-based screening measures, the need for a neuropsychologist to be present during baseline neurocognitive assessment of athletes has been continuously debated. In this study, we examined the relationship between athletes’ subjective feedback regarding environmental/test distractions and their baseline test performances. Methods: Data from sample of approximately 1,800 high school athletes was inspected for comments provided at the end of baseline neurocognitive testing with the ImPACT test battery. Scores from 231 athletes (12%) were removed due to questionable motivation or validity. In the resultant sample of 1,659, athletes reported problems related to environmental distractions (N ¼ 161), computer problems (N ¼ 199), and issues with test instructions (N ¼ 297). Results: Analyses of variance revealed that athletes reporting environmental distractions endorsed significantly more symptoms, as compared to athletes not providing feedback (p , .001, d ¼ .21), athletes reporting computer problems performed significantly faster on the Reaction Time composite (p ¼ .0054, d ¼ .14), and athletes reporting issues with test instructions endorsed significantly more symptoms (p , .001, d ¼ .19). Conclusions: Results demonstrate that subjective environmental, computer, and test instruction complaints have an important role in the evaluation of athletes and in the interpretation of test results. By virtue of professional training, neuropsychologists are skilled in testing the limits and conducting an inquiry phase during patient testing. Athletes completing computer-based neurocognitive measures, even in group settings, should receive the same attention and opportunity to provide feedback as patients receive during routine neuropsychologic assessment.