Impulsivity has been interpreted as a stable mediator of rate of change in arousal states. To test this hypothesis, 129 subjects differing in impulsivity were given placebo or caffeine at 9:00 a.m. or 7:30 p.m. Recognition memory was tested for the last 20 items from two lists of 24 items and two lists of 80 items. Scores from this paradigm reflect sustained attention, and are thus sensitive to changes in arousal. A four-way interaction between impulsivity, time of day, drug, and prior stimuli (p < .05) indicated that for those given placebo, recognition memory for long and late lists was poorer the higher the impulsivity in the morning; this pattern reversed in the evening. Caffeine reduced recognition errors. These results indicate that impulsivity is not a stable predictor of rate of change in arousal states. Instead, susceptibility to attentional lapses is mediated by impulsivity-related phase differences in diurnal arousal rhythms.