There is a consensus that loneliness correlates with an increased risk of cognitive impairment and rapid cognitive decline. However, it has yet to be determined how loneliness influences cognitively healthy aging. This study makes use of the large, nationally representative Canadian Longitudinal Study of Aging (CLSA) to address this question. Based on the baseline and first follow-up datasets collected three years apart (n > 20,000 healthy individuals), we found that higher perceived loneliness predicted decreased scores in the immediate recall test at baseline and in two tests of prospective memory at first follow-up 3 years after baseline. We also examined whether a single-item measurement of loneliness widely used in the field of gerontology, including CLSA, has predictive validity, i.e., can contribute to the prognosis of a future level of cognitive functioning. We found low predictive validity and low test-retest (baseline to follow-up) reliability of this measurement type. These findings impose constraints on proposed accounts of loneliness as a risk factor and methods of examining its relation to cognitive aging.