Emotional content is known to enhance memory in a content-dependent manner in healthy populations. In middle-aged and older adults, a reduced preference for negative material, or even an enhanced preference for positive material has been observed. This preference seems to be modulated by the emotional arousal that the material evokes. The neuroanatomical basis for emotional memory processes is, however, not well understood in middle-aged and older healthy people. Previous research on local gray matter correlates of emotional memory in older populations has mainly been conducted with patients suffering from various neurodegenerative diseases. To our knowledge, this is the first study to examine regional gray matter correlates of immediate free recall and recognition memory of intentionally encoded positive, negative, and emotionally neutral words using voxel-based morphometry (VBM) in a sample of 50-to-79-year-old cognitively intact normal adults. The behavioral analyses yielded a positivity bias in recognition memory, but not in immediate free recall. No associations with memory performance emerged from the region-of-interest (ROI) analyses using amygdalar and hippocampal volumes. Controlling for total intracranial volume, age, and gender, the whole-brain VBM analyses showed statistically significant associations between immediate free recall of negative words and volumes in various frontal regions, between immediate free recall of positive words and cerebellar volume, and between recognition memory of positive words and primary visual cortex volume. The findings indicate that the neural areas subserving memory for emotion-laden information encompass posterior brain areas, including the cerebellum, and that memory for emotion-laden information may be driven by cognitive control functions.