Lateral segregation and the formation of lateral domains are well-known phenomena in ternary lipid bilayers composed of an unsaturated (low gel-to-liquid phase transition temperature (Tm)) phospholipid, a saturated (high-Tm) phospholipid, and cholesterol. The formation of lateral domains has been shown to be influenced by differences in phospholipid acyl chain unsaturation and length. Recently, we also showed that differential interactions of cholesterol with low- and high-Tm phospholipids in the bilayer can facilitate phospholipid segregation. Now, we have investigated phospholipid-cholesterol interactions and their role in lateral segregation in ternary bilayers composed of different unsaturated phosphatidylcholines (PCs) with varying acyl chain lengths, N-palmitoyl-D-erythro-sphingomyelin (PSM), and cholesterol. Using deuterium NMR spectroscopy, we determined how PSM was influenced by the acyl chain composition in surrounding PC environments and correlated this with the affinity of cholestatrienol (a fluorescent cholesterol analog) for PSM in the different PC environments. Results from a combination of time-resolved fluorescence measurements of trans-parinaric acid and Förster resonance energy transfer experiments showed that the relative affinity of cholesterol for phospholipids determined the degree to which the sterol promoted domain formation. From Förster resonance energy transfer, deuterium NMR, and differential scanning calorimetry results, it was clear that cholesterol also influenced both the thermostability of the domains and the degree of order in and outside the PSM-rich domains. The results of this study have shown that the affinity of cholesterol for both low-Tm and high-Tm phospholipids and the effects of low- and high-Tm phospholipids on each other influence both lateral structure and domain properties in complex bilayers. We envision that similar effects also contribute to lateral heterogeneity in even more complex biological membranes.