For the image likelihood we chose to relax the assumption that tissue classes could be well-modeled by one or a mixture of a small number of Gaussians that are spatially stationary. Instead, we used a separate model for each structure for each point in space. This allowed us to account for within-structure heterogeneity that occurs commonly in thalamus, hippocampus and elsewhere, but also by keeping the distributions of e.g. pallidum and caudate separately, instead of modeling all “gray matter” together, we were able to use significantly sharper and hence more informative distributions, making the segmentation problem less ambiguous. For the image prior term, like others we used a prior on structure identity given spatial location, however we augmented this to include models of the stereotypical spatial relationships found between anatomical structures. For example, while hippocampus and amygdala are similar in terms of image intensity, their spatial location relative to one another is quite consistent: amygdala is always in front of and above hippocampus, never below or behind it. To encode this, we used a Markov Random Field (MRF) model, which
