We found significant age‐dependent associations of APOE and total hippocampal volume in a large, cross‐sectional, community‐based sample of adults. Before the age of 60, the APOE genotype had no discernible effects on total hippocampal volume. Significant modifying effects of ε3/ε4 and ε4/ε4 were found only in older participants. Specifically, the earliest age of divergence from the trajectory of hippocampal volume loss in ε3/ε3 carriers was detected at age 61 for ε4/ε4 carriers and 69 for ε3/ε4 carriers, demonstrating accelerated volume loss. Although we did not find a significant interaction between APOE ε2/ε3 and age, our findings suggest a potential role of APOE ε2/ε3 in mitigating hippocampal volume loss around age 76. These results underscore the complex interplay between APOE genotype and age in shaping hippocampal volume trajectories.