Given a set of M admixed individuals with two ancestral populations (P0 and P1), let the local ancestry for individual i at SNP s, γis ∊ {0,1,2}, be the number alleles inherited from a P1 ancestor. We use a mixed model approach to estimate hγ2 the contribution of variation in local ancestry to phenotypic variation for the phenotype Y=y1, y2, …yM. We first construct a local ancestry based kinship matrix Kγ, which is constructed similarly to the genotyped-based kinship matrix K, but with local ancestry substituted for genotypes at each SNP. We then find the parameters σγ2 and σε2 which maximize the likelihood of the mixed model Y~N(0,Kγσγ2+Iσε2). The heritability explained by local ancestry is given by h2g. Finally, we use the formula hγ2=h22FSTCθ(1−θ) to estimate h2.
