Estimates of Uk from this EM algorithm are sensitive to initialization. Furthermore, we noticed an interesting feature of the EM algorithm: each iteration preserves the rank of the matrices Uk, so that ranks of the estimated matrices are the same as the ranks of the matrices used to initialize the algorithm. We exploited this fact to ensure that some of the estimated Uk had low rank. This helped stabilize the estimates since rank-penalization is one way to regularize covariance matrix estimation.
