At high marker density, the relationship matrix is estimated as \(A=W W'/c\), where \(W_{ik} = X_{ik} + 1 - 2 p_k\) and \(p_k\) is the frequency of the 1 allele at marker k. By using a normalization constant of \(c = 2 \sum_k {p_k (1-p_k)}\), the mean of the diagonal elements is \(1 + f\) (Endelman and Jannink 2012).
The EM imputation algorithm is based on the multivariate normal distribution and was designed for use with GBS (genotyping-by-sequencing) markers, which tend to be high density but with lots of missing data. Details are given in Poland et al. (2012). The EM algorithm stops at iteration \(t\) when the RMS error = \(n^{-1} \|A_{t} - A_{t-1}\|_2\) < tol.
At low marker density (m < n), shrinkage estimation can improve the estimate of the relationship matrix and the accuracy of GEBVs for lines with low accuracy phenotypes (Endelman and Jannink 2012). The shrinkage intensity ranges from 0 (no shrinkage, same estimator as high density formula) to 1 (completely shrunk to \((1+f)I\)). The shrinkage intensity is chosen to minimize the expected mean-squared error and printed to the screen as output.
The shrinkage and EM options are designed for opposite scenarios (low vs. high density) and cannot be used simultaneously.
When the EM algorithm is used, the imputed alleles can lie outside the interval [-1,1]. Polymorphic markers that do not meet the min.MAF and max.missing criteria are not imputed.