We calculated polygenic scores from GWAS of each latent genomic factor (Externalizing, Behavioral Disinhibition, and SUD) among EUR individuals from the Collaborative Study on the Genetics of Alcoholism (COGA; N =7,530), a multi-site family-based study39 and All of Us (Nmax = 77,442), a national cohort study40,41. We also calculated PGS for the residual SUD effects (for alcohol, opioid, tobacco, and cannabis use disorders) in COGA. These analyses allowed us to compare the total variance explained by each of the PGS and identify the specificity, or loss thereof, in polygenic prediction of SUDs. In COGA, we report associations with alcohol use disorder (AUD), cannabis use disorder (CUD), opioid use disorder (OUD), and symptoms from the Fagerstrom Test for Nicotine Dependence (FTND)42. In All of Us, we report associations with AUD, tobacco use disorder (TUD), and drug use disorder (DUD), which includes all other illicit substances other than alcohol and tobacco.