We examined abuse of cannabis, stimulants (including cocaine) and sedatives in national Swedish registers in male–male monozygotic and dizygotic twins and near-age full siblings. Model-fitting indicated that these data were best explained by a common pathway model in which the latent liability to DA was highly heritable but also influenced by shared environment. Cannabis, stimulant and sedative abuse all loaded strongly on this common factor with estimates of the total heritability ranging from 64 to 70%. The large preponderance of the genetic risk was non-specific, coming from the common factor as were all of the shared environmental effects. Consistent with prior studies based utilizing personal interviews, most of the genetic risk for abuse of specific classes of psychoactive substance are non-specific. These results suggest that genetic variation in primary sites of action of the psychoactive drugs, which differ substantially across most drug classes, likely play a minor role in human individual differences in risk for DA.