Multi-item scales offer two key advantages for creating commensurate measures. First, for identical or harmonized items, we can test whether participants from different studies (and/or subpopulations within studies) respond to the items in the same way. Even if a subset of items fails this test, we may still be able to construct a commensurate measure while adjusting for between-study heterogeneity in the item responses. Second, we can often retain items from individual studies that cannot be harmonized across studies but that nevertheless provide information to improve measurement within a given study. Both of these advantages are realized by using psychometric models. Accordingly, our strategy for creating commensurate measures borrows heavily from work on measurement invariance testing in factor analysis (Meredith, 1993; Millsap & Meredith, 2007; Vandenberg & Lance, 2000; Widaman & Reise, 1997) and on linking and equating test scores in the educational assessment literature (Holland, 2007; Holland & Dorans, 2006). As shown in Figure 1, our analytic guidelines include four key steps: preliminary feasibility analysis, selecting an item set, developing a measurement model, and scoring.
