On the multivariate distribution of effect size estimates from single-case experimental designs

Many different effect size metrics have been proposed for use with single-case experimental designs (SCEDs). Metrics that have known sampling variances and are suitable for meta-analysis include the within-case standardized mean difference (SMD), the log response ratio (LRR), and the non-overlap of all pairs (NAP). These within-case effect size metrics can be used to make comparisons between pairs of phases within an SCED for a single outcome. However, in practice, many SCEDs include multiple outcomes, multiple phases, or both multiple outcomes and multiple phases. In such studies, it may be useful to estimate multiple effect sizes for inclusion in a meta-analysis. This requires calculation not only of effect size estimates and their sampling variances, but also the covariances between effect size estimates. Formulas for the covariances between effect size estimates are available but scattered around the methodological literature. This paper reviews and consolidates available formulas and demonstrates their relevance in the context of meta-analysis of SCEDs. I describe methods for estimating multiple effect sizes, along with corresponding sampling variances and covariances, for the within-case SMD, LRR, and NAP indices. An empirical example is included to illustrate the calculations.