Effect Sizes and Their Calculation (and Miscalculation) - Panel 1 Summary

The discussion session focused on methods for standardizing the language used when reporting effect sizes and the information that is presented in journals and reports. To assist in the reporting of effect sizes, one panelist suggested collaborating with the publishing editors from professional associations to establish guidelines rather than working with individual journal editors. This suggestion was based on the observation that there are disciplinary differences in the reporting of effect sizes and in providing the context for the interpretation. When calculating effect sizes, common errors may result in multilevel models when researchers use statistical software packages without understanding the underlying formulas or functions—particularly if the variance component used is unknown or not clear. The panelists emphasized that effect sizes are not necessary when reporting natural units, though they may be useful for making comparisons across different measures. Also, percentage change may be an appropriate reporting mechanism when the measure has a natural zero point, such as dollars earned, but is of limited use in psychological studies (e.g., cognitive outcomes, affective measures). In conclusion, the suggested guidelines for reporting results included clearly defining the target population, providing a description of the comparability of measures to assess variability for contextual interpretation, and preparing tables containing standard deviations and regression coefficients.