TI Methods Speaker Series: Harnessing the power of excess statistical significance

About the topic: In this presentation, Tom Stanley will show how excess statistical significance (ESS) can improve the credibility of research synthesis. This will generalize on the notion of an excess of statistical significance by Ioannidis & Trikalinos’ (2007) that formed the basis for their chi-square test for publication bias by accommodating heterogeneity. This version of ESS leads to several new tests for publication selection bias (PSB) that are better than the alternatives as well as a new estimator of mean effect (WILS) that greatly reduces bias and MSE at virtually no cost to efficiency. These methods do not depend on the mechanism or the model of publication selection bias. The findings of these methods cannot be dismissed as a small-study effect because they do not treat small studies differently than large studies nor use, in any way, the correlation of effect size with its SE. Mathematically, PSB→E(ESS) > 0; hence, ESS serves as a good indicator of PSB.

About the speaker: Tom Stanley is Professor of Meta-Analysis at the School of Business and Law, Deakin University, Melbourne Australia. For over 30 years, Tom taught various economics and statistics classes at Hendrix College, as well as the odd history class (e.g., Western Intellectual Traditions). Since the 1980s, his research has focused on the development and application of meta-regression methods and the mitigation of publication bias. Prof. Stanley is the convener of MAER-Net (Meta-Analysis of Economics Research Network) and a member of both DeLMAR (Deakin Lab for the Meta-Analysis of Research) and the Society for Research Synthesis Methods. He is Associate Editor of the Journal of Economic Surveys and Research Synthesis Methods.