Dear OB, HR, and RM Colleagues,
Hypothesizing after results are known (HARKing) refers to the questionable research practice of retroactive hypothesis inclusion of an unexpected finding or exclusion of a "failed" prediction. Many of us who were at the SIOP conference just over a week ago attended at least one of the many sessions during which HARKing was mentioned, including the excellent presidential address by Jose Cortina. Although HARKing is considered a questionable research practice, the following fundamental questions remain: What are the effects of HARKing, if any? Does HARKing affect research results and substantive conclusions or is it simply a nuisance? The following article to appear in Personnel Psychology (see the Abstract below) describes empirical evidence based on two separate studies regarding HARKing's downstream impact on reported effect sizes (i.e., correlation coefficients):
· Bosco, F. A., Aguinis, H., Field, J. G., Pierce, C. A., & Dalton, D. R. in press. HARKing's threat to organizational research: Evidence from primary and meta-analytic sources. Personnel Psychology. [available at http://mypage.iu.edu/~haguinis/pubs.html]
We hope that these studies will pave the way for additional research on this topic, which is particularly timely given the challenges raised recently regarding the credibility of our scholarly work.
All the best,
--Herman.
Herman Aguinis
John F. Mee Chair of Management
Professor of Organizational Behavior and Human Resources
Founding and Managing Director, Institute for Global Organizational Effectiveness
Indiana University
Kelley School of Business
http://mypage.iu.edu/~haguinis/
GO FROM MOMENT TO MOMENTUM
Abstract
We assessed presumed consequences of hypothesizing after results are known (HARKing) by contrasting hypothesized versus non-hypothesized effect sizes among 10 common relations in organizational behavior, human resource management, and industrial and organizational psychology research. In Study 1, we analyzed 247 correlations representing nine relations with individual performance in 136 articles published in Journal of Applied Psychology and Personnel Psychology and provide evidence that correlations are significantly larger when hypothesized compared to non-hypothesized. In Study 2, we analyzed 281 effect sizes from a meta-analysis on the job satisfaction-job performance relation and provide evidence that correlations are significantly larger when hypothesized compared to non-hypothesized. In addition, in Study 2, we documented that hypothesized variable pairs are more likely to be mentioned in article titles or abstracts. Also, we ruled out 13 alternative explanations to the presumed HARKing effect pertaining to methodological (e.g., unreliability, publication year, research setting, research design, measure contextualization, publication source) and substantive (e.g., predictor-performance pair, performance measure, satisfaction measure, occupation, job/task complexity) issues. Our results suggest that HARKing seems to pose a threat to research results, substantive conclusions, and practical applications. We offer recommended solutions to the HARKing threat.