psu-psychology/transparency

Department-wide conversation about transparency and reproducibility

transparency

This is the repo supporting the Penn State Psychology Department's conversation about transparency, openness and reproducibility in psychological research.

Peer-reviewed papers and commentaries

• Collaboration, O. S. (2015). Estimating the reproducibility of psychological Science, 349(6251), aac4716. http://doi.org/10.1126/science.aac4716.
  • Bohannon, J. (2015). Many psychology papers fail replication test. Science, 349(6251), 910–911. http://doi.org/10.1126/science.349.6251.910.
  • Barrett, L. F. (2015, September 1). Psychology Is Not in Crisis. The New York Times. Retrieved from http://www.nytimes.com/2015/09/01/opinion/psychology-is-not-in-crisis.html
    • Caplan, A. (2015, September 8). How Reliable Are Psychology Studies? The New York Times. Retrieved from http://www.nytimes.com/2015/09/08/opinion/how-reliable-are-psychology-studies.html
  • Frank, M. (2015, August 31). Babies Learning Language: The slower, harder ways to increase reproducibility. Retrieved from http://babieslearninglanguage.blogspot.com/2015/08/the-slower-harder-ways-to-increase.html.
  • Srivastava, S. (2015, September 2). Moderator interpretations of the Reproducibility Project. Retrieved from https://hardsci.wordpress.com/2015/09/02/moderator-interpretations-of-the-reproducibility-project/
  • Earp, B.D. (2015, September 2). Psychology Is Not in Crisis? Depends on What You Mean by "Crisis". Retrieved from http://www.huffingtonpost.com/brian-earp/psychology-is-not-in-crisis_b_8077522.html.
  • Abrams, D., Clayton, S., & Crandall, C. (n.d.). The Reproducibility Crisis: Less Than Meets the Eye. Retrieved September 20, 2015, from https://www.psychologytoday.com/blog/sound-science-sound-policy/201509/the-reproducibility-crisis-less-meets-the-eye
  • Bench, S. W., Rivera, G. N., Schlegel, R. J., Hicks, J. A., & Lench, H. C. (2017). Does expertise matter in replication? An examination of the reproducibility project: Psychology. Journal of Experimental Social Psychology, 68, 181–184. https://doi.org/10.1016/j.jesp.2016.07.003
• Bavel, J. J. V., Mende-Siedlecki, P., Brady, W. J., & Reinero, D. A. (2016a). Contextual sensitivity in scientific reproducibility. Proceedings of the National Academy of Sciences, 113(23), 6454–6459. https://doi.org/10.1073/pnas.1521897113
  • Inbar, Y. (2016). Association between contextual dependence and replicability in psychology may be spurious. Proceedings of the National Academy of Sciences, 113(34), E4933–E4934. https://doi.org/10.1073/pnas.1608676113
  • Bavel, J. J. V., Mende-Siedlecki, P., Brady, W. J., & Reinero, D. A. (2016b). Reply to Inbar: Contextual sensitivity helps explain the reproducibility gap between social and cognitive psychology. Proceedings of the National Academy of Sciences, 113(34), E4935–E4936. https://doi.org/10.1073/pnas.1609700113
• Etz, A., & Vandekerckhove, J. (2016). A Bayesian Perspective on the Reproducibility Project: Psychology. PLOS ONE, 11(2), e0149794. https://doi.org/10.1371/journal.pone.0149794
• Nosek, B. A., Alter, G., Banks, G. C., Borsboom, D., Bowman, S. D., Breckler, S. J., … Yarkoni, T. (2015). Promoting an open research culture. Science, 348(6242), 1422–1425. http://doi.org/10.1126/science.aab2374. Suggests guidelines for journals.
• Adolph, K. E., Gilmore, R. O., Freeman, C., Sanderson, P., & Millman, D. (2012). Toward Open Behavioral Science. Psychological Inquiry, 23(3), 244–247. http://doi.org/10.1080/1047840X.2012.705133.
• Nuijten, M. B., Hartgerink, C. H. J., Assen, M. A. L. M. van, Epskamp, S., & Wicherts, J. M. (2015). The prevalence of statistical reporting errors in psychology (1985–2013). Behavior Research Methods, 1–22. http://doi.org/10.3758/s13428-015-0664-2.
  • Reply by Schmidt, T. (2016). Sources of false positives and false negatives in the STATCHECK algorithm: Reply to Nuijten et al. (2015). arXiv:1610.01010 [q-Bio]. Retrieved from http://arxiv.org/abs/1610.01010
• Macleod, M. R., Lawson McLean, A., Kyriakopoulou, A., Serghiou, S., de Wilde, A., Sherratt, N., … Sena, E. S. (2015). Risk of Bias in Reports of In Vivo Research: A Focus for Improvement. PLoS Biol, 13(10), e1002273. http://doi.org/10.1371/journal.pbio.1002273.
• Vanpaemel, W., Vermorgen, M., Deriemaecker, L., & Storms, G. (2015). Are We Wasting a Good Crisis? The Availability of Psychological Research Data after the Storm. Collabra, 1(1). http://doi.org/10.1525/collabra.13.
• Shaw, D. M., & Erren, T. C. (2015). Ten Simple Rules for Protecting Research Integrity. PLoS Comput Biol, 11(10), e1004388. http://doi.org/10.1371/journal.pcbi.1004388.
• Shanks DR, Vadillo MA, Riedel B, Clymo A, Govind S, Hickin N, Tamman AJ, & Puhlmann LM (2015). Romance, Risk, and Replication: Can Consumer Choices and Risk-Taking Be Primed by Mating Motives? Journal of experimental psychology. General PMID: 26501730.
  • Neuroskeptic. (2015, November 15). Reproducibility Crisis: The plot thickens. Retrieved November 15, 2015 from http://blogs.discovermagazine.com/neuroskeptic/2015/11/10/reproducibility-crisis-the-plot-thickens/#.VkS7loT9mzf
• Other views on reproducibility and data sharing
  • Gelman, A. (2015, September 14). Why aren’t people sharing their data and code? Retrieved September 20, 2015, from http://andrewgelman.com/2015/09/14/its-not-so-easy-to-share-data-and-code-and-there-are-lots-of-bureaucrats-who-spend-their-time-making-it-even-more-difficult/
  • Gelman, Andrew. (2015, September 16). The aching desire for regular scientific breakthroughs. Retrieved September 20, 2015, from http://andrewgelman.com/2015/09/16/harsh/
  • Eisenberg, N. (2015, June). Thoughts on the Future of Data Sharing - Association for Psychological Science. Retrieved from https://www.psychologicalscience.org/index.php/publications/observer/2015/may-june-15/thoughts-on-the-future-of-data-sharing.html
  • Bishop, Dorothy. (2015, July 11). BishopBlog: Publishing replication failures: some lessons from history. Retrieved from http://deevybee.blogspot.com/2015/07/publishing-replication-failures-some.html
  • Ascoli, G. A. (2006). The ups and downs of neuroscience shares. Neuroinformatics, 4(3), 213–215. http://doi.org/10.1385/NI:4:3:213
  • Bishop, D. (2015, October 29). How to make biomedical research more reproducible. Nature. http://dx.doi.org/10.1038/nature.2015.18684.
  • Johnson, C. Y. (2015, October 15). What happened when a group of researchers tried to repeat a headline-grabbing study. The Washington Post. Retrieved from https://www.washingtonpost.com/news/wonkblog/wp/2015/10/15/what-happened-when-a-group-of-researchers-tried-to-repeat-a-headline-grabbing-study/.
  • Leek, J. (2015, October 1). A glass half full interpretation of the replicability of psychological science | Simply Statistics. Retrieved from http://simplystatistics.org/2015/10/01/a-glass-half-full-interpretation-of-the-replicability-of-psychological-science/.
  • Nagel, T. (2015, October 8). New approach could help reduce bias in research, Stanford scholar says. Retrieved November 2, 2015, from http://news.stanford.edu/news/2015/october/research-analysis-maccoun-100815.html.
  • Guerra, S. (2015, September 28). Reproduce or Bust?!: Bringing Reproducibility Back to Center Stage | Science in the News. Retrieved from http://sitn.hms.harvard.edu/flash/2015/reproduce-or-bust-bringing-reproducibility-back-to-center-stage/.
  • Earp, B. D. (2015, September 2). Psychology Is Not in Crisis? Depends on What You Mean by "Crisis". Retrieved November 2, 2015, from http://www.huffingtonpost.com/brian-earp/psychology-is-not-in-crisis_b_8077522.html.
  • Makel, J. P. & M. (2015, November 4). The Painful Necessity of Replicating Research - Education Week. Education Week. Retrieved from http://www.edweek.org/ew/articles/2015/11/04/the-painful-necessity-of-replicating-research.html.
  • Lindsay, D. S. (2015). Replication in Psychological Science. Psychological Science, 0956797615616374. http://doi.org/10.1177/0956797615616374.
  • politicalsciencereplication.wordpress.com. (n.d.). Political Scientists Trying to Delay Research Transparency. Retrieved from https://politicalsciencereplication.wordpress.com/2015/11/07/political-scientists-trying-to-delay-research-transparency/
  • Ortman, Andreas (2015, November 2). The replication crisis has engulfed economics. Retrieved November 10, 2015, from http://theconversation.com/the-replication-crisis-has-engulfed-economics-49202.

Resources

• The Center for Open Science
  • Open Science Framework (OSF)
  • Transparency and Openness Promotion (TOP) Guidelines.
• Databrary.
  • Open science, data sharing, analysis tool links.
  • Data sharing manifesto
• ICPSR.
• ScholarSphere.
• American Psychological Association (2015). Data Sharing: Principles and Considerations for Policy Development. Retrieved August 25, 2015, from http://www.apa.org/science/leadership/bsa/data-sharing-report.aspx
• PubPeer