A common response to posting peer reviews (e.g. eLife, EMBO J) is it's wasted effort because no-one will read them. Well, breaking news: not many people read the paper either...
EMBO estimate ~10% of readers look at the peer reviews. Given a peer review takes ~5 hours to generate and one would certainly conclude a paper takes >50 hours to generate, then a readers/hour-worked calculation surely argues in favor of posting peer reviews - no?
@richardsever @jkpritch It would be better to assess the impact of posting reviews. It’s a good thing for transparency, but I’ve not seen a comment on social media, in a talk, or in a paper stating “this paper claims X, but the reviewers were not convinced” or had a colleague tell me that they were going to cite a paper but didn’t based on posted reviews. Could change, but seems more likely that the stamp of publication (yes, even at eLife) will dominate.
@StearnsLab @richardsever @jkpritch I can give you one micro-use case: when I am prepping a class, journal club, article, or episode of the EMBO podcast, I always look for the peer review files. I think we are definitely underusing transparent review in teaching: both in general and in the specific instance of teaching peer review itself to students and postdocs. In any case, for articles or episodes with a historical focus, having access to the peer review files is great.
@StearnsLab @richardsever @jkpritch For systematic meta-analysis and the larger question of does peer review improve or not research articles a wonderful tool to develop would be one that links specific figures, blocks of text or datasets to reviewer requests and author responses. cc @tlemberger
