Review should contain a recommendation
hadley opened this issue · 10 comments
Here are some things that your comments to the authors should not contain:
- A recommendation of whether to accept or reject the paper
Isn't that the most important part of the review? (the tl;dr if you will)
I think this is a common thing that people do but I feel pretty strongly that they shouldn't. The AE/Editor will be making the decision about whether to accept or reject. The comments that you give the authors should be useful either way.
In the comments to the editor you can make a case for accepting or rejecting.
As an editor, I really appreciate it when the reviewer sums up their overall opinion into a single overall recommendation. I think you're mistaken to advise against it.
That can absolutely be in the comments to the editor. But it should absolutely not be in the comments to the authors. It is the wrong place to put it - your comments for the authors are meant to be separate from the recommendation. They are different components of the process.
The comments to the authors are meant to be the place where you point out specific, addressable (or not) issues with their paper.
I think this is a difficult point to argument because we have different underlying driving principles. For example, I never put anything in the comments to the editors box because I don't think it's fair to have a privileged channel of communication that the authors does not see. I want my review to be as transparent as possible so there are no mysteries to the author.
This matches my experience as an editor/AE where I see very few people providing separate comments to the authors and to the AE. Maybe it varies by discipline?
A couple of points:
#1 You will find that editors at most journals will ask you specifically not to record your recommendation in your comments to the authors.
#2 One reason for this is that the editor/AE are frequently the only unblinded individuals in the process. If both referees make their recommendations known and they conflict (for example) then the editor has to chose between them. The author will see this and have unblinded information about the editor's choice. In general this doesn't seem bad, but editors frequently have to make hard/unpopular decisions and it isn't fair to subject them to unblinded scrutiny.
#3 Another reason is that you eliminate degrees of freedom in the review process if both authors make their recommendations known. Since reviewers may have conflicts of interest, may be biased, or may ask for
unreasonable revisions I prefer to keep the discretion of the editor in play.
#4 Another reason not to do it is convention. I'm less convinced by that argument.
#5 I did add this sentence under comments to the editors, "It should not very consistent with the comments for the authors; if you aren't comfortable saying it to the authors directly, you should consider carefully whether it should be said. "
(1) I think this must be subdiscipline specific, because I have never been asked not to record my recommendations (also this this just argument (4) in another bullet point.
(2) I don't find this argument compelling because if you can't commit to and justify your opinions as an (A)E, you shouldn't be doing it.
(3) I don't find this argument compelling either because the reviewers can still ask for unreasonable revisions regardless of their overall recommendation. It's the job of the AE to synthesise the comments from the reviewers into an overall recommendation and required changes.
(Also regardless of our disagreements on the finer points, I think it's great to be having this discussion in the open!)
Re: (1) I have seen instructions/heard this statement in statistics journals (traditional and new age), genomics journals, biology journals, and computational biology/software journals. Which sub-disciplines are you talking about?
Re: (2) This is a fine sentiment in theory. But it breaks down in practice where many AEs are still relatively junior and could have their careers seriously damaged by having to reject papers by senior authors in the field.
Re; (3) Here is a concrete scenario - an AE recruits two referees in the area. They both ask for extreme things and say "reject this paper" in their reports. The AE says, "ignore those rejects and actually just do items 1-4 in report #1 and 4,5, and 8 in report #2". This puts the authors in a weird position (both referees said reject but I'm still doing revisions?), it puts the AE in a weird position (both referees said reject in the comments I'm sending out, but I'm saying its not rejected in an email) and leaves the whole thing a little unsteady.
I prefer that the reviewers focus on issues they find and more importantly ways the paper can be made better/worse. They give a recommendation to the AE - but the AE/editor make the decision about accepting/rejecting.
Agreed it is great to have this discussion publicly.
(1) Weird - but I publish mostly in stat computing/graphics/visualisation journals, so maybe we just have non-overlapping experiences
(2) That seems like an argument for blinded AEs - which, again, in my experience, is common
(3) Again, I don't see why that is weird. Even if the referees don't include recommendations, they might ask for a whole lot of unreasonable things that the AE says the author doesn't need to do. Why does including a recommendation make a difference?
I agree with you that the focus of the review should be on how to make the paper better, or failing that a discussion of why it is fundamentally flawed. But I think a key part of the review is the overall recommendation (including brief justification), and it's something the author should see, as well as the AE.
What I wonder is if there are imaginable situations where the advice w.r.t. accept/reject is not obvious from the other comments. I would think that if it is not clear, you need to go back and better explain your issues.
(additional data point on item 1; I have never received such instructions in logic journals).