I would hope that these forms could be expanded to include “author anonymous” review, where the author is always anonymous to the reviewers and the reviewers are anonymous to the author unless they elect to reveal their identities. This is a version of single-blind review practiced by some humanities journals, including the MLA’s.
Maybe I’m splitting hairs, but I find the phrase ‘ the text’ oddly specific. What about the figures and tables associated with the text. These are also often shared, cited, used by readers. Reviewers also have a role in ensuring the associated figures and tables are accurate
Assuming that your intention here is that articles are published after peer review, there’s no mention here about what factors determine publication (which is discussed in the section above on closed peer review, including the role of editors and Editorial Board)
Also, what about post-publication open peer review as operated on F1000 Platforms? There the articles are published first (with a DOI), and then formal peer review takes place (‘Published review’). However, as the article is already live readers can also comment on the article at the same time (‘Crowd review’). So, this is a hybrid model which is fully open.
Two important points here — thanks fo them.
I think the “factors that determine publication” vary so much from publisher to publisher that no system focused on making peer review more transparent could possibly encompass them. That’s to say, for some publishers the question of marketability is critical in the decision of whether to publish; for others, it isn’t. So it seems that really a theme connected to, but not part of, this effort is establishing the norm of publishers disclosing their evaluative criteria in some way.
Post-publication open peer review is, obviously, increasingly important. And it needs to be captured, somehow, in any information made available about a given scholarly object. The problem, or perhaps conundrum, is that publishers can’t be made responsible for including in the signals they offer on a given work they’ve published reviews over which they’ve exerted no editorial control. Imposing that responsibility would, almost certainly, mean that far fewer publishers would take up the challenge of participating at all in any signaling regime. Most publishers will only be willing to give warrants for those aspects of a publishing process over which they’ve exerted editorial control. That doesn’t mean, however, that a sufficiently modular system could be created under which post-publication reviews could somehow be “connected” to the signals a publisher had applied to any given work…
Do these defined “forms of review” constitute a helpful framework to identify peer review by?
o Research that open reviews (the idea of them) not statistically different than closed review (van Rooyen, doi: 10.1136/bmj.c5729)
o Research that ‘Open Interaction,” “Open Reports,” “Open final-version commenting,” and “Open Participation” are perceived by majority as making peer review “better” (Ross-Hellauer, doi: 10.1371/journal.pone.0189311)
o Four questions to focus on (Tennant, doi: 10.12688/f1000research.12037.3):
§ How can referees receive credit or recognition for their work, and what form should this take?
§ Should referee reports be published alongside manuscripts?
§ Should referees remain anonymous or have their identities disclosed?
§ Should peer review occur prior or subsequent to the publication process (i.e., publish then filter)?
Why not use traits of openness, rather than slightly flexible definitions (like those identified in Ross-Hellauer, doi: 10.12688/f1000research.11369.2)? As it stands, for example, a "Crowd review" would need to be moderated or else it could technically be closed via the identities of the commentators.
Interesting. The difficulty here is that “open” can be an adjective covering many domains — the question of rights (open access) the question of process (open review) or the question of outcomes (content of reviews open to the reader). For the purposes here, we’ll probably leave aside the question of rights — it is theoretically conceivable that there could be an open review process open to the reader that was behind a paywall. We’ve taken the approach of understanding openness from the point of view of the reader, if only because what publishers ultimately do is connect authors to readers.
“We propose definitions of four kinds of closed review”
Is the idea that a publisher’s peer review process would fit into one of the four definitions? It seems that some review processes would fall with definitions 1 and 3, or within 2 and 3, for example.
Also, it’s not clear how the definitions here account for situations where some, but not all, reviewers share their name with the authors. In those cases, isn’t the form of peer review somewhere between single-blind and double-blind?
The idea is that this is a proposal — not a dictat. Our trajectory here is to provide a set of suggestions to the convening groups of publishers — whether they are presses or scholarly societies — to carry forward the conversation we’ve begun. In the background work we undertook and in the conversations we’ve had with publishers, we didn’t come across such a hybrid as you describe here — some reviewers disclosed to the author, some not. But you’re right, this would be a combination of fully closed and partially closed.
“Exchanged review. Reviewers are shown each others’ work, under conditions of anonymity, and respond to the work of other reviewers as well as to the work of the author.”
What is meant by “conditions of anonymity“? (The reviewers wouldn’t necessarily remain anonymous.)
Also, I’m not sure many people would be familiar with the term “exchanged review“.
This is a review practice used by, for example, Science. The reviewers are shown the work of all reviewers, but they aren’t told the identity of either the author or the other reviewers.