Appendix 2: A Proposed Form of Signaling

Appendix 2: A Proposed Form of Signaling

Published onMar 28, 2018

We present here a series of proposed icons corresponding to the definitions set out in Appendix 1 above. We show as well a conceptual notion of how these icons could be arranged so as to communicate simply and clearly to readers the process by which various scholarly objects comprising a publication had been reviewed.

The basic peer review icon is a hexagon. Used by itself it would indicate to a reader that the work had been peer reviewed, but what not disclose what elements of the work had been reviewed or how the review had been undertaken.

Forms of closed review

Partially closed (“single blind”) review.

Cross review in the case of a partially closed review.

Cross review in the case of a fully closed review.

Forms of open review

Published review. Reviewer’s reports are openly published and signed by reviewers; they may be cited in subsequent scholarship.

Crowd review. Access to the commenting platform may be controlled, but the publisher does not choose who comments.

Managed crowd review. An open platform for reviewing the object is created, but a limited number of reviewers are commissioned by the publisher. They may or may not be identified as such to the community of reviewers.

Examples of the system in use

A manuscript with openly published reviews.

A proposal, subjected to fully closed review; a subsequent manuscript subjected to peer-to-peer review.

A manuscript subjected to fully closed cross review, with an attendant dataset subjected to partially closed cross review.

A manuscript simultaneously subjected to crowd review and partially closed review.

License

Creative Commons Attribution 4.0 International License (CC-BY 4.0)

Comments

3

?

Login to discuss

?

Sabina Alam:

I agree with Andy’s comments - I was about to write pretty much the same thing here! For the signals to work, and to be widely adopted/understood I think the ‘less is more’ approach is better. For example, do we really need the P, MS and DS, if the ‘historical’ overview isn’t being provided? Also, if you’re showing that something has received either closed or open review, doesn’t ‘PR’ become redundant?

I’m not sure what the solution is here, but my initial reaction is that as signals these are quite complex, and having too many badges could create more confusion

?

Nick Michal:

PeErhaps I missed this, but are you proposing an organization to oversee the meriting/auditing of these badges, like an equivalent to COPE? Or is this a system that orgs/journals can adopt, like OSF badges?

Andy Collings:

I think there needs to be discussion around the labels and definitions of the forms of peer review, before the signals are discussed. However, in general, these signals are perhaps overly complex and would probably not be intuitive to readers.