• Home
  • Development of Heparin and HS Glycotherapeutics
  • Lab
  • People

Ferniglab Blog

The personal blog of Dave Fernig, thoughts on science and unrelated matters

Feeds:
Posts
Comments
« A milestone
Quentin Nunes, PhD »

The real problems of post publication peer review

January 27, 2015 by ferniglab


Discussion surrounding post publication peer review (previous post here seems to be growing and one issue that is frequently raised is anonymity. In a PLOS Medicine editorial Hilda Bastian argues that current post publication peer review is over focussed on what apparently is wrong in papers and that anonymity is a threat to effective post publication peer review.
A PubPeer thread takes issue with these and some other points and I have also joined in (I am Peer2). We should remember that any notion of power has nothing to do with scientific capability – indeed there may even be an inverse relation. So providing those with the least power (so the most disenfranchised) a means to participate in post publication peer review is essential. Though we have no data on PubPeer, PubMed Commons is a venue for the established. There are some critiques, there is also a fair amount of hagiography too. I would hazard a guess that PubPeer is far more diverse in terms of the career stage of participants and in terms of their gender/social group. Certainly my anecdotal evidence suggests as much, and that is all I have to go on.

Anonymity is also linked to the idea that post publication peer review is heading into a swamp of negativity. Constructive engagement means that a paper has an impact on your research. That is, the data and/or their interpretation alter your world view. In other words a good paper. What happens next? This may range from seeing your own data differently to collaboration. The latter may involve putting together a new synthesis of a field, something I am attempting to do with colleagues. This paticular new synthesis will be published (open access, naturally) and will hopefully garner comments, critique and so on. We have new platforms for this, e.g., The Winnower, Science Open, which I find very exciting.

There are other forms of constructive engagement, and I would put my comment here and here in this category.

The above illustrates the major problem facing post publication peer review: a lack of engagement by authors. Even an acknowledgement along the lines of “Thanks, we are thinking about this” is unusual.

There is no need to reward the transaction from the point of view of the person posting a comment – critical reading is part of our daily fare. There is no need to reward the authors of the paper: they have a publication. There is a need for authors to realise that they absolutely must engage, otherwise their work will lose credibility. I think the problem of lack of engagement is so substantial that currently author engagement is the surprise, not the norm. Recent examples of author engagement are seen in the threads relating to a paper by Sophien Kamoun (it is worth noting that after what must have been an initial shock Sophien Kamoun and his lab were encouraged by the positive response of peers), and by Jim Carrington.

So critiques are just that. Civility and tone lower a little at times when there is no engagement by authors and peers find that there isn’t just one paper, but a set by an author with analogous critiques. This is certainly common in threads relating to multiple instances of gel splicing (why I think this particular practice is wrong is here). Nonetheless, the frustration of peers is kept pretty much under control, to their credit

I think that this is reasonable, after all we are engaged in science, not in a matter of faith, though as ever, people should stick to the data and let them speak for themselves. There are a number of labs whose papers I not longer bother reading, precisely because a substantial part of their oeuvre has important critiques, mirrored across multiple papers with no response. This looks to me like a potential class of “Pathological Science” and there is so much to read that I need as many filters as possible.

The real problems with anonymity are sockpuppetry and gish galloping, which occur more frequently than incivility. As Peer1 pointed out these are unavoidable. They are easy to identify and so one is free to enjoy the lengths people may go to defend the indefensible (see here for an example) or to skip to the next comment in the thread.

Anonymity is good, needs protecting and under threat. Lets keep it until such time as science no longer needs it.

Advertisement

Share this:

  • Twitter
  • Facebook
  • LinkedIn

Like this:

Like Loading...

Related

Posted in Post publication peer review, Research integrity, Science process, Science publishing | Tagged research, Research integrity, science, Science fraud, Science progress | 2 Comments

2 Responses

  1. on April 22, 2015 at 3:41 pm Blogroll: Comment etiquette : The Sceptical Chymist

    […] are typically associated with negativity, rather than constructive engagement. Nevertheless, Fernig argues the case for anonymity, stating that without it the academics with the least power (for example, early-career researchers) […]


  2. on May 28, 2015 at 12:55 pm Responding to questions raised on PubPeer | Ferniglab Blog

    […] and the wider community, something I have discussed in a number of posts (two examples being here and here). Two days ago, My colleague Mike Cross came by my office, having just delivered a pile of […]



Comments are closed.

  • Places of interest

    The one and only PhD comics, the guide to being a graduate and to mentoring.

    Improbable Research and the Ig Nobels

    Retraction Watch provides updates on retractions of articles.

    Office for Research Integrity, their video should be compulsory for all.

    Centre for Alternative Technology

    Lateral Science, has some quite stunning information - well worth a browse.

    Fascinating places that have been closed by lawyers

    Science Fraud, shut down due to legal threats on Jan 3 2013. and Abnormal Science

  • Blogroll

    • WordPress.com
    • WordPress.org
  • Funding agencies

    • Biotechnology and Biological Sciences Research Council
    • Cancer and Polio Research Fund
    • Engineering and Physical Sciences Research Council
    • Liverpool Pancreas NIHR Biomedical Research Unit
    • Medical Research Council
    • North West Cancer Research
  • Seminars

    • Cancer Research UK Centre
  • January 2015
    M T W T F S S
     1234
    567891011
    12131415161718
    19202122232425
    262728293031  
    « Dec   Feb »
  • Archives

    • November 2022
    • July 2022
    • April 2022
    • March 2022
    • May 2021
    • March 2021
    • August 2020
    • June 2020
    • April 2020
    • March 2020
    • December 2019
    • October 2019
    • July 2019
    • May 2019
    • April 2019
    • January 2019
    • September 2018
    • August 2018
    • May 2018
    • April 2018
    • February 2018
    • January 2018
    • December 2017
    • November 2017
    • July 2017
    • June 2017
    • April 2017
    • March 2017
    • January 2017
    • October 2016
    • June 2016
    • May 2016
    • March 2016
    • January 2016
    • December 2015
    • July 2015
    • June 2015
    • May 2015
    • March 2015
    • February 2015
    • January 2015
    • December 2014
    • November 2014
    • October 2014
    • September 2014
    • August 2014
    • July 2014
    • June 2014
    • February 2014
    • January 2014
    • December 2013
    • October 2013
    • September 2013
    • August 2013
    • July 2013
    • June 2013
    • May 2013
    • April 2013
    • March 2013
    • February 2013
    • January 2013
    • December 2012
    • November 2012
    • October 2012
    • September 2012
    • August 2012
    • July 2012
    • June 2012
    • May 2012
    • April 2012
    • March 2012
    • February 2012
    • January 2012
    • November 2011
    • October 2011
    • September 2011
    • August 2011
    • July 2011
    • June 2011
    • May 2011
    • April 2011
    • March 2011
    • February 2011
    • January 2011
    • December 2010
    • November 2010
    • October 2010
    • September 2010
    • August 2010
  • Follow me on Twitter

    My Tweets
  • Cloud

    American Civil War antithrombin III banana Biological imaging biotechnology Brexit Choanoflagellates chocolate chondroitin sulfate coagulation Confederate States covid19 DN Lee Education EU EU referendum Europe extracellular matrix FGF Fibroblast growth factor Food FRET sensors Gish Gallop glycosaminoglycans GMO government Graduate students heparan sulfate heparin history of science imaging Irvine Stephens Bulloch James Bulloch James Dunwoody Bulloch Liverpool microbiology Nanoparticle Nanoparticles Nanotechnology neuroscience nmr Open Access Open Data orange Parliament Peer Review PhD polysaccharide port sunlight Post publication peer review protein chemistry REF research Research Excellence Framework Research integrity Roast SARS-CoV-2 science Science and Technology Committee Science fraud Science Funding Science progress Scientific American Seminars sorbet speaking strawberry sulfation Sulfotransferase synthetic biology Teaching technology transfer Tourism Travel Universities

Create a free website or blog at WordPress.com.

WPThemes.


Privacy & Cookies: This site uses cookies. By continuing to use this website, you agree to their use.
To find out more, including how to control cookies, see here: Cookie Policy
  • Follow Following
    • Ferniglab Blog
    • Join 73 other followers
    • Already have a WordPress.com account? Log in now.
    • Ferniglab Blog
    • Customize
    • Follow Following
    • Sign up
    • Log in
    • Copy shortlink
    • Report this content
    • View post in Reader
    • Manage subscriptions
    • Collapse this bar
%d bloggers like this: