Feeds:
Posts
Comments

Are we there yet?


The question relates to what Langmuir termed “Pathological Science”, simply put “people are tricked into false results … by subjective effects, wishful thinking or threshold interactions“. There is a lot of pathological science and I only use the examples below, because I am most familiar with them; for nanoparticles, I have a personal interest in understanding these materials, since I use them to try to make biological measurements, e.g., here.

At long last, the paper of Julian Stirling and colleagues on the absence of stripes in self-assembled ligand shells on gold nanoparticles is published in PLOS One. This paper was a bit of a saga (see PubPeer comments here, Raphael’s post here and Brian’s post here).

To summarise very briefly, there is no evidence for ligands on gold nanoparticles organising into stripes. There is evidence of phase separation, but then many have published data consistent with phase separation of ligand mixtures on nanoparticles, my own example is in Fig. 2 of a paper we published in Langmuir in 2008.

So that is the end, right?

No, as noted on Raphael’s and Brian’s blogs, we still have papers being published in credible journals in 2014 on the properties of nanoparticles with stripes (see links in their posts, above).

This really highlights some of the problems with scientific publishing. Peer review can be weak/poor/non-existent. At the extreme, we have papers that have clear problems with data integrity, for example the “chopstick nanorods” paper
(see post on Retraction Watch) and the STAP stem cell papers (one of several posts on Retraction Watch). One has to ask how did these get past the PI and then past the editor and reviewers? Was everyone asleep, or happy to push out something topical to massage their CV and impact factor, with no regard to the quality of experimental evidence? After all, post-publication peer review picked up the problems within days, yet pre-publication peer review had weeks with the manuscripts.

The above are extremes, though there are plenty of more examples like these. Moving to the middle ground, we have a critical examination of experimental evidence and this is found wanting; yet the publishing system trundles on, oblivious. So while the evidence for stripes does not exist, I predict that papers making claims for such nanoparticles will continue to be published.

This raises interesting questions regarding science and publishing. In general terms, we seem to have entered N-ray territory without exerting critical faculties. It is interesting to note that in the case of N-rays, just as many decades later with Jacques Benveniste’s claims for water memory (see here and here) Nature sent a scientist(s) in to evaluate the evidence and found that this was lacking. This effectively killed off N-rays and water memory at the time. I would further note that in the 21st Century Nature and other journals have abdicated this responsibility to engage with data critically, witness the response of Nature to the STAP cell paper retractions. and discussions on PubPeer relating to the sensitivity of bulk detection of single/very low numbers of molecules (e.g., here).

So where journals used to engage in critical evaluation, they have now apparently given up on this task. Perhaps they are instead focussed on the nonsensical number enshrined in their impact factor?

I would argue that this leaves science publishing in a rather bad state. To his credit, while Francesco Stellacci has clearly felt threatened by criticism of his data and has not engaged in discussions on PubPeer, he has provided his original data to the authors of the PLOS One paper. This is extremely important and I will say it again: credit is due, regardless of process.

Since critical evaluation of data is the foundation of science, it is obvious that we should have access to original data. So while Open Access is touted as a solution, it is not. Open Access is part of the solution, Open Data is the solution. Until data are accessible by all to scrutinise, many “discoveries” will remain unchallenged, Pathological Science will flourish and Science will be the poorer.


I need a “Bullshit-O-Meter”, which would determine the purity of bovine excrement that at times heads my way. In a previous post, “Why doesn’t the sun go around the earth?”, I put forth my views on the case brought by Fazlul Sarkar that aims to lift the anonymity of PubPeer. This led to an e-mail from Weishi Meng, which starts (note I have redacted the co-addressee).

“Dear Drs. XXX and Fernig,

A disturbing trend is developing in what is now named “post publication peer review”, a slippery field where scientific journals are losing their grip. If nothing gets done, the hatred-driven blogs Retraction Watch and PubPeer will end up dictating policy on research ethics while putting in place a veritable McCarthyian system to target scientific researchers. And that would be a tragedy. Neither Retraction Watch nor PubPeer publish investigations subject to scientific peer review. These blogs have turned into lose cannons on the internet.

My latest post entitled “Aberrant Post Publication Peer Review at Retraction Watch and Pubpeer” addresses this problem and is now available at

http://scienceretractions.wordpress.com/
…”

The link leads to the site “scienceretractions”, which takes a view diametrically opposite to mine on how science works. Everyone is entitled to their opinion. Mine is simple: the post at “scienceretractions”, is pure, 100%, unadulterated bovine excrement.

1) Fazlul Sarkar has placed in the public domain scientific papers with images that are being discussed in labs and in online forums – PubPeer being just one. Others include the comments at a Retraction Watch post and a post by Derek Lowe (who appears to be a Monty Python fan!) entitled “Scam, Scam, Scam, Scam, Scammity Scam, Wonderful Scam

The “loose cannons” seem to be targeted very precisely at published data. I think a naval captain would be most impressed by such aim. “McCarthyian” refers to the suppression of discussion, not discussion itself. The “Bullshit-O-Meter” would at this juncture go up a notch.

2) We are assured that “Fazlul Sarkar is a professor with a prodigious scientific output of more than 500 peer reviewed publications, tens of millions of dollars in NIH funding, and drugs in clinical trial”.

Agreed. Who will pay if the data underpinning the clinical trials are not what they seem? The patients. We have precedent, for example Anil Potti at Duke.
(last of a long line of posts at Retraction watch here. With such stakes, a modicum of scrutiny and discussion is certainly worthwhile.

3) Just a little further on, the course of action of us anonymous cowards (a question I will come to below) should have taken
The rules of fair play, transparency and scientific standards (how about decency?) indicate that if they really felt there was something wrong with Dr. Sarkar’s results, they should have submitted their conclusions to the same peer review journals where Dr. Sarkar reported his work

Why does PubPeer exist and why did the now defunct “Abnormal Science” and “Science Fraud” come into existence? Because this obvious route has not worked in my view, something I have blogged about at length. Frank discussion of data is fair play and, importantly is science. Blind faith is religion.

4) The purity of the bovine excreta is manifested by ad hominems on the founders of Retraction Watch. OK, like me, they are established and surely have broad shoulders and a strong sense of irony. However, to then take it out on their intern is in my view cowardly in the extreme; cowardice can be an important measure of the purity of bovine excrement.

5) Bullying is an emerging theme of this bovine excrement and is the root of the suit filed by Fazlul Sarkar. It is precisely because Fazlul Sarkar has millions of dollars of grants, power and prestige that anonymity is important: those who spotted the coincidences in the images in his papers feared for their future.

This is why anonymity is so critical: we cannot have an open discussion, because many are too afraid to voice their opinion. Only the powerful and secure take issue with papers at Pubmed Commons, yet more papers are read by the less powerful and secure. There can be no difference in intellectual capacity between the two groups: both are humans. It is natural that the less powerful and secure generate the most questions about papers, since critical thinking is an essential part of the training of all young scientists and collectively, they read more papers.

6) The argument that scurrilous opinion will harm a career takes us to 100% neat bovine excrement, no added water. There is a right of reply on Pubpeer. A considerable number of authors engage in discussions of their work, often to the satisfaction of all. Sometimes there remain issues, and, one hopes, lessons learned regarding keeping original data. Bottom line: a scientist who engages in discussion has an enhanced reputation, one who heads for the legal firm down the road will likely suffer from the Streisand effect.

There should only be one outcome of a challenge, be it anonymous or not, to a publication (or several): a riposte on the quality of the data and their interpretation. Hiding behind ad hominems and notions of “stature” or “importance” cuts no ice, but it is likely to get the “Bullshit-O-Meter” redlining and may even break this delicate instrument.

I could go on, but even bovine excrement is subject to the law of conservation of matter and energy. A “Bullshit-O-Meter” can only have a dynamic range of 0% to 100% and since at this point it would already be redlining, it would not do to break it.

Finally, pre-publication peer reviewing and editing are far from perfect and by definition, nothing is ever proved in science, we can only provide some level of approximation. So discussion is essential. There are plenty of examples of “things that slipped the net”, the latest being this one in the fourth paragraph of the discussion in a paper published in a Wiley journal that is doing the rounds today:
Although association preferences documented in our study theoretically could be a consequence of either mating or shoaling preferences in the different female groups investigated (should we cite the crappy Gabor paper here?)“. Though Wily seem to have taken the paper down and are perhaps engaged in some “Post Publication paper revision”.


Bearing in mind that there are lies, damned lies and publication metrics (apologies to Benjamin Disraeli and Mark Twain), publishing in Elsevier journals may not be good for the health of your future citations.

The negotiations between the Dutch universities and Elsevier have foundered.
(here), which means that form 1 January 2015, Dutch researchers will no longer have access to Elsevier journals. So a significant segment of the research community will only be able to access papers published in Elsevier journals by requesting a PDF directly. This works, though I note that for me e-mailing the authors has been more effective than #Icanhazpdf on Twitter, but then my K-index is pretty pathetic.

The result is that access, reading and so citation by Dutch researchers will become biased – with two equivalent papers to cite, one published by Elsevier and one published by ANOther, they will generally reach for the second.

To ensure a soaring h-index, the only solution is not to publish in Elsevier journals. If you tweet about your papers, doubtless your K-index will similarly soar and the outcome can only be “Trebles all round”.

More seriously, with librarians and more enlightened members of staff egging on university management to bite the bullet on academic publishing, I can only hope that where the Dutch venture, the rest of us will follow. We don’t actually need the closed access commercial model, we have plenty of publishing options with open access and learned society publishers and, as ever, it is the paper that counts, not the container.

Update: a very useful dataset on the cost to UK universities of subscriptions to Elsevier, Wiley and other commercial publishers is here. It is a lot of money!


Virginie’s first paper on her thesis work, “Network based meta-analysis prediction of microenvironmental relays involved in stemness of human embryonic stem cells” was published yesterday at PeerJ. She first put it up as a preprint (v1 here
revised v2 here and then submitted it – my first experience of this and something I will certainly do again.
Continue Reading »


Dear Dr Worms,
I write this letter in a personal capacity to express my views on the letter you sent to Dr Amaya Moro-Martin on behalf of the ESF, regarding her opinion piece in Nature. Note the word “opinion” and refer to Article 10 of the European Convention on Human Rights (on page 11, it is only two very short paragraphs) Continue Reading »


There are many prizes for cultural activities, of which science is one. This week has seen the announcement of the Nobel prizes, a little earlier the IgNobels were awarded. There are, of course many other prizes. I have decided to set up my own.
A question that bugs me and which loomed large while I read the excellent review by Ding Xu and Jeff Esko from UCSD on “Demystifying Heparan Sulfate–Protein Interactions” is how many extracellular proteins are there? Continue Reading »


A recent post at Retraction Watch revealed that Fazlul Sarkar of Wayne State University is behind the attempt to lift the anonymity of commenters at PubPeer
The Spectroscope has posted The REAL threat to PubPeer is a real threat to science communication.

I agree entirely. Continue Reading »

Follow

Get every new post delivered to your Inbox.

Join 320 other followers