Are we there yet?

The question relates to what Langmuir termed “Pathological Science”, simply put “people are tricked into false results … by subjective effects, wishful thinking or threshold interactions“. There is a lot of pathological science and I only use the examples below, because I am most familiar with them; for nanoparticles, I have a personal interest in understanding these materials, since I use them to try to make biological measurements, e.g., here.

At long last, the paper of Julian Stirling and colleagues on the absence of stripes in self-assembled ligand shells on gold nanoparticles is published in PLOS One. This paper was a bit of a saga (see PubPeer comments here, Raphael’s post here and Brian’s post here).

To summarise very briefly, there is no evidence for ligands on gold nanoparticles organising into stripes. There is evidence of phase separation, but then many have published data consistent with phase separation of ligand mixtures on nanoparticles, my own example is in Fig. 2 of a paper we published in Langmuir in 2008.

So that is the end, right?

No, as noted on Raphael’s and Brian’s blogs, we still have papers being published in credible journals in 2014 on the properties of nanoparticles with stripes (see links in their posts, above).

This really highlights some of the problems with scientific publishing. Peer review can be weak/poor/non-existent. At the extreme, we have papers that have clear problems with data integrity, for example the “chopstick nanorods” paper
(see post on Retraction Watch) and the STAP stem cell papers (one of several posts on Retraction Watch). One has to ask how did these get past the PI and then past the editor and reviewers? Was everyone asleep, or happy to push out something topical to massage their CV and impact factor, with no regard to the quality of experimental evidence? After all, post-publication peer review picked up the problems within days, yet pre-publication peer review had weeks with the manuscripts.

The above are extremes, though there are plenty of more examples like these. Moving to the middle ground, we have a critical examination of experimental evidence and this is found wanting; yet the publishing system trundles on, oblivious. So while the evidence for stripes does not exist, I predict that papers making claims for such nanoparticles will continue to be published.

This raises interesting questions regarding science and publishing. In general terms, we seem to have entered N-ray territory without exerting critical faculties. It is interesting to note that in the case of N-rays, just as many decades later with Jacques Benveniste’s claims for water memory (see here and here) Nature sent a scientist(s) in to evaluate the evidence and found that this was lacking. This effectively killed off N-rays and water memory at the time. I would further note that in the 21st Century Nature and other journals have abdicated this responsibility to engage with data critically, witness the response of Nature to the STAP cell paper retractions. and discussions on PubPeer relating to the sensitivity of bulk detection of single/very low numbers of molecules (e.g., here).

So where journals used to engage in critical evaluation, they have now apparently given up on this task. Perhaps they are instead focussed on the nonsensical number enshrined in their impact factor?

I would argue that this leaves science publishing in a rather bad state. To his credit, while Francesco Stellacci has clearly felt threatened by criticism of his data and has not engaged in discussions on PubPeer, he has provided his original data to the authors of the PLOS One paper. This is extremely important and I will say it again: credit is due, regardless of process.

Since critical evaluation of data is the foundation of science, it is obvious that we should have access to original data. So while Open Access is touted as a solution, it is not. Open Access is part of the solution, Open Data is the solution. Until data are accessible by all to scrutinise, many “discoveries” will remain unchallenged, Pathological Science will flourish and Science will be the poorer.