Interview of Dr Boris Barbour (Co-director of PubPeer website & researcher at the Institut de Biologie de l'École Normale Supérieure. CNRS UMR8197. Inserm U1024).
During our participation in the “Journées Francophones de la Nutrition 2018”, in Nice, one talk from Boris Barbour caught our attention: talking about the causes and consequences of the lack of scientific integrity, and his presentation of ‘PubPeer’. PubPeer is a public-benefit Corporation created in 2012 with non-profit status in the United States. The overarching goal is to improve the quality of scientific research by enabling innovative approaches to community interaction. Indeed, scientists can comment and ask for more details about all aspects of a publication (design, statistics, interpretation, etc.), they can discuss directly with the authors, but also create a discussion with other participants.
-Can you define your role within the PubPeer platform?
I'm one of the organizers of the site, mostly involved in strategy, design, writing and representation. I also hold the formal position of treasurer on the board of the PubPeer Foundation.
-What were your motivations in being involved in this project?
I felt (and feel) that we publish far too much weak and unreliable research. This has consequences in terms of misleading readers/users of the unreliable publications, leading researchers to waste time and money (often taxpayers' money) in attempting to follow up erroneous work. It can also distort public policy, which would ideally be evidence-based. In the worst case, unreliable research can lead to erroneous medical guidelines causing unnecessary suffering. One obvious example is the Wakefield case, in which fraudulent claims of a link between vaccination and autism dramatically reduced vaccination rates and increased the frequency of measles epidemics. A fraction of measles cases kill, so the erroneous research caused quite avoidable mortality. Faced with the problems caused by erroneous research, we believe it is essential to provide a system to alert researchers as rapidly as possible to any potential problems in a publication.
-In your opinion, why is this kind of tools important for the scientific community? What are the advantages of the platform?
It is said that science is self-correcting, but the reality is that today that correction is too slow and uncertain. We operate in an environment that is extremely hostile (socially, practically, and financially) to all forms of correction or replication in science, in part because authors, journals, and institutions are all subject to conflicts of interest. Our site bypasses all of that, enabling any researcher to share his/her expertise instantly, worldwide. The strength and also the most controversial feature of PubPeer is that we allow anonymous comments. This offers protection against reprisals, either professional or legal, and has greatly encouraged commenting on our platform.
-How do you check the veracity of the comments on the website? Is there a control before publication?
Our central guideline is that comments must be factual, logical, such that they can be verified by other readers. Hearsay is forbidden. We check that comments from untrusted accounts respect this rule before they appear. Note, however, that this is not a scientific review and comments can be wrong, although overall their accuracy is remarkably good. Authors of articles that receive a comment are alerted by email and can reply immediately on the same page with the same prominence. Other users will also often correct erroneous posts. Anybody can report comments they believe to be abusive or that might infringe on our guidelines in other ways.
-In your opinion, what are the biggest challenges today to maintain the scientific integrity in research?
The most important is to access the raw data and analysis code of publications. Nearly all disputes on PubPeer would be resolved by access to the raw data; currently only a tiny fraction of authors choose to share them. Although the raw data of most studies probably won't be checked in detail, the benefits for integrity are several:
- authors will need to organise their own work better
- knowing that data can be checked will discourage some forms of misconduct
- it is much more difficult and risky to fabricate or falsify an entire data set than it is to tweak one illustration
- these benefits can change behaviour generally in a preventative manner
Data publication also allows aggregation and reuse – two important scientific benefits. Systematic full data access will shortly become national policy in France and the Netherlands and an increasing number of journals are adopting data access policies. Once raw data is widely available, we expect the nature of discussion on PubPeer to evolve towards discussions of the analysis and interpretation of the data.
Disclaimer from Mr Barbour: my work for PubPeer is carried out independently of my employers and host institutions; the views I express below should not be taken to represent their positions.
To contact us: Biofortisfirstname.lastname@example.org