Highlighting the use of evidence

I spotted this blog article on Friday morning and sent it on to one of my colleagues, and soon it was sent around all the blog team here at CABI. This whole topic of the use of an icon/logo to highlight the use of peer-reviewed evidence has caused a great deal of discussion and a mixture of feelings amongst the bloggers. I have given it a great deal of thought over the weekend and decided to report the issues and resulting discussions that have occurred.

Some background first: CABI is a not-for-profit organisation with a publishing division. We produce an online literature database as well as printed abstract journals and scientific books. In short we see our role as drawing people's attention to recently published information that is likely to be useful or interesting to them. The database includes items as diverse as peer-reviewed articles (the majority of the records), reports from a myriad of NGOs and government agencies, policy items and books. The blog team consists of trained scientists with a wide range of backgrounds and experience within industry and academia. For each of the blog articles that we post on the site, we use the diverse range of sources that pass across our desks. They are not meant as extensive reviews of the material, but as either opinion pieces or as articles to highlight research, surveys or hot topics developing within that subject area.

Back to the topic: The whole concept of highlighting when a blog article has used or is reporting on peer-reviewed evidence is an interesting one. In our discussions between the bloggers here at CABI, we are all in agreement that it is important that if evidence has been used within an article it is properly referenced, either by linking back to the original article/abstract or by providing the full reference or DOI for the article. However the use of an icon/logo on each blog post that uses peer-reviewed sources has been welcomed by some and met with sheer disgust from others in the blog team.

Why only peer-reviewed evidence, what is wrong with scientific reports? This was a typical response among the team and one that I myself feel strongly about. My PhD involved the use of the systematic review process ¹ to search, select and evaluate the scientific quality and reliability of the evidence, prior to combining datasets on a particular applied ecological problem. During my research I found that the highest quality evidence, using the most appropriate scientific methodologies, was actually returned by the reports that had not been published in peer-reviewed journals, but instead was in NGO and government department libraries or on little-known websites. Neglecting the multiple sources of information (scientific articles, conference proceedings, reports and books) that are available to both the scientist and the blogger is dangerous and can lead to spurious results.

Another side of this particular part of the issue has been raised by some of the other comments on the original post. The peer-review process is not a perfect tool. Remember that the majority of journals do not publish papers which report results that show either negative or no significant difference - they are not high profile enough to provide added impact factor to the journal. This is well documented and part of a problem called “publication bias”. Therefore again this raises problems with basing research or blog articles solely on peer reviewed material.

Overall, the majority of our blog team agreed that the use of maybe a series of icons would be useful for readers on blogs that cover a multitude of material. However, the team here at the handpicked… and carefully sorted blog believes that more important than an icon is the provision of an appropriate reference to the sources used within the article. Team members also feel that an author biography on the blog homepage is also important - something that our own blog is currently implementing (see down the left hand side of the blog). This gives the reader confidence that the author is experienced/appropriately trained to provide an accurate summary of the work being discussed in the article. CABI would welcome comments from our readers on what they would like to see, either about this blog or the world of science blogging and research in general.

¹ refers to paper: Roberts, P.D., Stewart, G.B. & Pullin, A.S. (2006) Are review articles a reliable source of evidence to support conservation and environmental management? a comparison with medicine. Biological Conservation132, 409-423.

Comments

[Jackie Mugah]

Good day,
I'd rather have references over icons.

(I'm not sure of the context in which the term 'icon' is used here, but my understanding is that it provides a general link, for instance to a general resource such as a website. I hope i get it right, since this is the basis of my response.)

References are better than icons, especially since they are very very specific, somewhat like the needle in the haystack, which is what someone looking for the source of any article would like to have.
Perhaps icons may be given as additions for anyone who would like to gain more general knowledge, for instance at the end of the text, and referenced (Ahaa!) to their specific location within the text.

I generally prefer refernces to icons. I find them more helpful.