Altmetrics and the future of bibliometrics

Journal impact factors have become one of the most frequently used tools for post-publication filtering and evaluation. However, journal level metrics which are based primarily on traditional citation data largely ignore the new channels of dissemination that have emerged in recent times. Indeed, the routes through which new research is found and shared today are more complex than ever before; whilst the might of PubMed and Google remains steadfast, articles are now discovered and shared through other domains as well, such as social bookmarking sites, Facebook and most notably, Twitter. And all the while, the growth in the volume of research also continues apace. So if our trusty friend ‘information overload’ insists on sticking around, we need to develop the right filters to allow us to make sense of it and to help identify the research that is really making an impact.

In other words, better discovery, navigation and management of research content is needed, along with richer, real-time indicators of impact. Article level metrics such as Altmetrics are broadly based on the idea of Scientometrics 2.0 – metrics which incorporate the emerging social channels of dissemination in order to generate richer measures of impact. Whilst nobody could argue that a handful of retweets and shares on Facebook is an indicator of research significance, social media and networking tools can certainly bring something to the table when it comes to dissemination,

So what do article level metrics or altmetrics look like in practice? PLoS integrate ALMs extremely visibly within their website interface through a metrics tab. This integration is key, and incorporates a broad range of measures including:

• Views and downloads
• Traditional metrics based on citation data 
• Social bookmarking sites (CiteULike and Connotea)
• Facebook & Twitter
• Blog aggregators of research blogs
• PLoS readers’ comments 

These metrics help authors and researchers to gauge who is reading an article, and if readers deem it significant enough to comment on or share with other researchers. The value of article level metrics to the author is that they can benchmark the ‘performance’ of their article against others, make decisions regarding where to publish in order to maximise reach, and communicate the impact of their research to funding agencies, potential employers and collaborators. For researchers, ALMs can also be used to assess the significance or value of a particular article within the field.

So what is the future for altmetrics? Phenomenon du jour or the future of bibliometrics? It is difficult to dismiss the value of the data captured by PLoS’s ALMs. They illuminate new and emerging dissemination channels, and richer data is always a good thing. The increased emphasis on research assessment and discovery demand comprehensive and sophisticated measures, and no doubt altmetrics will continue to be refined and developed over time into better measures. If more publishers start to collect this data, and institutional repositories can incorporate it through open APIs, altmetrics could be used to produce richer author- and institutional-level indicators as well.

Whether ALMs will ever manage to acquire the traction and authority of traditional impact factors is another question however. In contrast with established citation data, will many of these social tools still be around in five or ten years time? Who knows. As applications come and go, comparing data over time becomes impossible. Furthermore, the number of shares or links on Facebook or Twitter is likely to be influenced by the popularity of these tools themselves also - if nobody likes Twitter anymore, this will also manifest itself through lower ALMs. In this context, cross-sectional comparisons might still offer some insight, but longitudinal data will be largely meaningless. Preferences or fashions may also be discipline or community specific, making institutional or macro-level comparisons tricky. The fluidity and dynamic nature of social tools and behaviour makes constructing robust and systematic measures challenging at best. But as Tukey* argues: “Far better an approximate answer to the right question, which is often vague, than an exact answer to the wrong question, which can always be made precise”. With altmetrics, it seems to be a case of watch this space.

Tukey, J. W. (1962). The future of data analysis, Annals of Mathematical Statistics, 33, p. 1-67.