Practical Tips for Facilitating Research - Moira J. Bent (Review)

The title of this book emanates directly from Facet Publishing's new Practical Tips series, and in this case the content certainly backs up that claim. Throughout Practical Tips for Facilitating Research Moira J. Bent’s ‘wisdom of crowds’ approach offers insights and experiences that are very much based on real world examples and exemplars, both her own and those of colleagues working in other institutions. As a result, there is not only a real richness and breadth to the advice presented, but an underlying authenticity and credibility to it as well.

During the opening section on landscapes and models, Bent discusses how to understand who your researchers are, their needs, motivations, and workflows - an aspect that is fundamental to helping us facilitate research more effectively. Those who are relatively new to working in the academic environment may find this section particularly helpful to orient themselves in their new role and as a means of getting to grips with the perspective of a typical researcher. However, the recurring and useful “to think about” prompts also provide food for thought and reflection for even the most experienced readers. The references to theory are plentiful and well-researched, and give ‘just enough’ without dwelling on too much detail – it is, after all, a book very much focused on practice.

A similar pattern continues throughout the rest of the text, which covers a range of areas including collections and information literacy, as well as unpacking specific interventions where libraries can actively position themselves, such as RDM and systematic reviews. Suggestions are notably pragmatic, and very much cognisant of the practical realities of many resource-stretched libraries. For instance, in relation to the potential for libraries to become more involved in supplying data for research proposals, Bent suggests: "Before venturing down this route, consider whether you have sufficient resources to continue if the idea takes root. Treating the contribution as a pilot or experiment will ensure that you are able to draw back or even investigate if a percentage of the subsequent grant might devolve to the library in recognition of the work" (2016, p.131). Currency is crucial in a rapidly-changing area such as scholarly communications, and the text is very much up to date with reference to a number of recent developments such as the Leiden Manifesto.

Those who are exploring how to support research more strategically may find the chapter on organisational structures particularly useful. Again using examples from different institutions, Bent showcases some of the varied and different approaches that can all work well, depending on the specific context and objectives, for instance having a specialised research services function versus offering research support through subject or liaison librarians. I was particularly glad to see a section encouraging librarians to become researchers and writers themselves, as this is something that has personally given me a much greater understanding of the research process, and the needs and workflows of researchers.

What is very much apparent throughout the book is Bent’s awareness that a one size fits all approach does not typically work. Underpinned by a flexible rather than prescriptive format, the book prompts readers to consider the options that might work best for researchers in their own organisations, rather than suggesting that they try to transplant or replicate a successful service, model or programme verbatim from another institution. Unlike many books, the structure is deliberately designed to allow readers to dip into specific sections as needed, rather than necessarily having to read it from cover to cover – a real advantage for those time-poor librarians looking for a quick burst of inspiration or advice. Above all however, it’s a book that offers a variety of approaches, insight, and real-world examples that work – exactly what you need whether you are searching for a simple solution to quickly improve services, or ideas to help inform and shape a more fundamental or strategic change.

Practical Tips for Facilitating Research is published by Facet Publishing, March 2016, 299pp, £49.95.

Read More

Different types of doctoral theses - an analysis from DiVA data 2000-2015

For a while now I have been thinking about the different forms of publishing a doctoral thesis. Just from my own experiences I had a feeling that today there is a greater share of doctoral theses published as comprehensive summary (content of published articles) than ordinary monographs. Can I confirm that and if so why?


I started with asking my colleague, Ulf-Göran Nilsson - library strategist at Jönköping University, for data from DiVA. DiVA is the digital repository used by Jönköping University. DiVA is also a cooperation among 40 different universities mainly from Sweden, but also from Denmark and Norway. I wanted the data to be as aggregated as possible so I choosed all member universities and all different subject and only used the total number of doctoral theses registrated in the archive. After that I separated the data between the two different formats, comprehensive summary or monograph, to be able to see the trend over the years from year 2000 - 2015. I selected a five year interval between the selected data to be able to see changes over time. This is what I found.

As you all can see there is a remarkable trend towards publishing the doctoral thesis as a comprehensive summary instead of a monograph. What can be the causes for this? Can new demands for academic publishing within peer reviewed scholarly journals be a reason? Or, has the new process of evaluating a researchers success and career to be more focused on your appearance is scholarly journals influenced the choice of format? I have no answer but I think it is interesting to evaluate and discuss.


Another thought running through my brain is how this impact the research. Can the move from working with a complete monograph towards publishing a comprehensive summary make the research more fragmanted and gouged? Is that beneficial or can it be a threat? Again I have no answers but I still think it is a very interesting and important question to discuss.


What is your experiences on this matter...

Read More

The historical evolution of open access publications (OA) between 1996 and 2013

Science Metrics released a study in Oct. 2014 which measured the proportion of open access papers published in peer-reviewed journals at the European and world levels between 1996 and 2013. This study represents the largest scale measurement of open access availability performed to date.

The source for data collection was the Scopus database, which is the largest abstract and citation database of peer-reviewed literature with presently 55 million records, 21,915 titles and 5,000 publishers indexed.

Below is a redacted version of the report’s executive summary; the full report can be accessed here.

The Science Metrics study report assesses the free availability of scholarly publications during the 1996 to 2013 period. A sample of one-quarter of a million records was used to study the historical evolution of open access (OA) between 1996 and 2013 and a larger, one million records sample was used to perform an in-depth assessment of the proportion and scientific impact of OA between 2008 and 2013 in different types of OA, for different scientific fields of knowledge, and for 44 countries, the EU28, ERA, and the world.

As of April 2014, more than 50% of the scientific papers published in 2007, 2008, 2009, 2010, 2011, and 2012 can be downloaded for free on the Internet. This is an important finding as only two years ago, in April 2013, the proportion of papers that was freely available was just a hair below 50% (49.54%) in 2011 and did not reach that mark for any other year. On average, the citation advantage of OA papers is 40.3% while the citation disadvantage is 27% for non-OA papers (based on a total sample size of 209,000 papers).

The growth of OA appears as the result of four main forces: (1) historical growth in the interest in OA, which translates into new papers being increasingly available for free; (2) the growing interest in OA also translates into actors increasingly making old papers available for free; (3) OA policies that allow for delaying OA to scientific papers with embargo periods produce a concomitant disembargoing of scientific articles, which creates additional growth in old papers being made available for free; and (4) the number of published scientific papers is growing, so even for a stable proportion of OA, the number of OA papers would keep growing.

Other forms of OA—Gold OA Papers (that is, those with article processing charges published in subscription journals or so-called hybrid journals), Green DOA and Gold DOA (embargoed self-archiving and embargoed journals), ROA (Robin Hood or Rogue OA) and papers archived in non-institutional repositories such as ResearchGate—account for a large part of the pie. This large, heterogeneous set contributes the largest proportion of OA papers, and there is therefore an urgent need to disaggregate this category. More research and more careful classification and thus finer-grained measures are required to better understand how these various categories contribute to OA growth, what their pattern of time-delay is, what their transiency is (especially of the ROA), how important backfilling is and how far back it goes.

The fields with the greatest proportion of OA are General Science & Technology (Adjusted
OA=90%), Biomedical Research (71%), Mathematics & Statistics (68%), and Biology (66%).
OA is not as commonly used in Visual & Performing Arts (Adjusted OA=25%), Communication
& Textual Studies (31%), Historical Studies (34%), Engineering (35%), and Philosophy & Theology (35%).

Gold OA availability is greatest in general S&T (58% of the sampled papers) and lowest in general arts, humanities & social sciences (2.6%); it is also very low in the visual & performing arts (2.8%), built environment & design (3.5%) and engineering (4.1%). Other fields with high availability in Gold journals include biology (17%), agriculture, fisheries & forestry (16%), and public health & health services (16%).

All the fields derive an OA citation advantage. Paradoxically, many of the fields where the OA
proportion is low have a sizeable citation advantage, such as the visual & performing arts (80% more cited), communication & textual studies (66%), philosophy & textual studies (63%), historical studies (55%), general arts, humanities and social sciences (51%), and engineering (38%). An explanation to this is likely to be that papers from researchers in these fields are more likely to have their papers used as there are fewer OA papers available.

Citation statistics on Gold journals require careful interpretation. First, many Gold journals are younger and smaller, and these factors have an adverse effect on the citation rate and hence on measured citation scores. Authors frequently prefer reading and citing established journals, and it is therefore a challenge to start a journal from scratch, and to have authors submit high-quality articles. It takes time to build a reputation and to attract established authors.

It is interesting to note that the Netherlands, which is also scientific publishers' land of predilection, is the EU country with the largest share of papers available in OA form (74%) as a whole for papers
published in the 2008–2013 period and available for free download as of April 2014.

Within the European Union, Green OA is more widely used in Portugal (16.3%), Ireland (15.8%), France (14.0%), and Belgium (13.8%), and least used in Lithuania (4.5%), Malta (5.0%), Croatia (5.2%), and Romania (5.3%). The countries that least use Gold OA journals are France (6.6%), the United Kingdom (7.2%), and Belgium (7.4%).

Green OA is free, and the majority of publishers accept that papers can be self-archived in one form or another (pre-print, post-print with final revision, or PDF) with no delay. Moreover, two-thirds of Gold OA journals do not levy author processing charges (Suber, 2013). There are free avenues to OA and cost should not be construed as a barrier.

The percentage of peer-reviewed articles published in Gold OA journals indexed in Scopus for
1996 was only 0.9%, but grew to 12.8% for 2012, the annual growth rate being 18% for this period, which means that the proportion of articles in Gold OA journals doubles every 4.1 years (see Figure 7 below). Scopus covers less than half of the quantity of journals listed in the DOAJ, so this figure likely underestimates the true extent of the role played by Gold OA journals.

Source: science-metrix.com

For a discussion around the rationale on diamond open access (DOA) publishing, see Fuchs, C. & Sandoval, M. (2013). The diamond model of open access publishing: Why policy makers, scholars, universities, libraries, labour unions and the publishing world need to take non-commercial, non-profit open access serious. TripleC: Communication, Capitalism & Critique, 11(2), pp. 428-443.

Suber, P. (2013). Open access: six myths to put to rest. The Guardian. 21 October 2013.
http://www.theguardian.com/higher-education-network/blog/2013/oct/21/open-access-myths-peter-suber-harvard.

Read More

The Impact of Print Media and Wikipedia on Citation Rates of Academic Articles

Guest post by Daniel Price. Daniel lives in Israel, has an MA in Library and Information Science from Bar Ilan University, and works as a librarian at Shalem College in Jerusalem.

There is a clear desirability to publish a paper that has a strong scholarly impact, both for personal satisfaction knowing that one’s research has been viewed and built upon, and for professional reasons, since the number of citations a paper can correlate to promotion and tenure - the ubiquitous “publish or perish” (Miller, Taylor and Bedeian, 2011), which has now become an international phenomena (De Meis, Leopoldo, et al, 2003; De Rond and Miller, 2005; Min, Abdullah and Mohamed, 2013; Osuna, Cruz-Castro and Sanz-Menéndez, 2010; Qiu, 2010; Rotich and Muskali, 2013), increased salary and external funding (Browman and Stergiou, 2008; Diamond, 1986; Gomez-Mejia and Balkin, 1992; Monastersky, 2005; Schoonbaert and Roelants, 1996) and even the chance of winning professional prizes such as a Nobel Prize (Pendlebury, 2013).

Understandably then many studies have been carried out to discover the characteristics of highly cited papers (Aksnes, 2003) and the factors that influence citation counts. It is widely accepted that it is not just the quality of the science that affect the citation rate, but bibliometric parameters of papers such as its length (Abt, 1998; Ball, 2008; Falagas et. al. 2013; Hamrick, Fricker and Brown, 2010), number of references (Corbyn, 2010; Kostoff, 2007; Vieira and Gomes, 2010; Webster et. al., 2009), number of authors (Aksnes, 2003; Borsuk et. al., 2009; Gazni and Didegah, 2011; Wuchty et al., 2007), length of titles (Habibzadeh and Yadollahie, 2010; Jacques and Sebire, 2010), colons in titles (Jamali and Nikzad, 2011; van Wesel, Wyatt & ten Haa, 2014; Rostami, Mohammadpoorasl, and Hajizadeh, 2014).

A variety of external considerations is also known to influence the citation rate of academic papers. Intuitively a paper that has been publicised in the popular print media will be cited more as its publicity makes researchers more aware of it; however it can be argued that quality newspapers only cite valuable articles that would garner a significant number of citations in any case. That the first assumption was true was proven thirteen years ago in 1991 by comparing how many more citations articles published in the New England Journal of Medicine received if they were quoted in the New York Times during a 12 week period in 1978 when copies of the paper were printed but not distributed due to a strike compared to the following year of 1979. The results showed that articles covered by the Times received 72.8% more citations during the first year after their publication but only those discussed when the paper was actually distributed. Articles covered by the Times during the strike period received no more citations that articles not referenced by the Times, thus proving that exposure in the Times is a cause of citation (“the publicity hypothesis”) and not a forecast of future trends (the “earmark hypothesis”) (Phillips, 1991).

Phillips’ findings articles that covered in the New York Times receive more citations was confirmed in another study conducted 11 years later which also found however that exposure in less “elite” daily newspapers (but not in evening broadcasts of mainstream USA television networks) during a twelve month period from mid-1997 to mid-1998 also correlated with higher citation rates of a wider range of scientific papers, thus showing that scientific communication is not just carried out through elite channels. Importantly though, the author notes that his study does not prove the “publicity hypothesis” as the articles that were publicised could have been more intrinsically important and were only cited for this reason, although it does cast doubt on the “earmark hypothesis” since many articles that were not mentioned were cited (Kiernan, 2003).

In the present day much scholarly communication takes place on Web 2.0 tools and in the emerging field of “altmetrics” (Konkiel, 2013; Priem, 2014; Thelwall, 2013), studies focus on parametrics including whether it has been cited and discussed on academic blogs (Shema, Bar-Ilan and Thelwall 2014), tweeted (Eysenbach, 2011), and uploaded to a social media platform such as Mendley (Li and Thelwall, 2012).

Research has also investigated whether articles cited on the decidedly non-elitist Wikipedia. A study conducted in the beginning of 2010 found that 0.54% of approximately nineteen million Wikipedia pages cited a PubMed journal article, which corresponds to about 0.08% of all Pubmed articles. The researchers showed that journal articles that were cited in Wikipedia were cited more and had higher F1000 scores than a random subset of non-cited articles, a phenomenon they explained according to their hypothesised that Wikipedia users would only cite important articles that present novel and ground-breaking research (Evans and Krauthammer, 2011).

A larger study carried out two and half years later came to the same conclusion that academic papers in the field of computer science which are cited on Wikipedia would be more likely to be cited because the Wikipedia entries are written by talented authors who are careful to cite reputable authors and trending research topics (Shuai, Jiang, Liu and Bollen, 2013).

These conclusions support the “earmark hypothesis” that Phillips rejected and Kiernan doubted. Wikipedians are credited with identifying high impact journal articles soon after they are published and recommending them to other users.

In order to preserve a careful dialectic of both sides of the publicity/earmark hypotheses though, the possibility should be entertained that the large number of Wikipedia users may include researchers who, flooded with an information overload of thousands of articles, are motivated to read and quote certain articles because they saw them quoted on Wikipedia. Future research could investigate the information behavior of a large number of researchers, specifically their use of Wikipedia.

Bibliography:
Abt, H. A. (1998). Why some papers have long citation lifetimes. Nature, 395, 756-757.

Aksnes, D. W. (2003). Characteristics of highly cited papers. Research Evaluation, 12(3), 159-170.

Ale Ebrahim, N., Salehi, H., Embi, M. A., Habibi Tanha, F., Gholizadeh, H., Seyed Mohammad, M., & Ordi, A. (2013). Effective strategies for increasing citation frequency. International Education Studies, 6(11), 93-99.

Ball, P. (2008). A longer paper gathers more citations. Nature, 455(7211), 274-275.

Borsuk, R. M., Budden, A. E., Leimu, R., Aarssen, L. W., & Lortie, C. J. (2009). The influence of author gender, national language and number of authors on citation rate in ecology. Open Ecology Journal, 2, 25-28.

Browman, H. I., & Stergiou, K. I. (2008). Factors and indices are one thing, deciding who is scholarly, why they are scholarly, and the relative value of their scholarship is something else entirely. Ethics in Science and Environmental Politics, 8(1), 1-3.

Corbyn, Z. (2010). An easy way to boost a paper's citations. Nature. Available at http://dx.doi.org/10.1038/news.2010.406

Evans, P., & Krauthammer, M. (2011). Exploring the use of social media to measure journal article impact. In AMIA Annual Symposium Proceedings (Vol. 2011, p. 374). American Medical Informatics Association.‏

Eysenbach, G. (2011). Can tweets predict citations? Metrics of social impact based on twitter and correlation with traditional metrics of scientific impact. Journal of Medical Internet Research, 13(4).

Falagas, M. E., Zarkali, A., Karageorgopoulos, D. E., Bardakas, V., & Mavros, M. N. (2013). The impact of article length on the number of future citations: a bibliometric analysis of general medicine journals. PloS one, 8(2), e49476.

Gazni, A., & Didegah, F. (2011). Investigating different types of research collaboration and citation impact: a case study of Harvard University’s publications. Scientometrics, 87(2), 251-265.

Gomez-Mejia, L. R., & Balkin, D. B. (1992). Determinants of faculty pay: an agency theory perspective. Academy of Management Journal, 35(5), 921-955.

Habibzadeh, F., & Yadollahie, M. (2010). Are shorter article titles more attractive for citations? Crosssectional study of 22 scientific journals. Croatian medical journal, 51(2), 165-170.

Hamrick, T. A., Fricker, R. D., & Brown, G. G. (2010). Assessing what distinguishes highly cited from less-cited papers published in interfaces. Interfaces, 40(6), 454-464.

Jacques, T. S., & Sebire, N. J. (2010). The impact of article titles on citation hits: an analysis of general and specialist medical journals. JRSM short reports, 1(1).

Jamali, H. R., & Nikzad, M. (2011). Article title type and its relation with the number of downloads and citations. Scientometrics, 88(2), 653-661.
Kiernan, V. (2003). Diffusion of news about research. Science Communication,25(1), 3-13.

Konkiel, S. (2013). Altmetrics: A 21st‐century solution to determining research quality. Online Searcher, 37(4), 10‐15.

Kostoff, R. N. (2007). The difference between highly and poorly cited medical articles in the journal Lancet. Scientometrics, 72(3), 513-520.

Li, X., & Thelwall, M. (2012). F1000, Mendeley and traditional bibliometric indicators. In Proceedings of the 17th International Conference on Science and Technology Indicators (Vol. 2, pp. 451-551).

Monastersky, R. (2005). The number that’s devouring science. The chronicle of higher education, 52(8), A12.

Osuna, C., Cruz-Castro, L., & Sanz-Menéndez, L. (2011). Overturning some assumptions about the effects of evaluation systems on publication performance. Scientometrics, 86(3), 575-592.
Phillips, D. P., Kanter, E. J., Bednarczyk, B., & Tastad, P. L. (1991). Importance of the lay press in the transmission of medical knowledge to the scientific community. The New England Journal of Medicine, 325(16), 1180-1183.

Price, D. (2014). A bibliographic study of articles published in twelve humanities journals. Available at https://www.academia.edu/7820799/A_Bibliographic_Study_of_Articles_Published_in_Twelve_Humanities_Journals

Priem, J. (2014). Altmetrics. In B. Cronin and C. R. Sugimoto (Eds.) Beyond bibliometrics: harnessing multidimensional indicators of scholarly impact (pp. 263-287).

Rostami, F., Mohammadpoorasl, A., & Hajizadeh, M. (2014). The effect of characteristics of title on citation rates of articles. Scientometrics, 98(3), 2007-2010.

Schloegl, C., & Gorraiz, J. (2011). Global usage versus global citation metrics: the case of pharmacology journals. Journal of the American Society for Information Science and Technology, 62(1), 161-170.
Schoonbaert, D., & Roelants, G. (1996). Citation analysis for measuring the value of scientific publications: quality assessment tool or comedy of errors? Tropical Medicine & International Health, 1(6), 739-752.

Shema, H., Bar‐Ilan, J., & Thelwall, M. (2014). Do blog citations correlate with a higher number of future citations? Research blogs as a potential source for alternative metrics. Journal of the Association for Information Science and Technology.

Shuai, X., Jiang, Z., Liu, X., & Bollen, J. (2013). A comparative study of academic and Wikipedia ranking. In Proceedings of the 13th ACM/IEEE-CS joint conference on Digital libraries (pp. 25-28).

Thelwall, M., Haustein, S., Larivière, V., & Sugimoto, C. R. (2013). Do Altmetrics Work? Twitter and Ten Other Social Web Services. PloS one, 8(5), e64841.

van Wesel, M., Wyatt, S., & ten Haaf, J. (2014). What a difference a colon makes: how superficial factors influence subsequent citation. Scientometrics,98(3), 1601-1615.
Vieira, E.S., & Gomes, J.A.N.F. (2010). Citation to scientific articles: Its distribution and dependence on the article features. Journal of Informetrics, 4 (1), 1-13.

Webster, G. D., Jonason, P. K., & Schember, T. O. (2009). Hot topics and popular papers in evolutionary psychology: analyses of title words and citation counts in evolution and human behavior, 1979–2008. Evolutionary Psychology, 7(3), 348-362.

Wuchty, S., Jones, B. F., & Uzzi, B. (2007). The increasing dominance of teams in production of knowledge. Science, 316(5827), 1036-1039.

Read More

A Bibliometric Study of Articles Published in Twelve Humanities Journals

Guest post by Daniel Price. Daniel lives in Israel, has an MA in Library and Information Science from Bar Ilan University, and works as a librarian at Shalem College in Jerusalem.

The majority of bibliometric research has focused on the parameters of scientific and social scientific papers while the humanities have been somewhat neglected.

In a paper that can be accessed at here, I did a study of original research papers published in three journals in each of three subjects in the humanities: ethics, history and theoretical linguistics. I looked at the correlation between the number of authors, the length of the title and the paper, the number of references quoted, and how all these impact the article's impact factor.

Based on a paper written by two Iranian scholars from Tehran, I also sought to verify if there is a connection between the type of title of the papers and the number of times they are cited. The results are compared and contrasted to those found in previous research, and the extensive bibliography has plenty of suggestions for further reading.

Read More

PlumX – EBSCO roadshow, UCD, 30th Oct. 2014

Yesterday I had the pleasure of attending an EBSCO roadshow event, which covered their recently acquired Plum Analytics service (thanks to Michael Ladisch for the invitation).

Plum Analytics launched back in 2012 on the back of Jason Priem’s rationale around Altmetrics (see the manifesto for more), the idea being that traditional filters of scholarly communication (peer-review, citation counting, JIF) are simply not sufficient (and transparent) enough to account for what’s actually happening out there on the Web, which forms part and parcel of modern scholarly communications.

EBSCO jumped right into the mix here and now aims to offer a comprehensive metrics service through the aggregation of traditional metrics and alternative metrics in a side-by-side view context (they call this ALLmetrics).

The following metrics are considered:
1. USAGE (clicks, downloads, views, library holdings, video plays)
2. CAPTURES (bookmarks, code forks, favourites, readers (e.g. Mendeley), watchers)
3. MENTIONS (blog posts, comments, reviews, Wikipedia links)
4. SOCIAL MEDIA (+1s, likes, shares, tweets)
5. CITATIONS (PubMed Central, Scopus, patents)

Example sources include the likes of Amazon, Bit.ly, CrossRef, Delicious, Dryad (for data sets), dSpace, ePrints, Github, USPTO, Scopus, Stack Overflow and more… This list is added to with new sources as they bubble to the surface and establish themselves.

In a nutshell, EBSCO’s Allmetrics goes beyond traditional citation metrics and aims to offer a more holistic view of how researchers’ outputs are communicated and ‘used’. It does so by considering a multitude of popular digital environments where scholarly outputs feature.

The University of Pittsburgh was the first institution that adopted this service – PlumX/Pitt.
Below is a screenshot of one of their academics and his scholarly footprint.

EBSCO’s full presentation is available here. See https://plu.mx for more on PlumX.

At this point in time, no higher education institution in Ireland has adopted this service.
It will be interesting to see who will take the leap first.

As part of the wider discussion around Altmetrics, you also might want to consider Featherstone, R. f. (2014). Scholarly Tweets: Measuring Research Impact via Altmetrics. Journal Of The Canadian Health Libraries Association (JCHLA), 32(2), 60-63.

Read More

International Society of Scientometrics and Informetrics Conference / Vienna, July 2013 - Insights and Perspectives

Guest Post by Michael Ladisch, Assistant Librarian at UCD Library

Attending conferences is always a nice break from the daily routine. But more importantly, you learn about cutting edge research in a field, you can strengthen existing networks or establish new connections, you get ideas and exchange experiences. The location shouldn’t really matter as long as the event is well organised and it doesn’t need to be a tourist hotspot (ever been to Milton Keynes, UK, or Bielefeld, Germany?) But sometimes you are lucky and the conference is in an interesting place you’ve never visited before.

The biennial conference of the International Society of Scientometrics and Informetrics was held in Vienna, Austria, from 15th-19th July 2013. This is one of the main conferences in the discipline, attracting about 400 participants from 42 countries this year. Previous ISSI conferences took place in Brazil and South Africa, so the short distance to Austria (relatively speaking) made it possible for me to attend.

The venue was spectacular: University of Vienna, founded in 1365, is located in the heart of the city, walking distance from many of the famous sights, but an impressive building complex by itself (actually, there are university locations in other parts of Vienna as well; the 90,000 students wouldn’t have enough space in the main building).Especially during plenary sessions in the Grosse Festsaal (Main Ceremonial Chamber) self-discipline was needed not to be distracted by wonderful wall and ceiling paintings.

Coffee and lunch breaks were spent in the courtyard, which fortunately provided some shade as temperatures were close to 30 degrees all the time.

But this is not a travel blog - so, what was the conference about? Scientometrics is a sub-field of Information Science, using often (but not exclusively) bibliometric methods to measure and analyse research. It includes such topics as citation analysis, collaboration studies, studies in university ranking, altmetrics, patent analysis, journal impact measures, etc. etc.

About 150 papers and 100 posters were presented in three very intense days on recent research being done in the discipline. Naturally, I couldn’t attend everything; I always had to choose one of four parallel panels, so my report can’t be comprehensive. Those of you interested in specific topics can access the programme and conference proceedings (about 2200 pages in print) at http://www.issi2013.org.

The panels I went to were about citation analysis, citations in full texts, usage metrics, altmetrics (of course), document types (other than journal articles) and monographs, just to name a few. Presenters talked about research projects that might improve bibliometric methods and looked into new ways of measuring research impact which can then be used for funding, recruitment or promotion decisions.  Here are a few examples:

• Wouters: Counting publications and citations: Is more always better? (it is not – a higher citation count doesn’t necessarily mean a higher impact)
• Strotmann: Author name co-mention analysis (based on full text analysis, this is still research in progress, but might become a useful tool in the Social Sciences and Humanities)
• Guerrero-Bote: Relationship between downloads and citations (use of downloads as predictions for citations is limited and different in different areas) 
• Holmberg: Disciplinary differences in Twitter scholarly communication (researchers share more links and re-tweet more than the average Twitter user; there is less scholarly tweeting in economics and history)
• Cabezas-Clavijo: Most borrowed is most cited? (analysing loans in university libraries of Vienna and Grenada; no or only weak correlation between loans and citations)

What struck me at some point was that so many of the studies were based on Web of Science (WoS) and Scopus data. There were papers based on PubMed, Google Scholar or other data as well (see examples above), but in general bibliometric research is often focused on disciplines that are well covered by Thomson Reuters’ and Elsevier’s products. This excludes many areas, especially in the Social Sciences and Humanities. One paper (Chi: Do non-source items make a difference?) looked at the WoS representation of publications from two major Political Science departments in Germany (Muenster and Mannheim). Only 7% of the publications were listed in WoS. Here was the big elephant in the room: What are we NOT counting? What are we missing?

Though sometimes carried away by their WoS/Scopus based (and sometimes funded) research projects most bibliometricians are aware of this problem (and several papers addressed it). In two very interesting plenary sessions the consequences of bibliometrics research were discussed together with the request for improved and new methodologies. Due to lack of reliable data in some disciplines bibliometric methods should always be used with caution.

The vice-president for research of University of Vienna, Susanne Weigelin-Schwiedrzik, said quite directly: “We have to think twice before using your data. But you have the responsibility to realise your role in a more fundamental way. … We need to take decisions on the basis of data on impact. We look at published articles and at Impact Factors. As a researcher, I know that this is incorrect since these indicators do not directly reflect quality. But as a manager, I do not know what to do else.” (for more details see this blog post)

Probably the most important plenary session was on “Methodological and Ethical Problems of Individual-Level Evaluative Bibliometrics”. Paul Wouters (Univ. Leiden) and Wolfgang Glänzel (Univ. Leuven) presented their “10 Dos and Don'ts” in bibliometrics when it comes to assessing individual researchers. These points will be discussed in the bibliometrics community and might become something like a code of ethics. Some of the points are rather obvious (“Don’t rank scientists according to one indicator”), others were more surprising and will probably cause a stir (“Don’t use impact factors as a measure of quality for individual researchers”).

Hopefully, slides of the presentation will be available online (some of them are already on different platforms). If interested in a specific PPT we are asked to contact the authors. The direct link to proceedings is http://www.issi2013.org/proceedings.html. If you want to read the conference tweets search for hashtag #issi2013.

What else did I bring back from the conference? I felt that only one participant from Ireland (and I’m not even Irish) at such a high profile conference is not enough to represent our community. We should look more often and more closely at what’s happening in continental Europe. We are well connected with UK institutions, but there is also a lot of interesting stuff (not only in bibliometrics, but in library-related issues generally) going on in Belgium, the Netherlands, Germany, Spain and especially in the Scandinavian countries. Language is not a problem as everybody at such events is rather fluent in English. And travelling to Antwerp, Copenhagen or Vienna isn’t a much bigger effort than going to Glasgow or Canterbury. So, hopefully, at the next ISSI Conference 2015 in Istanbul there will be a larger delegation from Ireland. I can certainly recommend it.

Please, don’t hesitate to contact me if you have any comments or questions (michael.ladisch[at]ucd.ie) or follow my (infrequent) tweets at @MichaelUCDLib.

Read More

#altmetrics (and why you need to know about them)

Anyone who knows me, knows that if I am not talking about twitter, I am probably talking about altmetrics (see my previous blog posts). Quite a few people (faculty, clinicians, librarians) have asked me about article level metrics (ALMs) of late, so I have put together a brief overview on slideshare as it is a lot easier to explain the concept of altmetrics using visuals rather than text. It is really only the tip of the iceberg as regards ALMs, but the resources linked to within the presentation will provide a lot of further reading if desired. Feedback and comments welcome (by email, twitter or via the comments below).

#altmetrics 101 from Mish Dalton

Read More

Altmetric It Bookmarklet

I have been using the Altmetric It Bookmarklet myself for a while now, and more recently as a really useful and quick tool to demonstrate to researchers the importance of thinking "outside the impact factor" and traditional citation metrics. Talking about twitter and social networking tools is all well and good, but nothing hits home to scientists like hard data and numbers. Using the doi of an article, the Altmetric It Bookmarklet will tell you how many times it has been tweeted, blogged about, shared on facebook, added to Mendeley and more.

Impact is not just about citations!

Take an article I have been using in workshops recently to explore the Google Scholar v PubMed idea, Is the coverage of google scholar enough to be used alone for systematic reviews (doi:10.1186/1472-6947-13-7). It was only published last month online (so too soon for any citations to have filtered through) but has already been tweeted by over 100 different accounts. Showing researchers these numbers highlights the value of using tools like twitter to share and disseminate research. Many people, such as industry practitioners, will probably use and apply the findings of your research without ever citing it. This usage still represents impact. You can also get a detailed breakdown of who is sharing your research through the 'more details' link, which opens up a wealth of opportunities for connecting with others. 

However, altmetrics not only capture usage and impact in the social space, but may potentially be an interesting leading indicator for citations. I would love to look at the latter in more detail and explore the correlation between early tweeting/sharing and citations over time. 

Read More

Why Research Impact Is About Promotion, Not Impact Factors (and Why This Is Good News For Libraries)

A paper published in the current issue of the Journal of the American Society for Information Science and Technology (paywall, sorry :( ) estimates that the proportion of highly cited papers published in high impact journals has declined substantially in recent decades. In 1990 45% of the most cited 5% of papers were published in the top 5% of journals ranked by impact factor; by 2009 the proportion had dropped to 36% - representing a 20% fall in market share in 20 years. This is partly due to the growth in the number of publication outlets generally over the period, however it still appears a high impact factor is no longer the sine qua non it once was.

In my view, this is just the beginning. The increased use of altmetrics has already shown that tweets, bookmarks and other social sharing tools can have a substantial impact in disseminating research quickly. However, channels like these only represent the tip of the iceberg when it comes to promoting research. Whilst twenty years ago getting your paper into a high impact title like Nature or Science was enough to ensure it would be widely read, in today's competitive research environment it counts for a lot less. Now, research promotion is almost as import as research itself.

The video below is an excellent example of this idea. This short film essentially functions as a 'trailer' to promote a research paper by Cross & Wheatland (the YouTube URL links to the pre-print), and with over one million views to date is a simple but effective example of how creativity and promotion can increase research visibility - and potentially impact. This doesn't mean that every author should start making videos to accompany their journal articles, but it does provide an opportunity for libraries to position themselves as research promoters as part of their overall research support package.

This is an area where libraries can provide a real value-added service to researchers - leveraging their existing expertise and networks to help package, distribute and promote research. In most cases, libraries are already doing this on a lesser scale e.g. through open access initiatives, promoting institutional respositories and advising on bibliometrics. However the benefits from ramping up these efforts into a more co-ordinated and creative package, including aspects like SEO and social-media, are potentially significant. As Brian Mathews argues in this article in the Chronicle: "It’s no longer about just publishing a paper but creating a suitable outlet and campaign to share the findings. Our [librarians'] job becomes producers: designing and developing the channels, methods, processes and metrics to repackage content (academic papers) into formats apt for expanding the audience".

Read More

RSP webinars on Institutional Repositories

Two free RSP webinars are scheduled for December:

The role of institutional repositories after the Finch report - December 4th, 3pm (GMT)

Three repository administrators, Dr. Miggie Pickton (University of Southampton), Dominic Tate (Royal Holloway, University of London) and Sally Rumsey (Oxford University) will discuss the current role of the institutional repositories, the developments and implications after the Finch report and the necessary actions that each institution needs to take into consideration.

Registration is open at http://www.rsp.ac.uk/events/the-role-of-institutional-repositories-after-the-finch-report/


Impact metrics for repositories - December 12th, 11am (GMT)

In this free RSP webinar, Mark MacGillivray, PhD candidate at the University of Edinburgh and founder of the Cottage Labs, will present why repositories' metrics are useful and which type of metrics can be collected. Mark will give a brief introduction on the technical developments on repositories' metrics and will present some examples from his current work.

Registration is open at http://www.rsp.ac.uk/events/impact-metrics-for-repositories/

[Reposted from the JISC Repositories Mailing List. I have attended previous webinars from the Repositories Support Project and they are always excellent!]

Read More

Academic and Professional Publishing - R. Campbell, E. Pentz & I. Borthwick (eds.) (Review)

Whilst Academic and Professional Publishing has emerged from today's context of changing technologies, increasing journal costs and the move towards open access, Robert Campbell’s opening historical overview demonstrates that such unrest has been a feature of publishing for some time now. Even as far back as 1927, it was noted that “librarians are suffering because of the increasing volume of publications and rapidly rising prices”. Evolution in scholarly publishing is clearly not new; so why is it only now that it has managed to pierce the mainstream so pervasively? Perhaps because the sector has recently entered a new phase - what Campbell terms “re-invention” - precipitated by an unprecedented swathe of new technologies, platforms and business models, and this idea serves as a recurrent motif throughout the rest of the book.

In any book about academic publishing, the reader should expect some strong opinions concerning the routes to, and sustainability of, open access, but this is definitely not an Us vs Them (that is, Publishers vs Academics/Librarians) treatise. Indeed such discussions only represent a very small fraction of the book; instead the text deals with the full range of aspects involved in the scholarly publishing chain - from authorship and editorial processes through to research impact and the user experience, passing through the technical, legal and financial complexities along the way. Whilst a proportion of the chapters are contributed by commercial publishers such as Wiley and Elsevier, there are also other perspectives presented including freelance consultants and those working across the wider academic and research community.

Every chapter offers value to the reader, but there are some highlights in particular. Irene Hames delivers a fascinating insight into the peer review process: how it works; what constitutes best practice; emerging models; and the key role played by the editor. Hames’ intelligent arguments weigh up both sides of the debate, ultimately concluding that many of the problems or criticisms directed at peer review can be traced back to the variable qualities and lack of standardisation across journals, as well as common misconceptions (for instance, many assume high impact journals offer a better quality of peer review which is not necessarily the case).

Michael Jubb’s discussion of the scholarly ecosystem reveals some interesting lessons for those in the information profession, citing evidence that many researchers have a limited awareness of the range of information resources available to them, and instead tend to remain loyal to a few key sources that they trust or find easy to use, largely supplemented by advice from colleagues. John S. Haynes’ contribution manages to provide a concise overview of the various journal publishing business models and the implications of each – the traditional reader pays model, pay-per-view rentals, author charges and hybrid open access options. Keith Webster’s chapter highlights the evolving role of libraries in particular, including the shift towards digital collections and fourth-generation libraries centred on the ‘learning experience’, open access issues and the emergence of changing access points for users. Michael Mabe attempts to answer the obvious, if unspoken, question “Does journal publishing have a future?”. Unsurprisingly, the answer is not very clear-cut, concluding with “Yes, probably…” (there is also a “but” in there somewhere!) – reflective of both the future uncertainties, as well as complex relationships and intricacies inherent in the publishing industry.

At around 500 pages, the editors have skillfully succeeded in curating a comprehensive assessment of the academic publishing sector today, and one that is both respectful of its past traditions as well as cognisant of future threats and opportunities. By managing to incorporate valuable industry insight into technical aspects such as publishing standards, licensing and workflows, as well as more policy-oriented discussions germane to the broader research sector, the book should garner a wide readership. Such strengths will no doubt allow Academic and Professional Publishing to position itself as an essential text for anyone interested in the scholarly publishing environment.

Academic and Professional Publishing is published by Chandos, September 2012, 510 p., £60.

Read More

Predicting your future research impact

There's an interesting article in this week’s Nature which examines whether certain variables can predict the future h-index of researchers. The h-index is a popular measure of research impact which aims to incorporate both quality (citations) and quantity (number) of papers: a h-index of n means an author has published n articles receiving at least n citations each.

The authors' model is based on five variables:

• Number of articles written; 
• Current h-index; 
• Years since publishing first article; 
• Number of distinct journals published in; 
• Number of articles in top tier journals (In this case, the journal titles also happen to include Nature and related titles. This could be adapted depending on the discipline, although the authors clearly state that this model is derived specifically for neuroscience so may not be transferable)

Unsurprisingly, the equation for predicting next year’s h-index depends strongly on the current h-index (a coefficient of 0.97) but this influence lessens as the forecasting horizon extends. Conversely the significance of the number of distinct journals an author has published in typically increases over time as a predictive factor – indicative perhaps that the researcher in question has broader expertise and a wider influence, and thus a potentially higher h-index.

There is also an online calculator you can use which predicts your future h-index based on the above variables.

Future impact: Predicting scientific success. Daniel E. Acuna, Stefano Allesina, & Konrad P. Kording Nature 489,201–202 (13 September 2012) doi:10.1038/489201a

Read More

Altmetrics and the future of bibliometrics

Journal impact factors have become one of the most frequently used tools for post-publication filtering and evaluation. However, journal level metrics which are based primarily on traditional citation data largely ignore the new channels of dissemination that have emerged in recent times. Indeed, the routes through which new research is found and shared today are more complex than ever before; whilst the might of PubMed and Google remains steadfast, articles are now discovered and shared through other domains as well, such as social bookmarking sites, Facebook and most notably, Twitter. And all the while, the growth in the volume of research also continues apace. So if our trusty friend ‘information overload’ insists on sticking around, we need to develop the right filters to allow us to make sense of it and to help identify the research that is really making an impact.

In other words, better discovery, navigation and management of research content is needed, along with richer, real-time indicators of impact. Article level metrics such as Altmetrics are broadly based on the idea of Scientometrics 2.0 – metrics which incorporate the emerging social channels of dissemination in order to generate richer measures of impact. Whilst nobody could argue that a handful of retweets and shares on Facebook is an indicator of research significance, social media and networking tools can certainly bring something to the table when it comes to dissemination,

So what do article level metrics or altmetrics look like in practice? PLoS integrate ALMs extremely visibly within their website interface through a metrics tab. This integration is key, and incorporates a broad range of measures including:

• Views and downloads
• Traditional metrics based on citation data 
• Social bookmarking sites (CiteULike and Connotea)
• Facebook & Twitter
• Blog aggregators of research blogs
• PLoS readers’ comments 

These metrics help authors and researchers to gauge who is reading an article, and if readers deem it significant enough to comment on or share with other researchers. The value of article level metrics to the author is that they can benchmark the ‘performance’ of their article against others, make decisions regarding where to publish in order to maximise reach, and communicate the impact of their research to funding agencies, potential employers and collaborators. For researchers, ALMs can also be used to assess the significance or value of a particular article within the field.

So what is the future for altmetrics? Phenomenon du jour or the future of bibliometrics? It is difficult to dismiss the value of the data captured by PLoS’s ALMs. They illuminate new and emerging dissemination channels, and richer data is always a good thing. The increased emphasis on research assessment and discovery demand comprehensive and sophisticated measures, and no doubt altmetrics will continue to be refined and developed over time into better measures. If more publishers start to collect this data, and institutional repositories can incorporate it through open APIs, altmetrics could be used to produce richer author- and institutional-level indicators as well.

Whether ALMs will ever manage to acquire the traction and authority of traditional impact factors is another question however. In contrast with established citation data, will many of these social tools still be around in five or ten years time? Who knows. As applications come and go, comparing data over time becomes impossible. Furthermore, the number of shares or links on Facebook or Twitter is likely to be influenced by the popularity of these tools themselves also - if nobody likes Twitter anymore, this will also manifest itself through lower ALMs. In this context, cross-sectional comparisons might still offer some insight, but longitudinal data will be largely meaningless. Preferences or fashions may also be discipline or community specific, making institutional or macro-level comparisons tricky. The fluidity and dynamic nature of social tools and behaviour makes constructing robust and systematic measures challenging at best. But as Tukey* argues: “Far better an approximate answer to the right question, which is often vague, than an exact answer to the wrong question, which can always be made precise”. With altmetrics, it seems to be a case of watch this space.

Tukey, J. W. (1962). The future of data analysis, Annals of Mathematical Statistics, 33, p. 1-67.

Read More