Are altmetrics (aggregator) studies reliable at all?
To be presented at the third workshop (#FAIL2016) at ICWSM-16.
Abstract:
Traditionally, bibliometric studies heavily rely on third party data often provided by commercial publishers (e.g., Thomson Reuters’ Web of Science for citation data). We experience the same situation when conducting altmetrics research, i.e. bibliometrics with social media data. Challenges appear on two levels and, hence, offer great potential to question the reliability of altmetrics studies in general. It is the access to and the transparency of 1) social media data providers (e.g., Twitter) and 2) altmetrics aggregators (e.g., altmetric.com) which cause the difficulties.
We present results of a study of various altmetrics aggregators and the variance in altmetric data found, especially the number of scientific articles and the intensity of social media engagement reported for those articles (Jobmann et al., 2014; Peters et al., 2014). We will also reflect on what might cause these differences and the implications for the methodology in altmetrics studies.
References
Jobmann, A., Hoffmann, C.P., Künne, S., Peters, I., Schmitz, J. & Wollnik-Korn, G. (2014). Altmetrics for large, multidisciplinary research groups: Comparison of current tools. Bibliometrie – Praxis und Forschung, 3, http://www.bibliometrie-pf.de/article/viewFile/205/258
Peters, I., Jobmann, A., Eppelin, A., Hoffmann, C. P., Künne, S., &Wollnik-Korn, G. (2014). Altmetrics for large, multidisciplinary research groups: A case study of the Leibniz Association. In Proceedings of the Libraries in the digital age conference, Zadra, Croatia. http://ozk.unizd.hr/proceedings/index.php/lida/article/view/162/138
Personal statement to address difficulties during research:
When studying altmetrics by using altmetric aggregators comparisons of results is often difficult – or better: impossible – because of the a priori decisions made by the aggregators. For example: it is problematic to find out what action they count as – one reflection – of readership or what views are merged into the category “html views”. I hope that standards will help to solve this problem in future – although we then need to better understand the properties of the social media actions underlying altmetric data.
