What are metrics?
The term has its origins in the world of print media, when books and journal articles monopolized scholarly communication at a time that there were no personal computers, the Internet, or mobile phones. We can define metrics as a set of quantitative methods used to measure, track and analyze print-based scholarly communication.
What are the characteristics of metrics?
The definition of metrics reflects three distinctive characteristics within the spectrum of impact:
- the inseparable nature of metrics and traditional forms of scholarly communication like books and journal articles,
- the drive force of metrics is providing academics and researchers with reliable and valid measures for evaluating research outputs,
- the connection between metrics and scholarship.
What are metrics used for?
Metrics have been originally designed to help librarians assess journals to which they should subscribe. Today metrics are very popular among librarians and researchers for a variety of reasons:
- to make publishing decisions,
- to determine research quality,
- to look into literature on a topic,
- to analyze institutional performance before applying for funding,
- to advance researchers careers,
- to help librarians select journal titles.
What are the levels of metrics?
Impact metrics can be grouped in three levels, focusing on the items different metrics choose to measure rather than the methods they use to produce their results. The levels of metrics are the following:
What are the limitations of metrics?
The degree of importance of metrics as well as the concerns about the use of them has increased over the years. Therefore, we will report some of the limitations of metrics.
- Sources used: Traditional online databases of research outputs like Scopus or Web of Science use a range of sources to calculate the final index however with the presence of social media the range of sources has increased. This is a problem that altmetrics try to solve.
- Unfair bias: The way in which some metrics are calulated can unfairly bias certain disciplines depending on their practices.
- Lack of sophistication: Many metrics fail to consider adequately the differences between individual researchers especially when considering the career stage of the researcher.
- High metrics don't necessarily demonstrate high-quality research: A famous example is the Andrew Wakefield article published in the leading medical journal The Lancet , which was retracted many years later "as several elements of the paper were proven to be incorrect, contrary to the findings of an earlier investigation". A useful resource for tracking retractions is Retraction Watch .
- Manipulation of results: Deliberate gaming practices like:
- some academics arrange for their colleagues to cite their work,
- some researchers cite their own work,
- some academics authoring a work, credit a colleague as a co-creator without having them worked on it or in some cases seen the output. This is called "gift authorship".
- Metrics are quantitative measures: Their nature focuses only on numbers but to get a complete picture, qualitative as well as a range of metrics and not a single one.
What is the Responsible Metrics Movement?
The discussion on metrics limitations mentioned above lead several groups in the development of the Responsible Metrics Movement which gathered worldwide support. The movement aims to make sure that the use of metrics is more balanced to produce a more rounded picture of impact which should not be based only on numbers.
Watch a short video below developed by the Office of Scholarly Communication of Cambridge that outlines the key principles of the Responsible Metrics Movement.
Under the umbrella of this movement, three essential initiatives have been developed: