Skip to Main Content

Research Impact

Disciplinary impact

Like nuclear energy, the impact factor is a mixed blessing. I expected it to be used constructively while recognizing that in the wrong hands it might be abused.

Eugene Garfield 

Do all disciplines use the same metrics?

It is not right to ignore the fundamental differences between the outputs and expectations of different disciplines when we try to measure impact e.g. this is different between a humanities researcher and a social science one, due to the inherent differences in citation practices and publication trends within these academic fields.

There are three major disciplines the Sciences, the Social Sciences and Arts & Humanities. To understand the differences between units, you may find below some representative examples of the unique practices around research impact and output. Please note that not all units are included. 

Check the tabbed box below to understand trends and debates in each field.

 Biomedical Sciences

  • Trends
    •  Bibliometrics are dominating in biomedical sciences research. However there is a strong tendency for using altmetrics as well. 
  • Debates
    • The American Society for Cell Biology  released in 2012 The Declaration on Research Assessment (DORA) which recognizes the need to improve the ways in which the outputs of scholarly research are evaluated. 
    • The Becker Medical Library Model  for Assessment of Research Impact. This is a framework for tracking dissemination of research outputs and activities to locate indicators that demonstrate evidence of biomedical research impact. It is intended to be used as a supplement to publication analysis. It recognizes five types of impact tailored to the needs of  biomedical researchers, advancement of knowledge, clinical implementation, community benefit, legislation and policy, and economic benefit. This model suggests a wider spectrum of impact that includes a wide variety of contributions like gray literature, data outreach efforts and social media applications.

 Mathematics

  • Trends
    • Journal articles may not receive citations for several years and due to changes in the direction of the research may receive several citations two to three years later.
    • Monographs are still widely used in mathematics.
    • Peer evaluations remain the best primary qualitative measure for mathematicians.
  • Debates

 Physics

  • Trends
  • Debates
    • Prepublication versions of articles uploaded to open access repositories are considered valid enough to be cited. 

 Computer Science

  • Trends
    • Code is a relatively new scholarly output, much different from traditional types of sources like articles and books.
    • Computer scientists use designated online repositories to make their code or research data freely available. Such repositories are: GitHub which is (code-focused), and Figshare and Dryad which are (data-focused).
  • Debates
    • Computer scientists disagree with the use of bibliometrics and altmetrics in terms of measuring impact.

Here are covered only a few specific subjects within the social sciences that illustrate the issue of research output and impact measurement.

 Communication

  • Trends
    • There are two types of Communication researchers, those who focus on quantitative and theoretical methods and produce  monographs and journal articles and those who focus on public areas of communication like journalism and public relations who produce public-facing outputs like press releases, speeches, newspaper articles, fictional works etc. 
    • Extended use of bibliometrics and altmetrics. Altmetrics are widely used from those who publish the public-facing outputs.

 Political Science

  •  Trends
    • There is a highly diverse set of research interests ranging from e.g. theoretical analysis on democracy to studies of contemporary political events and campaigns. Consequently, the research outputs range from journal articles to opinion pieces, blog posts, television and radio interviews and social media engagement. 
    • Bibliometrics and altmetrics with the exception of times cited are not extensively used due to the extended use of shortlists with high impact rated journals suggested by the university departments. 
  • Debates
    • New scientists are being discouraged to publish outside the suggested journals. If they decide to do so, bibliometrics and altmetrics can be useful to them.

 History

  • Trends
    • Journal articles and monographs are the most common expected research outputs. 
    • Fuzzy metrics of the book publishing as well as journal metrics are important in this field of research.
  • Debates
    • Small steps in the adoption of altmetrics as an impact measure. 

 English and the Humanities

  • Trends
    • Monographs are still widely used to prove the impact of a researcher in this field.
    • English researchers frequently engage in producing journal articles, conference papers, digital projects, and data-driven works which shows the expansion of humanities into new disciplinary domains. 
  • Debates
    •  There is a need for better citation-based metrics for this field as it heavily relies on monographs and edited volumes.
    • There is resistance in the use of altmetrics which is an area of great potential growth.

 Dance and the Arts

  • Trends
    • Academia values the production of scholarly outputs as well as the production of creative works, such as performances, objects, inventions, designs, recordings, and compositions.
    • Impact is mostly based on book reviews, peer evaluations, and fuzzy metrics (e.g. ticket sales or number of shows).

What is the role of interdisciplinarity in research impact?

A growing trend within academia is interdisciplinarity, the combination of two or more academic fields around a research problem or area. Interdisciplinary projects offer opportunities for researchers:

  • to increase their impact outside their disciplines,
  • to be motivated when they come from fields with weak citation cultures.

However, interdisciplinarity reveals serious difficulties in measuring the impact after is it achieved. This resulted in the development of models like the Eigenfactor that helps the creation of percentage-based metrics that allow you to compare impact across a range of disciplines. Finally, altmetrics can be very promising in measuring the impact of projects that include data sets, presentations and other forms of interdisciplinary research.