About the author

Nick Oswald

Nick is a molecular biologist-turned-publisher. After a PhD in Developmental Biology and an eclectic seven years in biotech he is now Editorial Manager of Neuroendocrinology and the founder and Editor-In-Chief of Bitesize Bio. You are welcome to connect with Nick on LinkedIn

To enable tagging you will need to register on Bitesize Bio. We're sorry for the inconvenience, but it's free, only takes a few seconds, and it will enable you to view our seminars for free, ask questions from the professional community, and take part in the lively community of Bitesize Bio

How do you measure how good you are as a scientist?

Or how would you compare the impacts of two scientists on a field if you had to decide which would get a grant you were allocating?

Measuring scientific performance is both more complicated, and important, than it might seem at first. Various methods for measurement and comparision have been proposed, none of which are perfect.

And you should care about this because these metrics are increasingly being used by Funding Bodies and employers to allocate grants, jobs etc. So your perceived scientific peformance score could seriously impact your ability to get funding, or get a job.

So how could scientfic peformance be measured? The methods that might spring to mind at first are:

• Peer review. A good idea in principle, but it is subject to human nature so perceived performance will inevitably be affected by personal relationships. Also if a lesser known scientist publishes a ground-breaking publication he/she would likely get less recongnition than if the same paper was published by a more eminent colleague.
• Number of articles published. A long publication list tends to look good on your CV, but the number of articles published gives no indication of the impact that your publications have had on the field. A small number of publications that have been well heeded by colleagues in the field (i.e. they are cited often) are better than a long list of publications cited poorly, or not at all.
• Average number of citations per article published. So if it’s citations we are interested in, then surely the average number of citations per article is a better number to look at. Well, not really. The average could be skewed greatly by one highly-cited article so does not allow a good comparison of overall performance

The h-index

In 2005, Jorge E Hirsch of UCSD published this paper in PNAS in which he put forward the h-index as a metric for measuring and comparing overall scientfic productivity of individual scientists.

The h-index has been quickly adopted as the metric of choice for many committees and bodies.

Conceptually, the h-index is pretty simple. You just plot papers versus the number of citations you (or someone else) have received and the h-index is the number of papers at which the 45degree line (citations=papers) intercepts the curve, as shown in the diagram. i.e. h= the number of papers that have received at least h citations.

For example if you have an h-index of 20, it means you have 20 papers with at least 20 citations. It also means that you are doing pretty well with your science! Hirsch reckons that after after 20 years of research, an h index of 20 is good, 40 is outstandind 60 is truely exceptional.

The advantage of the h-index is that it combines productivity (i.e. number of papers produced) and impact (number of citations) in a single number. So both productivity and impact are required for a high h index; neither a few highly cited papers or a long list of papers with only a handful of (or no!) citations will yield a high h index.

In his paper, Hirsch shows that successful scientists do indeed have a high h-index. A simple example is that Nobel prize winners in physics all have high h-indices (84% had an h of at least 30).

Limitations of the h-index

But as attractive as it is to have a single number that measures scientific performance, it is only a rough indicator and should only be considered as such. Hirsch himself writes:

Obviously a single number can never give more than a rough approximation to an individual’s multifaceted profile, and many other factors should be considered in combination in evaluating an individual. This and the fact that there can always be exceptions to rules should be kept in mind especially in life-changing decision such as the granting or denying of tenure.

Limitations of the h-index include:

• It does not take into account the number of authors on a paper. A scientist who is the sole author of a paper with 100 citations should be given more credit than one who is on a similarly cited paper with ten co-authors.
• It penalises early career scientists. Outstanding scientists with only a small number of publications cannot have a high h-index, even if all of those publications and ground-breaking and highly cited. This Wikipedia article gives a great illustration of this: “Had Albert Einstein died in early 1906, his h-index would be stuck at 4 or 5, despite his being widely acknowledged as one of the most important physicists, even considering only his publications to that date.”
• Review articles have a greater impact on the h-index than original papers since they are  generally cited more often.

Calculating the h-index

There are several online resources you can use to directly calculate a scientist’s h-index. The most established are ISI Web of Knowledge, and Scopus, both of which require a subscription (probably via your institute ), but there are free options too, one of which is Publish or Perish.

If you check your own, or someone else’s h-index with each of these databases, you might get a different value. This is because each uses a different database to count the total publications and citations. ISI and Scopus use their own databases, and Publish or Perish uses Google Scholar. Each database has different coverage, so will come up with different h-index values. e.g. ISI has good coverage of journal publications, but poor coverage of conferences, while Scopus covers conferences better, but has poor journal coverage pre 1992 (this was shown by a 2007 study, click here for the abstract).

How do you measure up?

My personal feeling is that the  h-index provides a useful metric for scientific performance, but only when viewed in the context of other factors. It is no substitute for reading through the publication list, talking to referees, peers etc.

But there is no doubt that among those who regulate the funding of science there is an increasingly bean-counting mentality. So a single number that can be used to compare scientists, and that fits neatly into an Excel column must be very attractive to the powers-that-be. This probably accounts for the very rapid uptake of the h-index as an accepted metric and probably means that it is here to stay.

So how do you think that will affect YOUR job prospects? Does your h-index measure up?