Author Archives: Sean Burns
If a journal does not hold constant the number of citable items it publishes year after year, then its Impact Factor will always decline if the rate of citable items published in a two year period increases faster than the one year rate of cites to those citable items. Hence:
- Using the Impact Factor to compare a journal’s impact year after year may only make sense (if at all) if that journal holds constant the number of citable items it publishes year after year.
- Megajournals (potentially) do not hold constant the number of citable items they publish year after year.
- Therefore, the Impact Factor does not apply to megajournals.
Background & Explanation
The Impact Factor (IF) is a simple average. It’s the number of cites that accumulated in Year A to citable items (articles, etc.) a journal published in the previous two years, and then divided by the count of those citable items. To illustrate, to arrive at a 2015 IF for some journal, we sum the number of cites in year 2015 to the citable items a journal published in years 2013 and 2014, and then divide that by the count of those citable items.
There are many reasons why the IF is no longer a great (or appropriate) metric for evaluating or ranking journals (a web search will return many sources that criticize the IF). It was a useful metric when it was invented in the 1960s. Academic librarians, and others, even at that time, had to manage an increasing glut of scholarly journals and using the IF, aided with a decent understanding of Bradford’s Law, helped librarians manage print journal collections. It’s no longer necessary to use the IF for that, for a number of reasons that can be discussed at some other time. However, beyond such a scenario, the IF suffers because it is also an artifact of its time: the computational power and data storage capabilities in the 1960s were, obviously, limited and thus some selection process helped reduce the amount of data that needed crunching. Such selection shortcuts are less necessary now.
Therefore, these days the IF is not simply misused, it is also largely obsolete, and this is certainly true for megajournals like PLOS ONE and others. Specifically, because the IF is a simple average, using it to compare a single journal’s IF year after year only makes sense if that journal publishes a fairly constant number of citable items year after year: that is, if the denominator (count of citable items) in the IF stays [fairly] constant year after year while the numerator (citations to citable items) is left to vary. For most journals for most of modern history, this has been the norm. That is, journals generally publish X amount of articles per week, per month, or per quarter, and generally maintain this type of activity across time. Journal publishers do this because for most of modern history (i.e., before the web), they published in a print format and suffered the constraints that the economies of print placed on them (e.g., page constraints due to page costs). Hence, binding articles and disseminating them via an (e.g., weekly, monthly, or quarterly) issue made the best economic sense (limitations of scale in a print only era). Even though most journals have migrated to online formats and an increasing number no longer print physical copies, this remnant from the past, the constraint on pages per year and a set number of issues per year, remains a standard way of doing things, even if such a constraint is not placed on them by a publisher.
It could be that this way of managing a journal and publishing serially benefits certain kinds of work flows, and these benefits may be more pronounced for smaller research communities and journals, but this way of disseminating scholarly articles is not a necessity in digital publishing. Yet, the standard way of doing things is ingrained and many journals that are born online or born digital continue to publish periodically: that is, a fairly constant set of articles per issue and issues per year. Even some early journal software platforms, like Open Journal Systems, were built to accommodate this as a part of the software’s basic framework. What’s interesting, then, are those born digital journals that think past this assumption and have removed themselves from the constraints of printing. Enter the megajournal.
I’m fairly neutral (if not skeptical) about the existence of megajournals like PLOS ONE. I think those of us who study scholarly communication need to continue to flush out the possible implications with this kind of publishing and these kinds of platforms (that’s the job). That is, we don’t know enough yet to say whether these entities are good or bad for science. However, in the last several years, the IF for PLOS ONE has declined and several popular sources have drawn, what I think, are hasty generalizations about what that decline means. The only accurate thing we can say, right now, about the decline is that it is a result of publishing works, over a two year period, at a rate that is faster than the rate of incoming cites that accumulated in a one year period (for the 2-year IF). To illustrate this, below is a table showing the counts of cites, citable items, and 2-year IF scores taken from Thompson Reuters’ Journal Citation Reports for the journal PLOS ONE. I’ve added two additional columns that show the percentage change in the number of cites and the percentage change in the number of citable items. As we can see, the 2 year IF score declines each year simply because the percentage increase for the number of citable items published is larger than the percentage increase in the number of cites to those items:
|Year||Cites||Cites Perc Δ||Citable Items (CI)||CI Perc Δ||2-yr IF||CI Perc Δ > Cites Perc Δ|
(By the way, there’s not enough data in the table above to say anything remotely conclusive about trends, no matter what anyone says.)
I’m looking forward to the 2015 release of the Journal Citation Reports simply because it will be interesting to see the reactions to PLOS ONE‘s score. A couple of summers ago (and the one before that), when the 2014 scores were released, there were a number of blogs that predicted the decline or the fall of PLOS ONE because its IF fell again. Even though some of those posts explain the IF and refer to a decreasing number of submissions to or publications by PLOS ONE, which may be true but may be true for reasons such as the size of the scientific community, funding sources, etc. and not to authors’ reactions to a decline in PLOS ONE‘s IF, no one, as far as I can tell, actually refers to the simple problem with the moving parts of the IF as a fraction, and by entailment, why the rate of cites might be lower than the rate of citable items. That latter issue leads to many more interesting and illuminating questions.
This post originally appeared on my personal blog.