Category Archives: Research

Megajournals and the Impact Factor


If a journal does not hold constant the number of citable items it publishes year after year, then its Impact Factor will always decline if the rate of citable items published in a two year period increases faster than the one year rate of cites to those citable items. Hence:

  1. Using the Impact Factor to compare a journal’s impact year after year may only make sense (if at all) if that journal holds constant the number of citable items it publishes year after year.
  2. Megajournals (potentially) do not hold constant the number of citable items they publish year after year.
  3. Therefore, the Impact Factor does not apply to megajournals.

Background & Explanation

The Impact Factor (IF) is a simple average. It’s the number of cites that accumulated in Year A to citable items (articles, etc.) a journal published in the previous two years, and then divided by the count of those citable items. To illustrate, to arrive at a 2015 IF for some journal, we sum the number of cites in year 2015 to the citable items a journal published in years 2013 and 2014, and then divide that by the count of those citable items.

There are many reasons why the IF is no longer a great (or appropriate) metric for evaluating or ranking journals (a web search will return many sources that criticize the IF). It was a useful metric when it was invented in the 1960s. Academic librarians, and others, even at that time, had to manage an increasing glut of scholarly journals and using the IF, aided with a decent understanding of Bradford’s Law, helped librarians manage print journal collections. It’s no longer necessary to use the IF for that, for a number of reasons that can be discussed at some other time. However, beyond such a scenario, the IF suffers because it is also an artifact of its time: the computational power and data storage capabilities in the 1960s were, obviously, limited and thus some selection process helped reduce the amount of data that needed crunching. Such selection shortcuts are less necessary now.

Therefore, these days the IF is not simply misused, it is also largely obsolete, and this is certainly true for megajournals like PLOS ONE and others. Specifically, because the IF is a simple average, using it to compare a single journal’s IF year after year only makes sense if that journal publishes a fairly constant number of citable items year after year: that is, if the denominator (count of citable items) in the IF stays [fairly] constant year after year while the numerator (citations to citable items) is left to vary. For most journals for most of modern history, this has been the norm. That is, journals generally publish X amount of articles per week, per month, or per quarter, and generally maintain this type of activity across time. Journal publishers do this because for most of modern history (i.e., before the web), they published in a print format and suffered the constraints that the economies of print placed on them (e.g., page constraints due to page costs). Hence, binding articles and disseminating them via an (e.g., weekly, monthly, or quarterly) issue made the best economic sense (limitations of scale in a print only era). Even though most journals have migrated to online formats and an increasing number no longer print physical copies, this remnant from the past, the constraint on pages per year and a set number of issues per year, remains a standard way of doing things, even if such a constraint is not placed on them by a publisher.

It could be that this way of managing a journal and publishing serially benefits certain kinds of work flows, and these benefits may be more pronounced for smaller research communities and journals, but this way of disseminating scholarly articles is not a necessity in digital publishing. Yet, the standard way of doing things is ingrained and many journals that are born online or born digital continue to publish periodically: that is, a fairly constant set of articles per issue and issues per year. Even some early journal software platforms, like Open Journal Systems, were built to accommodate this as a part of the software’s basic framework. What’s interesting, then, are those born digital journals that think past this assumption and have removed themselves from the constraints of printing. Enter the megajournal.

I’m fairly neutral (if not skeptical) about the existence of megajournals like PLOS ONE. I think those of us who study scholarly communication need to continue to flush out the possible implications with this kind of publishing and these kinds of platforms (that’s the job). That is, we don’t know enough yet to say whether these entities are good or bad for science. However, in the last several years, the IF for PLOS ONE has declined and several popular sources have drawn, what I think, are hasty generalizations about what that decline means. The only accurate thing we can say, right now, about the decline is that it is a result of publishing works, over a two year period, at a rate that is faster than the rate of incoming cites that accumulated in a one year period (for the 2-year IF). To illustrate this, below is a table showing the counts of cites, citable items, and 2-year IF scores taken from Thompson Reuters’ Journal Citation Reports for the journal PLOS ONE. I’ve added two additional columns that show the percentage change in the number of cites and the percentage change in the number of citable items. As we can see, the 2 year IF score declines each year simply because the percentage increase for the number of citable items published is larger than the percentage increase in the number of cites to those items:

Year Cites Cites Perc Δ Citable Items (CI) CI Perc Δ 2-yr IF CI Perc Δ > Cites Perc Δ
2014 177706 35.07% 54945 47.59% 3.234 T
2013 131563 72.03% 37229 81.58% 3.534 T
2012 76475 68.00% 20503 84.30% 3.730 T
2011 45521 44.95% 11125 56.25% 4.092 T
2010 31404 7120 4.411

(By the way, there’s not enough data in the table above to say anything remotely conclusive about trends, no matter what anyone says.)

I’m looking forward to the 2015 release of the Journal Citation Reports simply because it will be interesting to see the reactions to PLOS ONE‘s score. A couple of summers ago (and the one before that), when the 2014 scores were released, there were a number of blogs that predicted the decline or the fall of PLOS ONE because its IF fell again. Even though some of those posts explain the IF and refer to a decreasing number of submissions to or publications by PLOS ONE, which may be true but may be true for reasons such as the size of the scientific community, funding sources, etc. and not to authors’ reactions to a decline in PLOS ONE‘s IF, no one, as far as I can tell, actually refers to the simple problem with the moving parts of the IF as a fraction, and by entailment, why the rate of cites might be lower than the rate of citable items. That latter issue leads to many more interesting and illuminating questions.

This post originally appeared on my personal blog.


“Neverland” or “networked publics”? : A review of “It’s complicated: The social lives of networked teens”

A few months ago, I started collecting data for my Ph.D dissertation. Since I study interpersonal and group behavior in multiplayer online games, I got quite involved in the online forums of my studied game. I also interviewed many gamers via Skype text chat. As I expected, most of these gamers are teens (aged 14 to 17).

Their stories are not so different from my assumption: They love their parents but they want to escape from the “boring” family lives; they crave for friendships but they struggle with finding “true friends” in school; they look forward to love and romance but they are afraid of responsibilities; they expect to grow up but they are scared of the “cold” adult world.

What is out of my expectation is how Internet and social media have become a “natural” part of teens’ social lives. Many of these teens described how disappointed they felt with their offline lives but how wonderful those friends they made online; and how they felt restricted at home and at school but how much freedom they enjoyed in social media platforms. Those “cool” places – Facebook, Youtube, Tumbler, Snapchat, Instagram, online games, etc.— become Peter Pan’s “Neverland” where they can fly and “never grow up.”

But teens are not trying to live virtually or escape from the real world. Instead, Internet and social media compensate their living world and extend their social lives. Their social lives are more complicated than we, as adults, assume. They are also, sometimes, more self-conscious than we expect. I am very impressed by those teens’ stories. So I read danah boyd’s new book It’s complicated: The social lives of networked teens.

This book is definitely a good work to help adults understand the “mysterious” teens in the Internet Era. Its main goal is to depict young people’s experience of using social media, and the role of social media in shaping their lives. boyd’s work is based on two essential understandings: First, teenhood is an awkward period “between childhood and adulthood, dependence and independence” (p. 17), and the penetration of social media into many aspects of teens’ lives makes contemporary youth more complex than before. Second, many adults worry about youth engagement with social media, but few listen to teens’ stories or understand them from teens’ own stance. How teens use social media to make sense of the world around them may be very different from adults’ imagination.

Thus, one of the most important contributions of this book is its focus on the teens’ own voice. Drawing on rich qualitative and ethnographic material that she collected from 2003 to 2012—and interview data con­ducted from 2007 to 2010, boyd provides vivid images of the old and new impacts of social media on teens’ lives, and the quality that social media add to or take away from teens’ social lives. Based on detailed quotes from interviews and in-depth analysis of teens’ true stories, audience can better understand why and how teens use social media from teens’ own perspective.

boyd’s another focus is “networked publics,” a main concept throughout the whole book.  boyd explains this concept at the very beginning: “[s]ocial media has become an important public space where teens can gather and socialize broadly with peers in an informal way. Teens are looking for a place of their own to make sense of the world beyond their bedrooms. Social media has enabled them to participate in and help create what I call networked publics” (p.5). Networked publics is related to Ito (2008, 2010) ‘s work on digitally networked media and on “hanging out, messing around, and geeking out,” as boyd mentioned in notes on page 222.  But boyd uses networked publics here in a broader sense: Networked publics represent a complex interaction between technological affordances (i.e., networked technologies) and networked people. “Through engagement with publics, people develop a sense of others that ideally manifests as tolerance and respect.”

Based on this understanding, this well-researched book is organized into eight chapters that address seven important issues concerning youth engagement with social media, including identity, privacy, addiction, danger, bullying, inequality, and literacy.  As a conclusion, the last chapter summarizes the impacts of networked publics on contem­porary youth: To create a world of their own (not shaped by parents and teachers), teens use social media to seek networked publics. Teens both construct and participate in such networked publics in their everyday lives, “to see and be seen,” to belong to a broader public world, and to “build networks of people and information” (p. 201).

As a researcher, I appreciate boyd’s endeavor to let us, adults who “have power over the lives of youth” (p. 28), better understand teens’ actual social lives in the networked era. boyd expresses her hope that adults and youth collaborate together to create a networked world that we all want to live” (p. 213), which is also our hope. As an adult who had teenhood not long time ago, this book makes me think about my own teenhood and how important it is for my life experience. Charles Dickens’ words may still be appropriate to describe how complicated someone’s teenhood can be in today’s network society:

It was the best of times, it was the worst of times; it was the age of wisdom, it was the age of foolishness; it was the epoch of belief, it was the epoch of incredulity; it was the season of light, it was the season of darkness; it was the spring of hope, it was the winter of despair; we had everything before us, we had nothing before us; we were all going direct to Heaven, we were all going direct the other way. 



boyd, d. (2014). It’scomplicatedThe social lives of networked teens. New Haven, CT: Yale University Press.

Ito, M. (2008). Introduction. In K. Varnelis (Ed.), Networked Publics (pp. 1–14). Cambridge, MA: MIT Press.

Ito, M., et al. (2010). Hanging out, messing around, and geeking out: Kids living and learning with new media. Cambridge, MA: MIT press.


Digital Divide Research: one myth, problem and challenge.

The Myth: Digital Divide has a small literature. Pretty much, almost every book or paper on the topic will say this. I used to believe that not enough work has been done on Digital Divide, until I started studying for my qualifying exam. Fortunately or unfortunately I found out that the literature is actually very large. The problem is that the digital divide research is spread throughout all kinds of disciplines, such as: ICT4D, Community Informatics, HCI, Social Informatics, Sociology and Communication studies. In fact, the literature is not new, because it goes way back when academics were studying the diffusion of telephones and televisions.

The Problem: Quantitative approaches are addressed to answer the wrong questions. A lot of the research done on digital divide is done quantitatively. They rely on the data collected by International Telecommunication Union, World Bank and other agencies. And what these researches do is to identify a digital gap and try to correlate that gap with some sort of social, economic or political issue.  For example, there is a cross country study done by Luis Andres, he says that, based on his quantitative analysis, in order to bridge the digital gap we need to liberalize the telecommunication market to promote internet provider competition. I agree, but Brazil has had this free market for about 15 years, and we still have a vast digital divide. So, obviously, this is not an issue for Brazil, something must be happening that is keeping the divide wide. What I’m trying to say here is that in order to fully understand and propose meaningful solutions, the digital divide research requires local and context based research. It doesn’t matter if it’s quantitative or qualitative, I don’t want to get into this argument, but we need to understand that each country has its own set of policies, people have different cultural backgrounds, so solutions need to be tailored and not based on general auto analysis.

The Challenge: “How to talk to policymakers?”. Policymakers of the digital divide tend to have a technological deterministic perspective. They focus on single factors, such as “access”, because they are convenient since they are easy to measure. These simple measures can be used to influence public opinion since lay people can relate to them. Policymakers also need to justify allocation of resources, which is easier to do when they can create benchmarks (Barzilai-Nahon, 2006). So policymakers are strung up on numbers, and how can we show them that subjective factors such as education and training can be of much better value to promote the digital inclusion than pure access? I don’t want to blame policymakers for approaching the digital divide quantitatively, but I’d like to leave this challenge for us, digital divide scholars, to realize a way to start conversations with people that can only see numbers.

Barzilai-Nahon, Karine. 2006. “Gaps and Bits: Conceptualizing Measurements for Digital Divide/s.” Information Society 22:269-278.

On Building Social Robustness and Enduring Computing

As many of you know, I am now directing a Social Informatics (SI) Group in a School of Informatics and Computing (SoIC) at Indiana University Bloomington. The SI group is quite unique in Informatics/Computer Science/Information Studies, it that is has chosen to oriented itself explicitly to the field of Science, Technology, and Society (STS, also referred to as Science and Technology Studies). I am also thinking about retirement in the next 3-5 years. Being in these situations has shaped the research agenda that follows.

My current research is all framed generally within Socially Robust and Enduring Computing. SREC is based on the notion that developing a notion of social robustness, comparable to the technical notion of robustness in Computer Science, is a goal worth pursuing. I have developed SREC with colleagues in Trento, Italy.

My main research time commitment at the moment is to a writing project on Value(s) with Maurizio Teli, a young researcher at the Foundation in Trento, where I spend a couple months every year. My interest in this area grew out of efforts to identify the forms whereby and the extent to which computing professionals are responsible ethically for the current economic and social crisis set off in finance. Maurizio’s and my value(s) project is a continuation of this work on the crisis and is linked to the project of David Graeber in Debt: The First 5000 Years, itself a work that builds on much of the recent anthropology of value. That is, we want to give a similar account of the ways in which value and values are and should be treated and thought about in the reproduction of current social formations. Such an account is made necessary by the ways in which contemporary reproduction is increasingly detached from the prior industrial dynamics but which has not yet established a new dynamic. In our view, establishing new social formation reproduction dynamics requires identification of new values, new institutions for pursuing those values, and new means to measure especially value relating to the success or failure of establishing these new values and institutions. A major point we wish to make regards the increasingly larger role in these new dynamics we see being played by common pool resources, the focus of Eleanor Ostrom, winner of the 2009 Nobel Prize in economics, and, until she died last Spring, a valued colleague here in Bloomington.

It is my hope that this writing will be paralleled by a research and demonstration project in Trentino on new systems, including information systems, for supporting the independent living of Seniors. This Suitcaseproject will build on my previous work in disability studies and technology, as well as more general ethnography in this region. Another aspect of the Trento ethnography is an attempt to understand what has made the region relatively hospitable to Participatory Design. PD is the focus of what I hope to and expect will be my last permanent contribution to the curriculum in the SoIC. In addition, I am working on another, related writing project, a text on Organizational Informatics, with Stefano De Paoli, another researcher. This text will incorporate much of the work behind my 2011 AAA paper in the business anthropology sessions as well as my current teaching, including my course on the Ethnography of Information.

A final areas of research, this time in collaboration with two SoIC graduate students, Nic True and Shad Gross, is on Massive Multiplayer Online Games (MMOGs). In this work, we engage the current interest in Big Data, intending to show how some of the epistemological shortcomings in its standard approaches can be address when it is triangulated with ethnography. In our case, we argue that a preliminary ethnography of gaming can provide clearer direction regarding what we should be looking for in the automated analysis of large corpora of game play data. This work is directly related to the effort in SoIC to create a professional masters degree in Big Data.

Presented in this way, it should be easy to see, as I said initially, the multiple ways in which this research agenda is a function of my current position. While I have participated in the AAA meetings and CASTAC occasionally since I went to Indiana in 2004, this occasional connection has not been enough to justify systematic orientation of my research toward anthropology. Ironically, when I studied the careers of anthropologists interested in STS in the 1980s, I found a similar phenomenon; there were few if any examples of individuals who developed these interests while sustaining strong connections with academic anthropology. I should mention that my efforts to interest Indiana University Bloomington Department of Anthropology scholars in this type of work has born little fruit.

I mention these things as a warning: Interest in the anthropology of science, technology, and computing is not automatically, or even generally, a good way to build a career in anthropology. Working in and through vehicles like CASTAC should thus be understood as essential to the work of anthropologists who wish to continue to do so.

This blog post can also be seen at:

The difficulty of interdisciplinary narratives

This post is also cross posted on

Interdisciplinarity is a concept that is frequently lauded but notoriously challenging to realize. Practical realities like publication and tenure requirements and conceptions of what counts as valid research in established fields tend to push even the best-intentioned academics toward more well-trodden and recognizable paths, despite the potential benefits of interdisciplinary perspectives and collaborations. There is also the intellectual challenge of bringing possibly quite different intellectual traditions into conversation with each other in substantive and non-dilettantish ways.

This last challenge in particular is one that I have been wrestling with in framing my dissertation research. I have been for the last few months working on my dissertation proposal, with my literature review chapter being inordinately irksome. My research deals with the ways in which interactive technologies can mediate engagement with the world in our everyday lives, and aims to develop grounded theoretical frameworks and critical perspectives that can inform technology design and critique. I am actively trying to bridge Social Informatics/STS, HCI design, and philosophy of technology in order to consider social implications of technologies in ways that can speak to their design.

My work is thus suspended in the middle of these three areas, with a number of others being potentially relevant (e.g., media studies, cultural studies, critical theory, information and information society studies, infrastructure, etc.). Just identifying which literatures and theoretical perspectives I should engage with, and how, and why, has been a daunting task. But now that I have this more or less pinned down and have been trying to corral it all into some manageable structure for my literature review I have discovered another, perhaps more basic but still vexing, challenge: that of crafting a narrative to provide the theoretical backdrop for my research.

My Informatics colleagues and I have joked about wishing we were doing something easy, like theoretical physics, because then at least we would have a fairly well-defined problem space (or an established paradigm, in the Kuhnian sense). A scholar working in such an established discipline is able to present (or assume) a boring, predictable setup for her research: the history of her discipline, its intellectual traditions, questions, and progress made toward addressing the problem at hand. Included in this nifty paradigmatic package are also shared conceptions of what count as acceptable problems, methods, and solutions. Of course, even research that fits this pattern has its own significant challenges that I do not mean to trivialize. But at least framing and rhetoric can be fairly straightforward.

When choosing to take a fundamentally interdisciplinary perspective, however, these things cannot be taken for granted. If the problem/question is not well set up (and sanctioned qua problem/question) by any single discipline, one can be left to at least some extent appealing to the audience’s intuitive and practical sense of something being a real problem in the world that requires such a newfangled approach. And one person’s interesting question is often another person’s uninteresting assumption.

Nonetheless, I am convinced that the issues I am wrestling with require multiple perspectives to understand and address, and that understanding and addressing them is important. So I’m going to keep wrestling, and if I figure out this whole interdisciplinary narrative thing I’ll let you know.

%d bloggers like this: