Rhetoric Matters

A copy of a posting to CRTNET:

I want to thank Tim Levine for posting his commentary on the “lameness” of NCA journals. I thank him not because I agree with his conclusions, but because his posting offered a very teachable moment for one of my classes.

I teach the Communication Department’s Introduction to Graduate Studies course at the University of Maryland, a course that is both about socialization to the university/department and to the discipline as well as an introductory discussion of research methods. The distribution of Tim’s posting in class generated considerable discussion, particularly about how research claims/arguments are made and the evidence or data used to support those arguments. Two lessons emerged from this discussion.

One lesson that the students quickly identified was the importance of care and precision in moving from data to the larger claims or arguments that are advanced in research. This, of course, relates to the second main lesson here—it is essential to understand how data/evidence are collected and used when making evaluative/normative claims about some empirical phenomenon.

Tim relies primarily on ISI’s Journal Citation Reports impact factor for his argument that NCA journals are “lame.” (Tim also employs Google Scholar—though my own experience with tracing citations through this source is that it is still quite new and unreliable, missing some obvious citations and including others that are unusual and rare.)

Interestingly, the year-by-year impact factors that ISI calculates are “the average number of times articles from the journal published in the past two years have been cited in the JCR year.” So, ISI will average the number of times articles published in QJS in 2006 and 2007 are cited in other ISI journals in 2008 for QJS’s 2008 impact factor.

The result, of course, is that there are wide shifts in impact ranking over the 11 years of ISI reports. Just last year, for example, Communication Monographs had the highest impact factor ranking of any NCA or ICA journal (and Communication Research)—it was #4 in 2007 with an impact factor of 1.512. The next closest journal was Communication Research at #5 (1.481). The Journal of Communication was #15 in 2007 with an impact factor of 1.156. Similar year-to-year shifts in overall impact factor rankings among Communication journals happen for virtually all of the journals Tim mentions. I’ve posted two graphs that display these shifts here: ISI Journal Graphs.

While it is true that ICA journals and CR are generally ranked higher in impact factor among Communication journals than are NCA journals, my students were quick to note, given the manner of data calculation by ISI and the significant shifts in this data, that these rankings may be explained by factors other than simply journal quality—they may involve citation and publication practices, editorial shifts and changes in journal focus, the differences in research and citation expectations in the humanities as opposed to the social sciences, etc. The students did conclude, though, that there is little here to justify the argumentative/normative conclusions that NCA journals are “lame,” a “home for irrelevant scholarship,” or a “poor choice for publication outlet.”

Tim asks in his post if he is “just missing something and reading the data wrong?” As we concluded in my course, that may be the case. It may also be the case that while the data on citation impact and patterns reveal much, they may not justify Tim’s rather sweeping indictment of NCA’s publications.