Thinking Through Peer Review

A number of factors have conspired to preoccupy my mind with the issues of academic thinking, writing, and publishing.  This month I joined the GradHacker team, so my musings will soon make their appearance at Inside Higher Ed.  So I’ve been thinking about the nature of blogs and blogging and the ever-shifting place of blogs in the pantheon of academic produce.  Katherine and I have also just submitted a forum piece on Twitter use (micro-blogging) at conferences.  But we’ve also, in the past month hosted a History Carnival, which turned our attention to the various blogs of many colleagues:  individual blogs, team blogs, and institutional blogs.  Finally, we’ve come upon our one year anniversary at Stillwater Historians, and while writing a blog, we’ve found, leads to near constant consideration of mission and objective, the occasion makes me all the more introspective.  And I suppose it also merits mention that I’m heading into my first year of really engaging the job market (though I’ve applied for a couple things here and there previously) and I’m forever considering how the various things I do effect my marketability—and blogs, it seems, are becoming more and more of what I do.

Original from Gideon Burton at

It was in talking with Alex Galarza of GradHacker that I came to realize that despite our interest in many of the same themes, our team approach, our interest in technology, and very similar ages of our blogs (both about a year), SH and GradHacker really couldn’t be more different in origin and organization.  This made me appreciate just how wide the spectrum of blogs has become.  There’s been a lot of talk about the many benefits of blogging for academics.  Increased interaction, networking, broader dissemination of research and ideas, a faster pace of intellectual discourse, and access to mainstream audiences—these are all good things.  Notwithstanding a few cautionary voices warning of the potential erosion of academic rigor and the displacement of peer review, I can’t think of anyone who has actually rejected the notion of academic blogging—though to be fair, how would I know?  It would be an unlikely message to show up on a blog, but perhaps that article is gathering dust on some peer reviewers coffee table as we speak!  I have no doubt it will be all the rage when it comes out…in two years.

The blog landscape is rich in diversity.  Solo research blogs abound and there are a lot of really good ones—many of them existing well beneath the radar.  There are independent team blogs featuring teams of all shapes and sizes. Some, like GradHacker, grew out of institutional roots, which can give a project a very different trajectory.  Some are hosted by prominent blog networks, like those of the Chronicle or Inside Higher Ed.  And I’m finding more and more great blogs hosted by university-based blog networks.  I can only imagine the wide range of editorial arrangements that accompany high-profile branding of scholarly blogs.  While traditionalists are quick to point out that blog content is not peer reviewed, much of what appears on team blogs emerges from a collaborative process of writing, sharing, critiquing, re-writing, editing, proofing, and repeating—and that’s all before it gets posted to the blog itself where it is generally open to public comment.  There are many team blogs out there for whom content is an intensely negotiated product.

While the blogosphere is growing and appears to be thriving, it has provided a platform for increasingly biting critique of journal publishing.  The most measured critiques regard it as a squeaky wheel.  But more outspoken detractors excoriate the corruption of the financial structure, the restrictions of access to intellectual material and the exploitation of knowledge producers and consumers alike.  Recently I was reading over the following critique and set of recommendations from Don Taylor at the London School of Economics Impact of Social Sciences blog, (a blog which features a number of interesting pieces on the interplay between new technologies and existing scholarly forms for discussion and dissemination):

  • The identity of reviewer and reviewee should be known to one another
  • The title of manuscripts under review should be public, along with the authors of the manuscript and the identity of the reviewers
  • How long the reviewers have been reviewing the manuscript should be public
  • How long authors have had a request for revision should be public
  • Upon publication, the correspondence between reviewers/editors/authors should be public (this is important because often people say “why didn’t you do this subanalysis”; often it was done, but cut from a published paper due to length restrictions)
  • The use of online early publication is a good thing; I wonder if it will eventually become the only modality? (I only take one journal in hard copy now, Health Affairs, and otherwise utilize Duke University’s global subscription service)
  • Gated papers hinder academic investigation and discourse, but I am unsure of how to fund journals without subscriptions

This piece goes back several months, but the LSE blog does a nice job of stringing together posts into a discussion.  It is itself a nice example of what a blog can do that a journal cannot—maintain a conversation with enough momentum to move in a timely fashion towards a fruitful place.  I’m not sure I agree with each of his bullet points here.  Though I’m not sure I disagree either.  Historians, I’m quite sure would bristle at the notion of discarding the blind reviewer—but for those of us working in fairly specialized areas, going to conferences, and actively networking, how blind are they really?  Isn’t it a little weird to champion knowing your literature and the players in your field only to feign ignorance at the point that should be the pinnacle of the scholarly workflow? Doesn’t that essentially make the arbiter of new and potentially ground-breaking scholarship the person or people who are either ignorant or pretending to be?

But my objective here is not a further indictment of journal publishing or conventional peer review.  What is intriguing to me is that blogging, both through its critique of journal publishing, and its ability to model the things journals do well and basically do them even better is changing the peer review process.  While the LSE blog doesn’t advocate replacing the peer reviewed journal, the type of discussion this and other team blogs, both independent and institutionally affiliated, promote is elevating a new kind of peer review—one that, while less formal, is also less rigid, more nimble, far more transparent, more conducive to discourse and exchange, and far more accessible to interested audiences.

Peer review is the gold standard of scholarly credibility.  As it should be.  The shifting landscape of the academic blogosphere shows that commitment to peer review has changed the way many of us approach the process and product of blogging.  Blogging, likewise, is changing the way we think about, value, and define peer review.  I think back on one of the central questions arising out of the digital humanities that has presented itself to public historians as well—the nagging question of “what counts?” as scholarship and how we negotiate that ever shaky ground between convention and innovation.  I visited this question in Am I a Digital Historian?  It is important fodder for early career scholars looking to make the best use of their time, and arrange their accomplishments into marketable form.  It’s also a nice object lesson in the importance of continually questioning our conventions and in the fluidity of intellectual activity and enterprise.

6 thoughts on “Thinking Through Peer Review

  1. Pingback: Thinking Through Peer Review | Digital Humanities Tool Box |

  2. Rob–I don’t want to be picky, but a Ph.D. candidate, as distinguished from a Ph.D. student is that the candidate is one who has completed all the coursework, but is ABD (all but dissertation). I suspect, if this is your fifth year in this category, that you meant Ph.D. student–or maybe your first year as a Ph.D. candidate.
    And you’re right. Things have changed drastically in academia (or is it academe?)
    And in my prejudiced view, there’s no such thing as too much garlic. Just don’t breathe in my presence.

  3. Rob-Thank you for writing such a timely and insightful post. Public History, and the work of Public Historians is often the most professionally rigorous historical work that the average person will ever be exposed to. I’m currently maintaining a museum theater blog for Plimoth Plantation, Inc. ( We strive to give people an insight into the process that goes into our productions. We hope that by maintaining blogs that feature our research and the conversations that we have within our institution, that the public will be better able to understand and weigh the value of both museum and academic interpretation of the past. One may hope that increased participation of Public Historians in the blogosphere will make the professional work that they do more transparent and more widely known.

  4. Thanks for a great post on an important topic for scholarly communications. I’ve discussed many of these issues in my blog,

    Would you mind giving me attribution for the image that you used (Godzilla – original: It appears that Cameron Neylon, from whose blog you got the photo, forgot to give attribution for this creative commons-licensed image). Thanks.

    • Yes. No problem, attribution updated. Thanks for your comment. I took a quick look at your blog and read your two most recent posts. Real good stuff–I added it to our Digital Humanities board on Pinterest, and I’ll be back to read a little deeper.

  5. Rob, I’ve been thinking about your post. Two thoughts: (1) I, too, have had the experience of reading a “blind” reviewer’s review all the while thinking that I knew exactly who the reviewer was.(2) I wonder if “the fluidity of intellectual activity and enterprise” that blogs encourage is really what (some? most? older?- I’m 31) historians resist. Maybe it’s the fact that “publishing” on one’s blog or a group blog is so “quick” and “easy”–at least on the surface of it–that it can lend itself to the impression of slack scholarship, hasty judgements/arguments.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s