Keeping your manuscript in the Queue: The journal peer review process from a reviewer’s point of view

Notes from Organizational Communication Mini-Conference 2009

Let me start with a story. When I was in graduate school, there was not as much of an expectation to publish while you were completing your degree. Nevertheless, mentors in my program encouraged me to craft a manuscript from my M.A. thesis and submit it for review. So I did. To Quarterly Journal of Speech. It was the first manuscript I had ever submitted. And I submitted it to one of the toughest journals to get into. Not surprisingly—at least in retrospect—it was rejected. Not only rejected, but rejected with what seemed then like an entire ream of reviewer comments. I was crushed and even ashamed. How could this happen to me? My mentors had been so confident that my work was worth publishing! How could I look them in the eye and tell them I wasn’t even invited to revise and resubmit? I took everything and put it in a drawer and didn’t take it out again. I don’t think even to this day I have read all the way through the reviews.

Well. Luckily, I recovered. I managed, eventually, to send out another ms for review. Looking back, I know that I should not have been so surprised. The simple truth is that most manuscripts submitted for review are never published. Top notch journals in our field reject roughly 9 out of every 10 manuscripts submitted. The experience is so different from graduate seminars, where we routinely give or receive top marks on research papers. Can you imagine a seminar where only 10% of the papers received a passing grade?

Sure, it’s the case that many manuscripts submitted to a journal are never published in that journal because they are rejected—sent back with a “thanks, but no thanks”. Even more important, many aren’t published because authors receive an invitation to revise and resubmit, and then they never do it. They take their work out of the queue. What I’ve learned from my successful colleagues is that they regard everything except an “acceptance” as a revise and resubmit. Ask anyone in our field who publishes regularly in peer reviewed journals: How many of your manuscript submissions have received editorial decisions of “accept without revision”? Without telling them I was going to do this, I asked this exact question of the scholars attending the seminar where I gave this presentation. Only 1 had received such a decision…and then he shared that he wondered whether that meant he should have submitted it to a more competitive journal! When successful scholars receive a “reject” from one journal, it also is taken as a revise and resubmit…to a different journal.

I have been reviewing for peer reviewed journals for two decades, and can count on one hand the number of manuscripts I recommended be “accepted without revision”. Kudos to those of you who have or who will achieve that feat. My main point in this essay is that you should not submit a manuscript with an expectation of having it accepted on the first review. A reasonable and achievable goal for us mortals is this: to keep our work in the queue. Anything in the queue still has the potential of being published.

Consider Multiple Points of View

There are at least 3 different parties in any peer review process, and each has a different point of view: (1) the authors, (2) the editors, (3) the reviewers.

As the author, you know the work most intimately. You’ve done the work that needed to be done to produce this manuscript. You know what choices were made, you made decisions about what arguments to make, what literatures to connect to, and what contributions to highlight. Sometimes, you may have agonized for an entire afternoon on a sentence, a phrase, a word.

An editor’s focus is the journal. Your work—for better or worse—is the editor’s currency. A journal’s reputation stands or falls on the quality of the content of the articles within it. Editors must meet production goals, publication deadlines, and other publisher expectations such as library adoptions, or maintaining and improving ISI rankings. Good editors also recognize that a strong journal influences the field in important and lasting ways. They see the big picture, and each manuscript they receive will contribute toward that big picture.

Both authors and editors rely on reviewers as the mediator between these two perspectives, between the single manuscript and the big picture. Reviewers are expected to attend carefully to particular manuscripts, while at the same time possessing sufficient knowledge in the field of research so that they can provide good advice as to whether or not the manuscript would be a contribution to that field.

I don’t want to paint an overly rosy picture of reviewers. Sometimes reviewers have different (and not wholly constructive) points of view, like whether their own work is being cited, or whether their preferred position is represented, or whether the study is done in the way they would have done the study. Sometimes reviewers abuse the privilege of anonymity, to provide scathing and possibly hurtful critique. Sometimes they are careless in their reading. Sometimes, reviewers are simply inexperienced and so a little overeager (suggesting changes to nearly every paragraph).

Sometimes, any of these things are true. But the system has a way to check these possibilities, and correct them overall. Because everyone recognizes the highly subjective nature of this enterprise, there are typically 3 reviewers in a blind review process. When editors respond to authors, they typically provide a decision letter, along with the 3 reviews. Usually, reviewers get a copy of these as well. You learn a lot, as a reviewer, to see where you agree and disagree with other reviewers. Despite the stories of the infamous “third reviewer” (and as an author I’ve faced my own third reviewers), though reviewers may highlight different things, most of the time I’ve found that the final recommendations are about the same. Over time, inexperienced reviewers learn from the examples of other reviewers and get better. My point here is that good reviewers also have a stake in being helpful, reasonable, and constructive. In part, it’s because we know that others see our reviews. In part, it’s because we want to be engaged in advancing the field. It’s also because good editors stop soliciting reviews from people who don’t help them make good editorial decisions. Reviewers, too, can be rejected.
The Reviewer’s Responsibilities

As a reviewer, I figure I have at least 3 different responsibilities. One responsibility is to the author. You have the right to expect that I will give your manuscript a fair and careful reading, that I will try to respect the scholarly decisions you have made even when they might not be the ones I would have made, that I will honor your larger research agenda and not attempt to fit the manuscript into a different agenda I might prefer, and that I will critique and evaluate the manuscript according to high but not unreasonable standards.

A second responsibility is to the field. Reviewers, in an important sense, are gatekeepers to the field or the discipline. Any number of journals have been founded precisely because reviewers of existing journals would not provide favorable reviews to manuscripts that did not meet given standards or expectations of what would be appropriate for publication.

A third responsibility is to the editor. And here is the point of view I have arrived at over the years: I believe it is irresponsible for a reviewer to work with every author to make their manuscript publishable. Let me return to one of the main points I started with: the overwhelming majority of manuscripts submitted to a journal are never published. They cannot be. Even if an editor decides to reject some fraction of submissions without review, most manuscripts go into the “queue” and go out for review. Let’s think about the scale of this enterprise. Let’s say a journal publishes 1 out of every 10 submissions. In one issue of the journal, let’s say there are 5 articles. That is, roughly, 50 submissions for 5 slots. To publish just that one issue, the editor must reject 45 manuscripts. Editors must be predisposed to eliminate manuscripts. Editors that don’t successfully eliminate manuscripts create backlogs in which it may take years for a manuscript to come to print, jeopardizing the reputation of the journal, the timely dissemination of important knowledge, and often the careers of scholars who need publications for advancement or promotion.

The critical role the reviewer plays is to help the editor know which submissions to eliminate, and which to keep in the queue.

Making a Recommendation: One Reviewer’s Account of the Decision-Making Process

The point of view I’m suggesting refigures the meaning of “revise and resubmit,” commonly called R&R. In a real sense, the alternative to an R&R is not an acceptance…it is a rejection. This is what successful scholars know. You work for the R&R. An R&R keeps your manuscript in play.

I don’t expect to review manuscripts that are ready to publish without revision. To do so is a real treat. Likewise, I am never disappointed to recommend a revision. A recommendation of R&R is a statement of support.

Here’s what the decision-making process looks like for me, in a practical sense. First off, I read from start to end. Truthfully, the first decision-point for whether I am likely to recommend rejection arises sometime in the first 2-3 pages, by which time the authors should have clearly articulated the justification and contribution of the manuscript. Other reviewers have different shortcuts for making this initial decision – some read literature reviews first, or methods, or perhaps analysis. The particular starting point is not important. Each of these are “necessary but not sufficient” elements of a publishable manuscript. Reviewers have their favorites for which of these is the first test. How well you do on this test sets the tone for how the reviewer will read the rest of the manuscript.

I don’t adopt a defensive stance when I review. What I mean is that I don’t expect authors to have to argue for their manuscript to be published. In fact, I start my reading in “accept” mode – I perfectly welcome the possibility that your manuscript might be one of those gems that stand apart. The reality is that most manuscripts eliminate themselves, by failing to meet some basic tests.

What are some of these tests? What are some things you can do to increase the chances your manuscript will stay in the queue?

1. Submit only if you are serious. Avoid the “what the heck, let’s send it in” submission. Or the “we couldn’t get into the journal we really want, so we decided to send it to you.” That’s disrespectful of the journal and the reviewers’ time.

2. Does the manuscript fit the journal? Familiarize yourself with issues of the journal. Does the topic fit the journal? Do the research methods fit? Does the journal favor certain literatures over others?

3. Do the authors demonstrate sufficient authority or expertise in the area? Authors who don’t have sufficient expertise often miss key arguments, research, or theory. When you refer to theories and research, be sure your use is accurate.

4. Take care in preparing the manuscript. Check your spelling and grammar. Write well.

5. Does the argument make a contribution? Do not submit the manuscript if you cannot phrase your argument and its contribution in 1-2 paragraphs. Be aware of the different between simply having an idea, and making a contribution.

6. Is there sufficient evidence and data to support the claims being made? I need to be able to trust your claims. I have reviewed many manuscripts that make elegant claims, only to have them insufficiently supported.

7. Do the results make a contribution? Even a very good paper might not be published, because in the end it doesn’t make a sufficient contribution to be worth taking up the limited resource of a slot in the journal.

Any manuscript that reasonably passes these tests makes my job as a reviewer harder. And that’s good for you! A manuscript that meets these tests deserves serious consideration for publication. As a reviewer, I must now engage the paper in terms of the details of its argument and its potential. I have the burden of proof to argue for why changes to the manuscript are necessary before it is ready for publication.

You’ve kept it in the queue. Congratulations.

rev. 1-10-2010

Print Friendly