Reviewer 2 Must be Stopped!
Written By T. (Ned) Choungprayoon, Ph.D. Candidate at Stockholm School of Economics and Lina Altenburg, Ph.D. Candidate at KU Leuven (Belgium)
The first-time reviewer development workshop by the International Journal of Research in Marketing (IJRM) and the Journal of Marketing (JM) happened recently at the European Marketing Academy (EMAC) Conference 2023, in Odense, Denmark.
In our academic journey, at least once, we all have encountered “reviewer 2” (it could also have been reviewer 1 or reviewer 3), either by receiving his/her harsh, unfair, and not-so-constructive comments, or unintentionally being one. Despite the joke of being “reviewer 2” commonly brought up all the time at the dinner table, or the memes spreading around on the internet, we rarely seem to discuss “how NOT to be a reviewer 2”.
During the EMAC Conference 2023, we had a chance to sit in the IJRM/JM Reviewer Workshop. This was the first official workshop focusing on how to become a good reviewer and what makes a good review and it was jointly hosted by two journals. The workshop intended to make participants acquainted with how to write fair, thorough, and constructive reviews.
Reviewer development workshop
The workshop was facilitated by the two editors-in-chief of IJRM (Martin Schreier) and JM (Hari Sridhar). The workshop brought together junior scholars who take their first steps in reviewing, with senior scholars (Area Editor) and members of the Editorial Review Boards of IJRM and JM. Participants were divided into small groups to discuss their review of an assigned paper, sharing their thoughts and commentaries on the paper and on the way the review should be written. It was a unique opportunity to simulate a reviewing process where reviewers could compare different perspectives and talk openly to one another about what they think and what they should write.
The workshop ended with guidelines on how to write a good review, presented by P.K. Kannan (amongst others, a former editor at IJRM) and Ajay Kohli (amongst others, a former editor of JM). Their key points were that it is important to divide the review comments into major and minor comments and provide constructive suggestions for overcoming the paper’s weaknesses. Thereby, it is important to keep the “path to publication” in mind (even if that path is with another journal), hence, what needs to be done for the paper to reach a publishable quality?
How to stop being a reviewer 2
During the facilitator-led, highly vivid interactive session, participants were asked to follow the broad guideline for writing a review as shown below;
What’s interesting about the paper?
Is anything “wrong”? How can we fix it?
Specific takeaway for AE Report
The participants first discussed these points in their small groups. Then, the small in-group talk extended to the overall workshop discussion about how they think about the assigned paper in the eye of a “reviewer”. In the end, the goal of the workshop was not to review the specifically assigned paper but to help participants to adopt the mindset of providing a good review: This mindset should not only be about how reviewers help the author to improve the work but also how the reviewers help the Area Editor to make the decision.
"Instead of starting with a “what’s wrong” mindset (classic reviewer trap), reviewers need to ask the following question first: What’s interesting about the paper? Is it interesting enough already? If not, how can we make it more interesting?”
"It is useful to lay out all the concerns you have with the manuscript, but equally important to offer suggestions and pathways for improvement”
-Shrihari (Hari) Sridhar-
Takeaways from the workshop
This was a great workshop, and you all should have been there. However, till next time, here is a list of key takeaways we gathered from the participants:
Writing a review can be reframed positively and constructively. If the paper has potential, the review should be about how to improve the paper so it can reach publishable quality.
The review process is not only about pointing out all the flaws but also about identifying the best part of the paper and its potential contribution.
It is important to think of yourself (the reviewer) as a part of a team composed of an author team, an editor, and an area editor.
Words from the workshop
“We never before got a chance to get insights from such experienced reviewers as during this workshop.”
Lukas Wolf (Right) (Participant), Research Associate at Friedrich-Alexander-Universität Erlangen-Nürnberg, Germany
“The workshop changed the mindset on how you approach the review because I learned that reviewing is not about listing as many bad things and weaknesses as possible but more about looking at the broader picture and asking if the paper is interesting enough apart from its weaknesses. Being more encouraging and helpful by helping the author improve the paper is also an important role for a reviewer.”
Roland Schroll (Left) (Participant), Assistant Professor of Marketing at the University of Innsbruck
“It is very helpful to organize something like this because when I was a junior, I had to learn by doing and by writing and seeing what other people write and receiving my own reviews to get the notion of the ideal reviews. I really appreciate the initiative. It is also very helpful to get young faculty to write reviews and give them templates of what to focus on and tell them that it is not about rejecting the paper but about seeing what is good in the paper and getting excited about the paper.
Working with the group mixed between junior faculty and senior faculty made me learn that everybody has their own perspective, everybody looks at the paper from their own angle and their points are all valid.”
Els Breugelmans (Facilitator), Professor at KU Leuven, Belgium
“The key takeaway was to change my approach to writing the reviews. I initially thought that I had to find all the flaws in it and point out what’s bad. Now, I think the better approach is to think about the important points of the paper and to find a part that can be totally incredible and provide a guideline to the flaws which the author can improve.”
Ph.D. Student and Research Assistant in Quantitative Marketing at Goethe University Frankfurt
“The junior faculty and students that we have in our group, came in very prepared and seemed to really appreciate the opportunity, which I found very refreshing. They did a very thoughtful job of answering the questions, so I think they genuinely seem to be very motivated to learn how to do a better job at reviewing. Simply getting people to think about reviewing, can help people grow their ability to provide thoughtful quality reviews.
Some of the junior faculty and students have ideas and thoughts that I haven’t considered. Moreover, it was a pleasure to get to know them.”
Kelly Hewett (Facilitator), Associate Professor, Reagan Professor of Business, and Haslam Family Faculty Research Fellow, Haslam College of Business, University of Tennessee, USA
This article was written by
T. (Ned) Choungprayoon
Ph.D. Candidate at the Stockholm School of Economics (Sweden)
Ph.D. candidate at the KU Leuven (Belgium)