Dr. Tomokazu Nakamura, who just got a university position as Assistant Professor this past spring, has been asked by the academic society to which he belongs to conduct peer reviews of two papers for the first time.
One of the papers he has been asked to peer review is quite similar to the topic on which Dr. Nakamura is himself working. Looking at the schedule, if this peer reviewed paper is accepted, it is likely to be published before Dr. Nakamura publishes the results of his own research. Dr. Nakamura was tempted to either reject the paper in front of him or delay reporting the outcome of his peer review, to give himself time to put together the results of his own research and publish them.
Unbeknownst to the editorial committee member who asked him to peer review it, Dr. Nakamura was able to deduce that the other paper had been submitted by an assistant professor from the laboratory of Professor Sumiko Yamazaki, with whom Dr. Nakamura interacts regularly in the course of his research activities. Dr. Nakamura is involved in a project being undertaken by Professor Yamazaki, which is funded by a JSPS KAKENHI Grant-in-Aid for Scientific Research. While he found quite a few problems in the submitted paper, Dr. Nakamura thought that if it passed peer review and was published, it might be beneficial to the continuation of the project.
The term “peer review” means a review of a researcher’s paper or other research output or an evaluation/critique of their research activities as part of a review for a research grant, which is conducted by a peer—that is to say, a fellow researcher.
Peer review is essential to ensuring the quality of research. The process of peer review provides an opportunity to improve the content of the research by pointing out problems and commenting on things that the person who submitted the paper or applied for the research grant has not noticed. The publication of research that has been qualitatively enhanced in this way leads to the development of the field as a whole. In this sense, requests from academic journals to conduct peer reviews or from research funding organizations to review grant applications are necessary and recognized as a contribution to the field.
The results of peer review affect whether or not a paper is accepted for publication or a grant approved. Accordingly, researchers must take on peer reviews with a commensurate sense of responsibility. To what does one need to pay attention when conducting peer reviews?
Since the days of Ancient Greece, people have drawn upon other people’s ideas in their writings. Scholarly exchange transcending time and place has existed since ancient times. With the invention of printing technology in the early modern period, the results of research began to be published, but letters were the main means of communication used by people when assessing each other’s work. In 1665, the Royal Society of London’s first secretary, Henry Oldenburg (1618-1677), founded the society’s journal, Philosophical Transactions, to serve as a forum for publishing research findings and began asking the experts around him for their views, in order to guarantee the quality of the papers therein. This was the beginning of the peer review system that has continued through to the present day. Peer review is not only applied to academic papers; it has also been introduced when reviewing applications for research grants. In most realms, researchers now need the ability and tenacity to overcome the hurdle of peer review in order to get published.
The editorial committees of academic journals and review committees of research funding organizations rely on peer review by multiple researchers who are highly familiar with the topic in the submitted paper or grant application. Procedures vary depending on the academic journal or research funding organization, but just like in most fields of the natural sciences, peer review in the humanities and social sciences is carried out based on either a double blind peer review, in which both the person submitting the paper/grant application and the reviewer are kept anonymous from each other, or a single blind peer review, in which only the reviewer is anonymous.
When peer reviewing a paper, the reviewer scrutinizes it thoroughly from the following perspectives.
The peer reviewer comprehensively evaluates the paper from the aforementioned perspectives and then notifies the editorial committee of their assessment, stating whether the paper should be accepted, accepted with minor revisions, accepted with major revisions, or rejected. Depending on the assessment, the editorial committee might, in rejecting the paper, recommend that the author(s) rewrite it and submit it again.
When peer reviewing research grant application forms, reviewers evaluate not only the content of the research, but also the implementation structure and whether the budget amount and execution plan are consistent with the research plan. The overall review structure differs from one research funding organization to another. For example, the Japan Society for the Promotion of Science (JSPS) establishes a committee composed of experts in the topic covered in the submitted application form, which then reviews the application. Such processes differ between ministries and agencies, and even within a single ministry or agency, according to the nature of the project for which applications have been solicited. In addition to reviewing whether or not to accept a project, funding organizations also conduct re-evaluations over the course of the projects they have accepted, to consider whether or not to continue them. The frequency of these reviews varies from one organization to another.
Generally conducted as part of the review of submissions to academic journals and applications for research grants, peer review succeeds when researchers approach it in the spirit of contributing to their field of research as professionals in that field. The reviewer has responsibilities to the person submitting the paper or grant application, the editorial committee or funding organization committee, the readers, and society as a whole, based on their sense of professionalism as a researcher.
In its White Paper on Publication Ethics, the Council of Science Editors—an international body of academic editors—lists the following six items as ethical responsibilities of reviewers in all academic fields.
Submitted papers and grant applications contain data obtained using new theories and techniques, and other as-yet-unpublished academic knowledge. When editorial committees/grant review committees and reviewers are exchanging papers or application forms, they must be rigorous in their management of electronic files and printed matter. For example, they must take steps to keep the content of the material they are viewing out of sight of their colleagues and the students they supervise. In addition, leaking information about the progress or results of peer review to a third party—whether in writing, orally, or via social media—is strictly prohibited.
The peer review process is carried out not to reject papers/applications, but to improve their content. If there are any deficiencies in the material reviewed, you should explain the reasons why you think it is deficient and provide specific suggestions that will assist in revising it. Your comments from peer review will be sent to the person who submitted the material so that they can make revisions in line with your comments for the purpose of resubmission or consider submitting the paper to another academic journal.
The following example is quoted from the Advice for Reviewers issued by the Japanese Association of Sociolinguistic Sciences.
When it comes to research grant applications, the applicant will mostly receive only a simple notice of acceptance or rejection, but in the case of the Japan Society for the Promotion of Science (JSPS) KAKENHI grants, unsuccessful applicants are provided with an evaluation that includes its approximate ranking in the field of application, to which they can refer when applying the following year.
Reviewers are appointed by the editorial committees of academic journals and review committees of research funding organizations. If you receive an inquiry as a candidate for reviewer, you should consider your own expertise in the research field that is the subject of the paper or application before agreeing to take on the peer review, as you might have the option of turning the request down, depending on the circumstances.
In humanities and social sciences research, there are fields and subjects with only a limited number of researchers. Consequently, it will likely be difficult to ensure that you are well-versed in every aspect of the content of the paper or application under review. You should conduct the peer review from your own perspective and if you find that there are parts regarding which you cannot provide appropriate comments, it is vital that you properly inform the journal’s editorial committee or funding organization’s review committee of this fact. This will ensure that the committee can properly judge the content of the peer review and will also assist in selecting a reviewer on subsequent occasions.
A fundamental principle is that peer review evaluations and comments must be based on objective data and facts, free from bias in the form of the reviewer’s own interpretations or assertions. Evaluations leading arbitrarily to particular conclusions are inappropriate. You should take care to provide useful comments that will assist the author/applicant undergoing evaluation in making improvements to the content later on.
The concise definition of conflict of interest is a situation giving rise to the possibility that the reviewer’s interests could distort their judgment or undermine the objectivity required in their primary duty.
Situations that appear to be conflicts of interest are those in which a third party would have concerns about the possibility that the integrity and appropriate judgment required in an individual’s duties might be distorted.
Please bear the following points in mind with regard to conflicts of interest.
Having a conflict of interest is not in itself considered research misconduct, but a situation in which a conflict of interest exists has the potential to become a breeding ground for research misconduct. A typical example of a conflict of interest is a situation in which, when publishing the results of joint research with a company or research funded by a donation from a company, the researcher publishes only results favorable to that company. This leads to the fabrication or falsification of research results.
Academic conflicts of interest can also arise when reviewing papers and the like. For example, a reviewer who has agreed to conduct a peer review might provide positive comments if the paper supports the reviewer’s own theory. Conversely, if the papers submitted for peer review has been written by a rival researcher, the reviewer might make excessively critical comments, delay the decision to accept the paper under review, or even decide to reject the paper, so that the reviewer can publish their own paper first or make it easier to have their own grant application accepted.
Even when the double blind peer review process is used, a researcher asked to review a paper can sometimes deduce the identity of its author from the paper’s theme or content, as the number of researchers is very limited in some fields in the humanities and social sciences. For example, if you deduce that the author is a researcher with whom you have a competitive relationship (or is a student supervised by such a researcher) and you believe that it is difficult to conduct an objective peer review, the appropriate course of action is to withdraw from the peer review or consult the editorial committee. This also applies in the case of grant application reviews. There is no need to regard conflict of interest as a problem where your sole relationship to the author or grant applicant is that you belong to the same institution.
Conflicts of interest also include conflicts based on beliefs. This refers to situations in which an individual’s principles or faith affects the objectivity of their research. For example, there could be cases in which, due to their stance on animal rights and the prevention of cruelty, an individual cannot give a neutral view on research involving experiments on animals, no matter what the nature of that research is.
Due to the nature of their role, a reviewer is required to comply with the wishes of the editorial committee as far as possible and this includes meeting the deadline for the peer review. The Advice for Reviewers issued by the Japanese Association of Sociolinguistic Sciences states the following.
If you know from the outset that you will be unable to meet the specified deadline, you should usually refuse the peer review request. When refusing the request, you must do so as soon as possible, to give the editorial committee time to choose another reviewer.
Few of Japan’s academic societies disclose their peer review regulations to the public. This is why the content above focuses primarily on the regulations of academic societies in the U.S. Academic societies will likely be required to do something about this in due course, as the environment for education in the responsible conduct of research is put in place.
Not all reviewers necessarily have the consideration for ethics that should underpin peer review. There may be cases where the results of the peer review make it hard to conclude that an appropriate evaluation was carried out.
In this situation, the author should contact the editorial committee to dispute or appeal against the judgment. For example, the Regulations for the Editorial Committee of the Journal of the Japan Association for Social Policy Studies state the following.
The same applies if evidence of a failure to abide by the duty of confidentiality during the peer review process comes to light. Leaking information—whether in writing, orally, or via social media—to a third party about the content of a paper you have peer reviewed or talking about it to your colleagues or students under your supervision is a breach of confidentiality. Having a postdoc or student under your supervision carry out the peer review on your behalf is also problematic. If the review was not carried out in an appropriate form and the reasons for lodging the appeal are appropriate, the editorial committee may ask for the paper to be re-reviewed or add another reviewer.
In addition, if an author feels that the content of their submitted paper has been plagiarized by the person whom they surmise reviewed their paper, they should consult the editorial committee, providing evidence of this. Even where there is a suspicion of plagiarism, the author could find themselves being sued for defamation by the person whom they surmise reviewed their paper unless the author follows the appropriate procedure in querying it or making an allegation.
At the same time, aside from the problem of this kind of lack of ethical consideration, there can be cases in which a reviewer with the abilities required to review the content is not available, however fairly and impartially they might approach the task, or cases in which such a review cannot feasibly be carried out, due to the novel methods or subjects employed in the research or its multidisciplinary/interdisciplinary nature. It would be fair to say that this is a limitation of the peer review system.
In the course of fulfilling your duties as a reviewer, you might become aware of research misconduct such as plagiarism based on your own research experience to date; for example, you might notice that you previously read a paper similar to the one that has been submitted, or realize that the argument is similar to your own research findings. Such papers should be rejected on the grounds of misconduct, but the author of the paper in question might then submit the same paper to another academic journal. Revealing the content of a peer review conducted for another journal in the past is a breach of confidentiality in the strict sense, but academic journals sharing information about an author who has engaged in misconduct might be a necessary act of self-cleansing by academia, in order to prevent publication of a fraudulent paper. However, as identifying misconduct is not the primary purpose of the peer review process, it is not the reviewer’s responsibility if a fraudulent paper is published.
In the humanities and social sciences, problems can also arise with peer review of papers for institutional bulletins. Papers carried in the bulletin of the university to which you belong are generally reviewed by teaching staff from your own laboratory or department. This is not peer review in the strict sense and treating it as such will be regarded as padding out your record of achievement. (While “peer” means someone of the same status or ability, it does not refer to someone in the same organization and, in fact, indicates someone outside your organization.) When submitting a paper to a bulletin published by another university, you should check the situation, as most would not be considered peer reviewed.
Peer review is a system via which researchers conduct quality assurance of papers and research output. Everyone in academia must have an appropriate understanding of the ethics of peer reviewing papers for publication and grant applications and must follow this in practice. This includes not only the individuals who take on the task of reviewing papers and grant applications, but also journal editorial committees, funding organization review committees, authors, and readers.