Scales used to evaluate projects
To provide a standard measuring stick across projects, reviewers use a ten-grade scale in their overall evaluations of projects.
To standardize the evaluation of both its grant proposals and projects in progress, the Japanese Agency for Medical Research and Development (AMED) has adopted a ten-grade evaluation scale. In this scale, a score of ten is exceptional, while a score of one is extremely poor.
This scale has been adapted for the evaluation of grant proposals. For example, a score of ten implies that a grant proposal is world class and represents strategically vital research and development in healthcare and medicine in Japan. At the opposite end of the spectrum, a score of one indicates that a grant proposal offers no advantages but has a number of major disadvantages.
In this scale, the cut-off score is six, namely scores of six and above indicate that proposals that should be funded, whereas scores of five and below represent projects that should not receive funding.
By modifying the descriptions for each grade, the same evaluation scale is used to assess the progress of projects that are already underway. In this case, a score of ten indicates that world-class results are expected that should substantially contribute to the development of healthcare in Japan, whereas a score of one is used for a project that has clearly not progressed as planned and has completely failed to obtain the anticipated results.
While the diagram shown below is for research and development projects, it can be easily adapted for other purposes (for example, infrastructure development) by simply replacing the descriptions with the appropriate terminology.
It is important to note that the ten-grade evaluation scale applies only to the overall assessment of a grant proposal or research project. Individual items in a review, such as compatibility with the relevant AMED program, scientific or technological significance, and appropriateness of the project plan, are scored on a point scale of zero to four. In this scale, three points is outstanding, two points is satisfactory, one point is somewhat inadequate, and zero points is inadequate. Comments should also be added with advice regarding the strong points of a proposal or project, points that need to be clarified, and areas that could be improved. The reason for the assignment should be noted, especially when a score of zero or three has been assigned. The strengths and weaknesses of a project should also be given in the overall evaluation; these strengths and weaknesses should be reflected in the scores assigned to individual items in the review.
Reviewers of grant proposals and of projects in progress are asked to apply these ten-grade and four-point scales in their reviews. They should thus be familiar with the criteria for each score (see accompanying charts). These tools both simplify the review process and provide standard measuring sticks for evaluating the highly diverse projects that AMED evaluates and funds.