Charles Ayoubi (ESSEC Business School)

Learning by Evaluating: Evidence from Academic Peer Review

Abstract: The evaluation of innovative projects is an essential task for scientific and business organizations alike. While prior work has focused on the quality of evaluations and how they affect innovators, little is known about how the evaluation process affects evaluators themselves. We propose that evaluations provide evaluators with valuable learning opportunities by exposing them to relevant, cutting-edge knowledge. We test the argument in the setting of academic peer review, using administrative data from the Institute of Physics Publishing comprising 104,306 reviewer-manuscript pairs across 55 physical sciences journals. We find that evaluating a manuscript more than doubles the likelihood of a reviewer using that knowledge (by citation) within three years compared to qualified evaluators who were invited but unavailable. Evaluators’ geographic and intellectual distance to the work mediates the learning effects of evaluation. While unavailable evaluators from the same country as a paper’s corresponding author have significantly higher baseline citation rates, evaluators from different countries who reviewed the paper achieved similarly high citation levels, indicating that structured evaluation of external ideas can compensate for geographic boundaries in knowledge transfer. Intellectual proximity between evaluators’ recent work and the work being evaluated is associated with stronger learning effects. These findings show that expert evaluations benefit not only organizations (by improving resource allocation) but also the evaluators themselves, which helps explain why they participate in the laborious and seemingly under-incentivized work.