Dan Goodman on Nostr: npub1g9lm9…875lx respect to you for actually building something to fix peer review ...
npub1g9lm9wzy9mllzux4ehxrj43ptng5w0nrcq3cea676fj3m7klt8tq0875lx (npub1g9l…75lx) respect to you for actually building something to fix peer review but I don't think this is the right solution.
Firstly, it creates bad incentives. PIs will get their trainees to write the reviews under the PI's name. People will minimise the time spent on the review. If the system allows for it they'll just submit "great job" and if not they'll feed the paper into ChatGPT which will generate a large amount of verbiage highlighting unimportant surface level issues. This will waste authors' time and give the misleading impression that the paper has received meaningful scrutiny.
Secondly, the whole thing is based on the idea that the main problem with peer review is that not enough people are agreeing to do it. I don't think that's true. The problems from my point of view are (1) poor quality of reviews because people are too pressed for time and because reviewers are badly matched to papers. This proposal will exacerbate this problem by encouraging low effort reviewing and raising the costs of declining to review. (2) pre publication review slows down science and adds little value because it's not possible to find all the issues with a paper before it's published and people start to try to build work on it. (3) current editorial and review processes put too much power in too few hands which leads to bias and distorted science. (4) the requirement for peer review before publication encourages authors to hide problems with their papers.
Firstly, it creates bad incentives. PIs will get their trainees to write the reviews under the PI's name. People will minimise the time spent on the review. If the system allows for it they'll just submit "great job" and if not they'll feed the paper into ChatGPT which will generate a large amount of verbiage highlighting unimportant surface level issues. This will waste authors' time and give the misleading impression that the paper has received meaningful scrutiny.
Secondly, the whole thing is based on the idea that the main problem with peer review is that not enough people are agreeing to do it. I don't think that's true. The problems from my point of view are (1) poor quality of reviews because people are too pressed for time and because reviewers are badly matched to papers. This proposal will exacerbate this problem by encouraging low effort reviewing and raising the costs of declining to review. (2) pre publication review slows down science and adds little value because it's not possible to find all the issues with a paper before it's published and people start to try to build work on it. (3) current editorial and review processes put too much power in too few hands which leads to bias and distorted science. (4) the requirement for peer review before publication encourages authors to hide problems with their papers.