{"id":124,"date":"2009-06-15T06:00:22","date_gmt":"2009-06-15T10:00:22","guid":{"rendered":"http:\/\/blogs.law.harvard.edu\/pamphlet\/?p=124"},"modified":"2009-06-05T17:10:54","modified_gmt":"2009-06-05T21:10:54","slug":"an-economic-solution-to-reviewing-load","status":"publish","type":"post","link":"https:\/\/archive.blogs.harvard.edu\/pamphlet\/2009\/06\/15\/an-economic-solution-to-reviewing-load\/","title":{"rendered":"An economic solution to reviewing load"},"content":{"rendered":"

Hal Daume at the NLP blog<\/a> bemoans the fact that “there is too much to review and too much garbage among it” and wonders “whether it’s possible to cut down on the sheer volume of reviewing”.<\/p>\n

<\/p>\n

He lists several possible approaches, most of which apply only to conference papers (which is appropriate since his field [and mine] is computer science, in which peer-reviewed conference proceedings constitute the bulk and much of the best of the archival literature). But to avoid parochialism, I’ll restrict attention to ideas that might apply as well to journal reviewing. His recommendations center on the idea of tiering the review process, allowing the editor to preemptively reject an article, or enlisting a full contingent of reviewers only if the article passes muster by a smaller cohort. Many journals already use these approaches, but they have the negative side-effect of reducing the scope of review for rejected papers, which may reduce the overall quality and fairness of the review process. (He doesn’t mention a different approach to tiering, which we might call “trickle-down reviewing”, in which a single review process is used for a set of journals with different publishing thresholds, so that papers rejected from a high prestige journal don’t generate requests for new reviews from a lower prestige journal that the authors resubmit to.)<\/p>\n

There is an economic solution to the problem that bears consideration: Charge for submission. This would induce self-selection; authors would be loathe to submit unless they thought the paper had a fair chance of acceptance. Consider a conference or journal with a 25% acceptance rate that charged, say, $50 per submission. (The right amount may be different; I use this figure just as an example.) Authors who tended to write and submit average quality papers would be confronted with a cost of some $200 (in expectation<\/a>) per published paper. If they wanted to reduce that cost, the expedient method would be to submit fewer papers and papers with higher average quality. The most plausible approach is to refrain from submitting the lowest quality papers, but other methods of improving quality would work as well. This has several positive effects: reduced reviewing load, higher average submission quality, less “salami-slicing<\/a>“, and revenue generation to boot.<\/p>\n

To avoid disenfranchisement of scholars with more limited means, fee waivers should be supplied in exigent circumstances, as they are for page, figure, and other publication charges by many journals. The application process for the fee waiver would be separated<\/a> from the editorial process to prevent mercenary considerations from affecting editorial decisions.<\/p>\n

Submission fees have a further benefit over publication fees in eliminating any economic incentive for lowering quality standards as a means for increasing revenue, as discussed in detail by McCabe and Snyder<\/a>.<\/p>\n

Submission fees could go a long way toward solving problems not only with reviewing but also journal financing and overpublication, a win-win-win situation, all without limiting entr\u00e9e to publication.<\/p>\n","protected":false},"excerpt":{"rendered":"

Hal Daume at the NLP blog bemoans the fact that “there is too much to review and too much garbage among it” and wonders “whether it’s possible to cut down on the sheer volume of reviewing”.<\/p>\n","protected":false},"author":2110,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[68],"tags":[],"class_list":["post-124","post","type-post","status-publish","format-standard","hentry","category-scholarly-communication"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p5pLfN-20","jetpack-related-posts":[{"id":431,"url":"https:\/\/archive.blogs.harvard.edu\/pamphlet\/2010\/05\/01\/worlds-most-excruciatingly-ironic-conference\/","url_meta":{"origin":124,"position":0},"title":"World’s most excruciatingly ironic conference?","author":"Stuart Shieber","date":"Saturday, May 1, 2010","format":false,"excerpt":"Could this be the world's most excruciatingly ironic conference? \u00a0The Second\u00a0International Symposium on\u00a0Peer Reviewing (ISPR 2010) is soliciting papers. Their call for papers emphasizes the sorry state of peer-review, calling for\u00a0\"more research and reflections [that] are urgently needed on research quality assurance and, specifically, on Peer Review.\" What could be\u2026","rel":"","context":"In "scholarly communication"","block_context":{"text":"scholarly communication","link":"https:\/\/archive.blogs.harvard.edu\/pamphlet\/category\/scholarly-communication\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":2445,"url":"https:\/\/archive.blogs.harvard.edu\/pamphlet\/2022\/07\/25\/moderating-principles\/","url_meta":{"origin":124,"position":1},"title":"Moderating principles","author":"Stuart Shieber","date":"Monday, July 25, 2022","format":false,"excerpt":"Some time around April 1994, I founded the Computation and Language E-Print Archive, the first preprint repository for a subfield of computer science. It was hosted on Paul Ginsparg\u2019s arXiv platform, which at the time had been hosting only physics papers, built out from the original arXiv repository for high-energy\u2026","rel":"","context":"In "computational linguistics"","block_context":{"text":"computational linguistics","link":"https:\/\/archive.blogs.harvard.edu\/pamphlet\/category\/linguistics\/computational-linguistics\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":1203,"url":"https:\/\/archive.blogs.harvard.edu\/pamphlet\/2012\/03\/06\/an-efficient-journal\/","url_meta":{"origin":124,"position":2},"title":"An efficient journal","author":"Stuart Shieber","date":"Tuesday, March 6, 2012","format":false,"excerpt":"\u201cYou seem to believe in fairies.\u201d Photo of the Cottingley Fairies, 1917, by Elsie Wright via Wikipedia. Aficionados of open access should know about the Journal of Machine Learning Research (JMLR), an open-access journal in my own research field of artificial intelligence, a subfield of computer science concerned with the\u2026","rel":"","context":"In "computer science"","block_context":{"text":"computer science","link":"https:\/\/archive.blogs.harvard.edu\/pamphlet\/category\/computer-science\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":1866,"url":"https:\/\/archive.blogs.harvard.edu\/pamphlet\/2013\/11\/21\/thoughts-on-founding-open-access-journals\/","url_meta":{"origin":124,"position":3},"title":"Thoughts on founding open-access journals","author":"Stuart Shieber","date":"Thursday, November 21, 2013","format":false,"excerpt":"\u2026 altogether too much concern with the contents of the journal\u2019s spine text\u2026 \u201creference\u201d image by flickr user Sara S. used by permission. Precipitated by a recent request to review some proposals for new open-access journals, I spent some time gathering my own admittedly idiosyncratic thoughts on some of the\u2026","rel":"","context":"In "open access"","block_context":{"text":"open access","link":"https:\/\/archive.blogs.harvard.edu\/pamphlet\/category\/scholarly-communication\/open-access\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":56,"url":"https:\/\/archive.blogs.harvard.edu\/pamphlet\/2009\/05\/27\/some-background-on-open-access\/","url_meta":{"origin":124,"position":4},"title":"Some background on open access","author":"Stuart Shieber","date":"Wednesday, May 27, 2009","format":false,"excerpt":"I assume that readers of the open access discussions on this blog are familiar with the state of play in the area, but just in case, here's some background. Peter Suber defines open access in his A Very Brief Introduction to Open Access as follows: \"Open-access (OA) literature is digital,\u2026","rel":"","context":"In "meta"","block_context":{"text":"meta","link":"https:\/\/archive.blogs.harvard.edu\/pamphlet\/category\/meta\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":1000,"url":"https:\/\/archive.blogs.harvard.edu\/pamphlet\/2011\/11\/16\/how-should-funding-agencies-pay-open-access-fees\/","url_meta":{"origin":124,"position":5},"title":"How should funding agencies pay open-access fees?","author":"Stuart Shieber","date":"Wednesday, November 16, 2011","format":false,"excerpt":"\u201c...a drop in the bucket.\u201dDrop I (2007) by Delox - Martin De\u00e1k via flickr. Used by permission (CC by-nc-nd) At the recent Berlin 9 conference, there was much talk about the role of funding agencies in open-access publication, both through funding-agency-operated journals like the new eLife journal\u00a0and through direct reimbursement\u2026","rel":"","context":"In "open access"","block_context":{"text":"open access","link":"https:\/\/archive.blogs.harvard.edu\/pamphlet\/category\/scholarly-communication\/open-access\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]}],"_links":{"self":[{"href":"https:\/\/archive.blogs.harvard.edu\/pamphlet\/wp-json\/wp\/v2\/posts\/124","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/archive.blogs.harvard.edu\/pamphlet\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/archive.blogs.harvard.edu\/pamphlet\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/archive.blogs.harvard.edu\/pamphlet\/wp-json\/wp\/v2\/users\/2110"}],"replies":[{"embeddable":true,"href":"https:\/\/archive.blogs.harvard.edu\/pamphlet\/wp-json\/wp\/v2\/comments?post=124"}],"version-history":[{"count":14,"href":"https:\/\/archive.blogs.harvard.edu\/pamphlet\/wp-json\/wp\/v2\/posts\/124\/revisions"}],"predecessor-version":[{"id":158,"href":"https:\/\/archive.blogs.harvard.edu\/pamphlet\/wp-json\/wp\/v2\/posts\/124\/revisions\/158"}],"wp:attachment":[{"href":"https:\/\/archive.blogs.harvard.edu\/pamphlet\/wp-json\/wp\/v2\/media?parent=124"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/archive.blogs.harvard.edu\/pamphlet\/wp-json\/wp\/v2\/categories?post=124"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/archive.blogs.harvard.edu\/pamphlet\/wp-json\/wp\/v2\/tags?post=124"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}