pyOpenSci/software-peer-review

Add process document for managing reviews

NickleDave opened this issue · 2 comments

As discussed with @lwasser in Slack

We should note the following:

  • label a pre-submission with submission-requested if that is the status
  • when an editor has been found (e.g., through slack by email), assign them the issue. then we know it has an editor!
  • add new pre-submissions to the project board (until such time as this is automated)
  • when reviews are waiting for a maintainer response, e.g. while addressing issues spotted during the initial editor checks, add the pending maintainer response label

Some additional notes:

  • Need note on when to remove New Submission! tag. I assume this is removed before 1/editor-checks but not mentioned in guide as far as I can tell
  • The docs are thorough! but they are a bit difficult to read since there are so many top-level ToC items and each step has a different format. Recommend making three top-level ToC items: Editor Description, Editor Checklist, and Alternative Paths. The titles are also pretty conversational which adds to noise when trying to skim ToC.

So maybe current structure:

  • Experience needed to become an editor
  • Two types of editors
    • Guest Editors
    • "Full" Editors
  • What does an editor do? (Responsibilities)
  • Editor support of other reviews
  • Editor-in-Chief rotation
  • How long does an editor serve on the editorial board?
  • Editor checklist: Get Started With Leading a Package Review
    • ✔️ 1. First, tag the submission issue on GitHub
    • ✔️ 2. Respond to the submitter in the GitHub issue
    • ✔️ 3. Identify scientific Python package reviewers
    • ✔️ 4. Onboard reviewers
  • Editor responsibilities during the review
    • ✔️ 5. What to do when reviews are in
  • Putting a review on hold & handling non-responsive authors
    • ✔️ 6. How to accept a package into the pyOpenSci ecosystem
  • Closing notes about the editorial process
  • ✔️ OPTIONAL: Instructions for Submitting to JOSS
  • Last Steps Before Closing the Review Issue

can become something like this (sorry github markdown is transforming my numbers to letters)

  • Editor Description
    • Types of Editors
    • Editor Responsibilities
      • Lead Reviews
      • Support Other Reviews
      • EiC Rotation
  • Editor Checklist
      1. Editor Checks
      1. Seek Reviewers
      1. Onboard Reviewers
      1. Coordinate Review
      1. Complete Review
  • Alternative Paths
    • Nonresponsive Authors
    • Nonresponsive Reviewers
    • Submitting to JOSS

"Onboarding editors & reviewers" and "Finding reviewers" is located in multiple places. Recommend consolidating to one place :)

This section is also duplicated as a section title and a page title, which makes me think that maybe "Editor" guide needs to be a section with multiple pages (as i assume all of them eventually will need to be:

  • Editor Guide
    • Position Description
    • Review Checklist
    • Finding Reviewers
    • Onboarding Reviewers
Screen Shot 2023-12-04 at 6 10 58 PM