CTPUG/wafer

Feature Request: Assign reviewers specific submissions

Closed this issue · 2 comments

gforsyth commented

I had a conversation with @hodgestar at SciPy 2019 about potentially using wafer to run the SciPy conference website and I think we'd like to go ahead and do that.

I think we can add in the features we need and push them back upstream, but I wanted to raise the features we're missing in issues in case

a) someone has a real hankering to implement them
b) people have strong objections / cautionary tales around said features
c) anyone has guidance around the best way to go about doing any of this

We run a multi-track conference and I see that support for that has already been added (which is great!) -- our next wrinkle is that we usually assign reviewers to talks. I believe this has a few components to consider:

  1. Roles such that a reviewer only sees the talks assigned to them
  2. Role for a track-chair to see all talks in a given track
  3. Some metadata attached to a reviewers profile to indicate which tracks they feel comfortable reviewing talks in

I'm happy to break this up into smaller issues, but wanted to start the conversation. Thanks for this project, it looks great!

drnlm commented

1 + 2 would require pulling in one of the per-instance permission plugins for Django, since it can't be handled by the default permissions system. That feels like it's a bit too heavy a construct for the core wafer system, and should be handled as an app that integrates with wafer.

  1. Feels like something that can live in the default review system, but isn't required for most conference.

I think the correct approach here is to see how we can make it easy to replace the basic review system, so conferences with more specialised requirements can do so, rather than trying to extend wafer's review system to cover all the possible cases.

hodgestar commented

@gforsyth Hello!

We'd definitely love an improved and more flexible review system and we're happy for you to work on it. As @drnlm mentioned, we need to strike a balance between conference specific features and overcomplicating core wafer.

At the moment we've inherited Review, ReviewAspect and Score models from DebConf, and I wouldn't mind seeing that cleaned up a bit. A Review object links a reviewer and a talk and just has a (versioned) markup field for reviewer comments. A ReviewAspect is something that can be scored (e.g. "Relevance to Python", "How Topical is the Talk", "Experience of Speaker" ) and a Score links a score to a ReviewAspect and a particular review. There is a review form that I think is not used within core Wafer at all.

With some clean-up and small extensions to the review table, I think you might be able to achieve what you want in the following way:

  1. Roles such that a reviewer only sees the talks assigned to them

I think is achievable without per-object roles by only allowing people with the talks.add_review permission to start new reviews, but once a review object has been created, to allow reviewers to see and edit their reviews and to see the associated talks.

There should probably be a non-admin page that allows people with the talks.add_review permissions to manage (create, delete, view) reviews and their status, and somewhere for reviewers to see a list of the talks they're reviewing.

Track chairs could then use this page to assign reviewers to talks by creating review objects. Reviewers could perhaps use the same page to see the reviews they've been given.

  1. Role for a track-chair to see all talks in a given track

I was wondering if this really requires a role? If you trust track chairs a bit, perhaps just a way for them to filter talks by track in some places would be enough?

  1. Some metadata attached to a reviewers profile to indicate which tracks they feel comfortable
    reviewing talks in

Given that we have part of a review system, perhaps it's not unreasonable to have a Reviewers model that provides a list of users who are reviews (and links them to tracks they're prepared to review). Track chairs could be allowed to create reviewers and set their preferences, and that would help filter the users when creating review objects.

@gforsyth Thoughts? If you like any of these ideas, perhaps it would be easiest to create a small PR for one and then we can discuss more concrete details? You're also welcome to come chat to us in IRC (Freenode #wafer) if you like.