Create a README explaining the different types of evaluation

Question

Create a README explaining the different types of evaluation

Opened this issue 8 years ago · 16 comments

@payne-chris-r created a list that captures our expectations and the developers' responsibilities for the different evaluations. This could be added as a link at the end of installfest for the developers to read as other developers finish installfast and before the entire group goes to the kitchen for drinks and celebrations.

There are potentially other ways to do include this material other than a README at the end of installfest which should be discussed.

Practices and Studies:
We're looking to make sure you have made an effort, and answered the questions to the best of your ability. If we determine you've at least given it an honest go, they will be marked as 1. If you don't open a PR, or it appears as though you haven't even tried to answer the questions, it will be marked as a 0.

Diagnostics:
Similar except they will be scored on a question-by-question basis and we will try to give you more accurate--code-based feedback. If you don't open a PR before the cut off, it will be marked as zero. If you've copied the solution branch you've completely missed the point, you will be spoken with.

Challenges:
Aren't scored. We will do our best to give feedback on any open PR on these, but our main priorities are for other material first. Projects, diagnostics, studies/practices, etc. then challenges.

Projects:
You'll probably get the most feedback on these, and they are the largest determining factor in your success in WDI. The idea is for you to Meet all requirements that are asked of you before "Project Presentation" day. If you do not, the bugs/issues that we require you to address will be outlined in an email. You will be given a few days to address them, and you will be asked to resubmit. If at that point you still have not addressed those issues, OR introduced any additional bugs in the process, you will receive a 0 (or Does not meet expectations for that project. If you succeed in meeting expectations that are asked of you, you will receive a 1. If you go above and beyond on any of your projects you may be given an Exceeds Expectations at the discretion of the instructional staff.

Answer 1 · 2017-03-26T15:28:14.000Z

@ga-wdi-boston/core

An addendum I chatted with @payne-chris-r about: Submission is the only success criterion for the above for the program, except projects and challenges. Challenges need not be submitted. Projects must be submitted and meeting expectations is necessary to successfully complete WDI.

Answer 2 · 2017-03-26T15:58:55.000Z

Practices and Studies: We're looking to make sure you have made an effort, and answered the questions to the best of your ability. If we determine you've at least given it an honest go, they will be marked as 1. If you don't open a PR, or it appears as though you haven't even tried to answer the questions, it will be marked as a 0. All practices and studies MUST be submitted.

Diagnostics: Similar to Practices and Studies, except they will be scored on a question-by-question basis and we will try to give you more accurate--code-based feedback. If you don't open a PR before the cut off, it will be marked as zero. If you've copied the solution branch, you've completely missed the point, you will be spoken with to make sure you're in WDI for the right reasons. All diagnostics MUST be submitted.

Projects: You'll probably get the most feedback on these, and they are the largest determining factor in your success in WDI. The idea is for you to Meet all requirements that are asked of you before "Project Presentation" day. IF you do not, the bugs/issues that we require you to address will be outlined in an email. You will be given a few days to address them, and you will be asked to resubmit. If at that point you still have not addressed those issues, OR introduced any additional bugs in the process, you will receive a 0 (or Does not meet expectations for that project). If you succeed in meeting expectations that are asked of you, you will receive a 1. If you go above and beyond on any of your projects you may be given an Exceeds Expectations at the discretion of the instructional staff. All challenges MUST be submitted, and presented on Presentation Day.

Challenges: Aren't scored. We will do our best to give feedback on any open PR on these, but our main priorities are for other material first. In general, that priority list goes: Projects, diagnostics, studies/practices, then challenges. Challenges DO NOT need to be submitted.

Answer 3 · 2017-03-26T18:20:44.000Z

It should be noted that we may ask you to meet additional requirements and have higher standards for resubmission of projects.

Answer 4 · 2017-03-26T18:23:45.000Z

Diagnostics: (In addition to what is above) Diagnostics are a tool for self-evaluation, both yours and ours. They let you know what areas you need to concentrate on, and let us know what we might need to review (if many people got the the same questions wrong). You are not expected to answer diagnostics from memory, and can use any material you wish (except the solution branch) to help you answer them.

Answer 5 · 2017-03-26T18:30:39.000Z

Should this be a README, or part of the Wiki?

Answer 6 · 2017-03-26T19:52:23.000Z

In any case, it should match the document that only some people have access to.

@austinc203 Why is that document not more widely shared?

Answer 7 · 2017-03-27T00:05:49.000Z

1 source of truth 👍

These might also be relevant:

Answer 8 · 2017-03-27T20:34:11.000Z

I am in favor of making this document part of a wiki, or it's own repository. @austinc203 @m-owenquinn

Answer 9 · 2017-03-30T16:05:21.000Z

Snapshot Summary

diagnostics: mark score, provide feedback
practice: mark completion, provide feedback
study: mark completion, provide feedback
challenges: mark completion, feedback must be specifically requested

Answer 10 · 2017-03-30T16:24:57.000Z

I disagree. A pull request on a challenge is a request for feedback.

Answer 11 · 2017-03-30T16:40:14.000Z

@gaand that makes sense and would mean that we always provide feedback on a PR made on time. @jrhorn424 does this match your intentions?

Snapshot Summary:

Feedback will only be provided for PR's that are made on time.

Diagnostics: PR required, mark score, provide feedback
Practice: PR required, mark completion, provide feedback
Study: PR required, mark completion, provide feedback
Challenges: PR optional, mark completion, provide feedback

Also, I believe this issue has trickled away from the Orientation repository and has information relevant to the instructor wiki.

I have referenced https://github.com/ga-wdi-boston/instructors/issues/54 and I have added the above to the following in the instructor wiki:

Answer 12 · 2017-03-30T18:55:01.000Z

Effort required by consultants decreases as you descend the list.

Answer 13 · 2017-03-30T18:56:39.000Z

I favor wiki over repository. But I don't have a good argument for it, so, whatever. 😛

Answer 14 · 2017-03-30T23:41:21.000Z

The fact we've already started using wikis for other things is an argument in and of itself. Not a good one, but it is one.

Answer 15 · 2017-04-17T22:58:47.000Z

If searching worked better against the wiki, and it may work better than my experience of it, then I would support wiki more.

Answer 16 · 2017-04-18T14:25:27.000Z

I'm not totally convinced of the Wikis benefit beyond being able to circumvent the git work flow/material prep work flow (circumventing does allow us to make changes more quickly). Potentially adding a EvaluationExpectations.md part to the Orientation repository might be enough.