jamescooke/flake8-aaa

Add "teaching" mode

daknuett opened this issue · 8 comments

daknuett commented

In order to make this plugin more helpful for young programmers a (let's call it) "teaching mode" would be nice.

This mode should not only provide a detailed description why an error has been raised (maybe including a link to the according docs) but also some tips how to improve the tests further (like adding a result = in a pytest.raises block).

daknuett commented

I am willing to help as soon as I have the time to do so (about March 2019).

jamescooke commented

Hey @daknuett - thanks for the suggestion. So that we can be on the same page, could you give a bit more context around the problems that you think a "teaching mode" might solve?

daknuett commented

I hope that this mode would allow inexperienced developers to improve their tests without someone teaching them.

I have made the experience that people that started programming recently (around 1 year ago) cannot find help themselves (for instance why flake8-aaa gives an error code). To teach them how to get the information themselves and how to improve their tests a constructive error message (or warning for other stuff) would be really helpful.

However experienced devs don't need those messages, so one should be able to turn that feature off.

jamescooke commented

Hi @daknuett - thanks for that illustration 👍

My guess is that the goal for Flake8-AAA to be able to help people "to improve their tests without someone teaching them" is pretty amazing - it would be great to achieve 🙏

As context, I have been really impressed by demos of Elm's compiler messages (although I've not used the language myself) - see https://elm-lang.org/blog/compiler-errors-for-humans

There are two things available which might help:

Could these two things be combined? Would it be helpful to show the resolution for an error when run on the command line?

This would potentially look like:

 4 ACT|    result = x + y
           ^ AAA03 expected 1 blank line before Act block, found none

             Resolution: Add a blank line before the Act block.

Do you think that would be helpful and take us a step forward?

daknuett commented

The new command line functionality gives a verbose output showing the line and offset when an error is found.

That would be indeed the fundamental of helping people.

We have the "Resolution" blocks in the documents for each error code

That would be the most important part in my opinion. Maybe it should include a link to the documentation.

Those two things would bring us a great step forward. Also I think that it is important that one can turn off this feature, because many experienced devs will be annoyed by too much output. (After using flake8-aaa a few weeks you just know the error codes and how to fix them)

I am currently observing a course in python and C. What I have observed so far is that the biggest problem is that nobody knows what tools to use and how to interpret the errors. Like nobody headed of valgrind. Or what to do when python says it cannot add a list and a set.

jamescooke commented

@daknuett I'm going to close off this ticket - feel free to comment. Let me explain why.

I really like the idea of a linter / checker that links users to docs to help them solve the issue / bug. But while reflecting on that over the last few months, I think this is an overstretch for this Flake8 plugin - this is out of scope for this project. If we were going to extend Flake8-AAA to give detailed guidance / links when things go wrong, then that will involve a reasonably complex command-line invokable script which wraps Flake8 and interprets its output.

Instead this is a Flake8 plugin and so returns errors to the Flake8 caller in the way that it expects. In that respect its scope is simple - to have well specified errors with good documentation that help improve people's tests.

BUT...

I would be up for seeing if there is the possibility of a separate / new project incorporating a command line tool that could make linting errors more friendly and helpful. Something similar to https://github.com/aroberge/friendly-traceback, but for Flake8 and gang. As part of that, we might be able to help beginners with all Flake8's standard errors, plus errors from other plugins like this one.

Let me know your thoughts.

jamescooke commented

As of #149, errors will have their own dedicated page with help on resolution. When run from command line, error output will point at the errors' dedicated pages using something like the Resolution: suggestion above:

 4 ACT|    result = x + y
           ^ AAA03 expected 1 blank line before Act block, found none

For more information:
  AAA03: https://flake8-aaa.readthedocs.io/en/stable/errors/AAA03-expected-1-blank-line.html
daknuett commented

Well, it has been quite a while. Sorry, that it took me so long to answer. Personally I like the idea of giving students the program they will use in production with some extra functionality for teaching. It makes it easier for students to understand what is going on in production.

But I understand that you don't want to bloat flake8-aaa with some the "teaching" stuff. It would be functionality that only very few would use.

  • 👍1