Fastest

Creates unit tests from examples in the docstring and more.

Install

$ pip install fastest

Usage

$ fastest

watches all .py files and creates coverage for entire project.

$ fastest --path=$(pwd) --source=py_module

where path is the the project root, and source is same as the value passed to the command coverage run -m unittest --source=$source test

$ fastest --exclude=dont_check_this_dir/*,these__*.py

To exclude files/folders use --exclude and the file watcher will ignore them. The test/* folder that faster creates is excluded by default.

$ fastest --poll-duration=10

Builds files, runs tests and coverage every 10s, default = 1s

Things that happen when you run python main.py --path=$(pwd):

Checks for a test file at the project root, it creates if it doesn't find one.
Watches .py files for changes.
Creates unittests if a function has examples in its docstrings like so:

# .
# ├──module_a
# ├──module_b
#    └── utils.py
#
def add(x, y):
    """
    ----
    examples:
    1) add(3, 4) -> 7
    """
    return x + y

This will create a unittest in the test directory, assertEqual(add(3, 4), 7) within Class test_<file>_<function>(self) (for the given directory, tree: Class test_utils_add(self))

Runs all tests that are created.
Creates a coverage report (in html format).
Print the link to the coverage reports' index.html.

How to make best use of Fastest

Keep your functions light:
- Be paranoid about separation of concerns.
- Too many conditions are a hint that you might need another function.
- Complex loops and if-else are not scalable code, a single mistake would take that tower down and feature additions would involve someone going through that brain-teaser.
Use libraries but wrap them with your own functions. Like: Use requests or the inevitable database? wrap them with your own functions.
- Helps with adding customizations in one place (configuring things like base url, and similar configs)
- Helps mocking so that entire code-base can be unit tested.
Docstrings may get outdated if your work pace is too fast to maintain quality documentation. Now adding examples now would help you create tests which prevents your descriptions from going stale, if the tests fail, probably the documentation needs a second look too. This is enforced within Fastest, as documentation IS contributing to tests.

Examples:

Allows creation of variables within the docstrings, which includes lambda functions!

def quick_maths(a, b):
   """
   ----
   examples:
   @let 
   a = {
       'apples': 3,
       'oranges': 4
   }
   @end
   
   1) quick_maths(a['apples'], a['oranges']) -> 7
   ----
   """
   return a + b

You can run any valid python code within @let--@end blocks.

Can include installed modules external to your project.

def current_time():
   """
   ---
   examples:
   @need
   from datetime import datetime
   @end
   1) current_time() -> datetime.now()
   """
   return datetime.now()

If types are added to docstring, Fastest will create tests for checking type of the value returned against empty of arguments.
```
def chain_strings(str1, str2):
    """
    :param str1: str
    :param str2: str
    :return: str
    """
    return str1 + str2
```
Fastest will create a assertInstanceIs(chain_strings('', ''), str) for the above snippet.

To create an assertRaises test-case:

def crashes_sometimes(input_string):
   """
   ----
   examples: 

   !! crashes_sometimes(None) -> ValueError
   ----
   """
   if not input_string:
       raise ValueError
   return input_string

The syntax marked with !! crashes_sometimes(None) -> ValueError handles exceptions that the code throws

Fuzzing json apis

Note: work in progress

An experimental feature that can test your api's (currently supporting json request bodies only!). A happy-case example is required and fastest will figure out the rest, it will e-mail the errors to you.

example-case:

router.post('/test', function(req, res, next) {
  const { messageObject } = req.body;
  res.json({ prop: messageObject.message });
});

The above is a snippet from an express-server's post request. It expects a messageObject within the req.body. This api returns a response containing messageObject.message without ever checking if it is present.

Hate mistakes like these? Make sure to check out the fuzz.log.

url: /test
request_body: {'messageObject': None}
response: <!DOCTYPE html><html><head><title></title><link rel="stylesheet" href="/stylesheets/style.css"></head><body><h1>Cannot read property 'message' of null</h1><h2></h2><pre>TypeError: Cannot read property 'message' of null
    at /home/ltbringer/programming/js/sample-json-api/routes/index.js:11:34
    at Layer.handle [as handle_request] (/home/ltbringer/programming/js/sample-json-api/node_modules/express/lib/router/layer.js:95:5)
    at next (/home/ltbringer/programming/js/sample-json-api/node_modules/express/lib/router/route.js:137:13)
    at Route.dispatch (/home/ltbringer/programming/js/sample-json-api/node_modules/express/lib/router/route.js:112:3)
    at Layer.handle [as handle_request] (/home/ltbringer/programming/js/sample-json-api/node_modules/express/lib/router/layer.js:95:5)
    at /home/ltbringer/programming/js/sample-json-api/node_modules/express/lib/router/index.js:281:22
    at Function.process_params (/home/ltbringer/programming/js/sample-json-api/node_modules/express/lib/router/index.js:335:12)
    at next (/home/ltbringer/programming/js/sample-json-api/node_modules/express/lib/router/index.js:275:10)
    at Function.handle (/home/ltbringer/programming/js/sample-json-api/node_modules/express/lib/router/index.js:174:3)
    at router (/home/ltbringer/programming/js/sample-json-api/node_modules/express/lib/router/index.js:47:12)</pre></body></html>
status_code: 500

This feature is still under progress because:

Signatures that have resulted in errors need to be avoided.
A simple to use format in which server request-bodies and other config should be shared is not yet formalized.
Error-Grouping: ability to detect when multiple signatures are leading to similar error or affects same area of code is not present.
Optimizing happy-case: signatures that don't cause errors and an appropriate response is returned by the server under test, need not be sent back to the server again.

Goals for Fastest

Help maintaining tests, code-coverage and documentation.
Help with performance issues within code.
Provide testability score for code.
Test functions for auto-generated inputs where the code would crash.

Fastest uses itself for creating tests and manages a 100% on the coverage!