_Simple but not Simpler_
__Practicality Beats Purity __
___Automation Brings Salvation___
- Auto-formatters
- Functions
- Docstring
- Imports
- Whitespaces
- Naming Conventions
- Exception Handling
- Logging
- Testing
- Flask
- The Holy Grail of Being Pythonic
- References
-
Use black with default settings (max character length 88 lines).
-
Use flake8 to catch linting errors.
-
Use isort to sort the imports.
=> However, be careful while sorting imports of application modules. It might accidentally add circular dependencies. For example in Flask applications, changing the import order in flask
app/package/__init__.py
can mess up the codebase.from flask import Blueprint package = Blueprint("package", __name__) from . import views
To avoid this, add
# isort:skip
like this:from flask import Blueprint package = Blueprint("package", __name__) from . import views # isort:skip
=> Black is not compatible with isort. So it's better to run black after running isort.
-
Avoid mutable data types as default function/method aruments.
# bad def make_list(val, lst=[]): lst.append(val) return lst make_list(1) # => [1] make_list(2) # => [1, 2], instead of the new init [2] # good def make_list(val, lst=None): if lst is None: lst = [] lst.append(val) return lst init_list(1) # => [1] init_list(2) # => [2]
-
Call function paramaters by specifying their names
def point(x, y, z=None): # ... # bad - unclear what the params mean point(1, 0, 5) # good point(x=1, y=0, z=3)
-
Use numpy style docstring. This is a good format that uses extra vertical space for maximum readability.
def dumb_add(num0, num1, num2, num3, num4): """[summary] Parameters ---------- num0 : [type] [description] num1 : [type] [description] num2 : [type] [description] num3 : [type] [description] num4 : [type] [description] Returns ------- [type] [description] """
-
Single line docstring.
def dumb_sub(num1, num0): """Subtracting num0 from num1."""
-
Inline comments should start with lowercase letter.
# going through the student list for idx, student in enumerate(students_list): ...
-
Do not use wild card import
# bad from mod import * from package.mod import * # good from mod import func_0 from package.mod import func_1
-
Do not import unused modules (Flake8 will point out unused imports, make sure you remove them before committing).
-
Sort the imports by import then from, and sort alphabetically (
isort
will automatically do this for you).# bad from a_mod import foo import e_mod import b_mod from z_mod import bar, baz # good import b_mod import e_mod from a_mod import foo from z_mod import bar, baz
-
Use soft tabs (space character) set to 4 spaces as per PEP8.
# bad def foo(): ∙∙return bar # bad def foo(): ∙return bar # good def foo(): ∙∙∙∙return bar
-
Use snake_case when naming variables, functions, and instances. Use it for file names too as they will be used in imports.
# bad import myMod anOBJEct = {} thisIsAnObject = {} def ThisisAFunction(): # good import my_mod anobject = {} this_is_an_object = {} def this_is_a_function():
-
Use PascalCase only when naming classes.
# bad class exampleDummyFactory(): # ... fact = exampleDummyFactory() # good class ExampleDummyFactory(): # ... fact = ExampleDummyFactory()
-
Avoid single letter names. Use descriptive and meaningful names - tell what the function does, or what data type an object is. Use description_object instead of object_description
# bad def a(): # ... # good def analogy(): # ... # bad - no convention to know what data type it is df_raw_data = pd.DataFrame(raw_data) id_dct_num = {"a": 1, "b": 2} # good - convention to tell data type by the last term raw_data_df = pd.DataFrame(raw_data) id_num_dct = {"a": 1, "b": 2} # bad - meaningless names, lost context LIST_1 = ["Jack", "Alice", "Emily"] # ... many lines of code later for item in LIST_1: add_person(item) # good NAME_LIST = ["Jack", "Alice", "Emily"] # ... many lines of code later for name in NAME_LIST: add_person(name)
-
Use singular or base words in naming; avoid using plural and instead append singular with the data type.
# bad def moves_object(x, y): # ... # good def move_object(x, y): # ... # bad - inconsistent naming for same data type and usage teacher = ["Michael"] students = ["Jack", "Alice", "Emily"] books = pd.DataFrame({"title": ["lorem", "ipsum"]}) for t in teacher: add_human(t) for student in students: add_human(student) for book in books: add_item(book) # wrong; iterate column name instead of book # good teacher_list = ["Michael"] student_list = ["Jack", "Alice", "Emily"] book_df = pd.DataFrame({"title": ["lorem", "ipsum"]}) for teacher in teacher_list: add_human(teacher) for student in student_list: add_human(student) # naming as df suggests it shall be treated as a dataframe for idx, book in book_df.iterrow(): add_item(book)
-
Do not write bare try-except block
# bad try: do_something() except: pass # bad try: do_something() except Exception: pass
-
In most cases, the caught exception type must be as specific as possible. Something like KeyError, or ConnectionTimeout, etc.
# good try: do_something() # catch some very specific exception - KeyError, ValueError, etc. except ValueError: pass
-
If some code path simply must broadly catch all exceptions - for example, the top-level loop for some long-running persistent process - then each such caught exception must write the full stack trace to a log or file, along with a timestamp. Not just the exception type and message, but the full stack trace.
# good def get_number(): return int("foo") try: x = get_number() except ValueError: pass except Exception: logging.exception("Caught an error", exec_info=True)
-
Instantiate your logger in your package's
__init__.py
module. See how it's done inrequests
library here. -
Define a basic logging class
# demo of a logger in __init_.py import logging logging.getLogger(__name__) logging.basicConfig( level=logging.INFO, format="%(asctime)s [%(levelname)s] %(message)s", handlers=[logging.FileHandler("packg/debug.log"), logging.StreamHandler()], )
-
Import and use it like this
# using predefined logging in mod.py from . import logging def dumb_div(a): try: res = a // 0 except ValueError: res = a // 1 except Exception: logging.exception("Exception Occured") res = None return res dumb_div(5)
Folder structure
# folder structure of the package packg ├── debug.log ├── __init__.py └── mod.py
-
Use pytest to write your tests
-
Strive to write many small pure and idempotent functions, and minimize where mutations occur.
-
Whenever you fix a bug, write a regression test. A bug fixed without a regression test is almost certainly going to break again in the future.
-
Use direct assertations and explicit comparisons; avoid negations.
# bad - other values can be falsy too: `[], 0, '', None` assert not result assert result_list # good assert result == False assert len(result_list) > 0
Try to follow divisional structure while designing your microservice APIs. Here's an example of divisional structure in a Flask project.
yourapp/
__init__.py
admin/
__init__.py
views.py
static/
templates/
home/
__init__.py
views.py
static/
templates/
control_panel/
__init__.py
views.py
static/
templates/
models.py
Read more on divisional structure here.