plone.recipe.codeanalysis
provides static code analysis for Buildout-based Python projects, including flake8, JSHint, CSS Lint, and other code checks.
This buildout recipe creates a script to run the code analysis:
bin/code-analysis
By default plone.recipe.codeanalysis
also creates a git pre-commit hook, in order to run the code analysis automatically before each commit.
plone.recipe.codeanalysis
comes with a Jenkins integration, that allows to use the same code analysis settings on your local machine as well as on Jenkins.
It also allows to run code analysis to any arbitrary folder:
bin/code-analysis src/Products.CMFPlone
Just add a code-analysis section to your buildout.cfg:
[buildout]
parts += code-analysis
[code-analysis]
recipe = plone.recipe.codeanalysis
directory = ${buildout:directory}/src
The directory option is not required. Though, if you don't specify a directory the code analysis will check every file in your buildout directory.
This configuration is helpful for working on already existing packages. If you create a new package you might want to enable all checks. This configuration looks like this:
[code-analysis]
recipe = plone.recipe.codeanalysis[recommended]
multiprocessing = True
jenkins = False
directory =
${buildout:directory}/src
pre-commit-hook = True
# JS
jshint = True
jshint-bin = ${buildout:bin-directory}/jshint
jshint-suppress-warnings = False
jscs = True
jscs-bin = ${buildout:bin-directory}/jscs
jscs-exclude =
${buildout:directory}/dev/bower_components
${buildout:directory}/node_modules
# CSS
csslint = True
csslint-bin = ${buildout:bin-directory}/csslint
# ZPT
zptlint = True
zptlint-bin = ${buildout:bin-directory}/zptlint
# TS
tslint = True
tslint-bin = ${buildout:directory}/bin/tslint
tslint-exclude = ${:jscs-exclude}
# Conventions
clean-lines = True
clean-lines-exclude = ${:jscs-exclude}
# i18n
find-untranslated = True
i18ndude-bin = ${buildout:bin-directory}/i18ndude
return-status-codes = True
flake8-exclude = bootstrap.py,bootstrap-buildout.py,docs,*.egg,*.cpy,*.vpy,overrides
[node]
recipe = gp.recipe.node
npms = csslint jshint jscs tslint
scripts = csslint jshint jscs tslint
This extra enables a host of flake8 plugins. They are mostly coding Plone's styleguide (specially the Python section).
These are the current extras installed:
- flake8-blind-except: warns about catching any exception, i.e
except:
- flake8-coding: warns about python files with missing coding header
- flake8-debugger: warns about debug statements found in code (like pdb...)
- flake8-deprecated: warns about deprecated method calls
- flake8-isort: warns about imports not sorted properly (note that an extra configuration is needed)
- flake8-pep3101: warns about old-style formatting, i.e
'format a %s' % string
- flake8-plone-api: warns about code that could be replaced by plone.api calls (note that this is forbidden for Plone core packages)
- flake8-plone-hasattr: warns about using
hasattr
as it shallows exceptions - flake8-print: warns about
print
being used - flake8-quotes: warns about using double quotes (plone style guide says single quotes)
- flake8-string-format: warns about errors on string formatting
- flake8-todo: warns if there are
TODO
,XXX
found on the code - flake8-commas: warns if the last element on a method call, list or dictionary does not end with a comma
- flake8_strict: warns if a multi-line method call has its first element on the first line
plone.recipe.codeanalysis provides a Jenkins setting that allows to run it on a Jenkins CI server and to process and integrate the output via the Jenkins Violations plugin.
Usually you don't want the recipe to create Jenkins output files on your local machine. Therefore it makes sense to enable the Jenkins output only on the CI machine. To do so, just create a jenkins.cfg that extends and overrides the default buildout file (that includes the other settings):
[buildout]
parts += code-analysis
[code-analysis]
recipe = plone.recipe.codeanalysis
jenkins = True
The Jenkins job itself should run bin/code-analysis
:
python bootstrap.py -c jenkins.cfg
bin/buildout -c jenkins.cfg
bin/jenkins-test --all
bin/code-analysis
The Jenkins Violations plugin needs to be configured to read the output files generated by this configuration.
pep8 (to read the flake8 output):
**/parts/code-analysis/flake8.log
csslint:
**/parts/code-analysis/csslint.xml
jslint (to read the jshint output):
**/parts/code-analysis/jshint.xml
checkstyle (to read the jscs output):
**/parts/code-analysis/jscs.xml
If jenkins is set to False, you can still store the output on the filesystem by setting flake8-filesystem = True
. This is ignored if jenkins is set to True.
output:
**/parts/code-analysis/flake8.txt
Code repository:
Continuous Integration:
Issue Tracker:
If you need to bypass checks for some reasons on a specific line you may use # noqa
in Python or // noqa
in Javascript files. This works for most of our checks.
The recipe supports the following options:
- directory
Directory that is subject to the code analysis.
- pre-commit-hook
If set to True, a git pre-commit hook is installed that runs the code analysis before each commit. Default is
True
.- multiprocessing
If set to
True
,code-analysis
will fork multiple processes and run all linters in parallel. This will dramatically increase speed on a multi-core system, specially when usingcode-analysis
as pre-commit hook. Default isFalse
.- jenkins
If set to True, the flake8, jshint and csslint code analysis steps will write output files that can be processed by the Jenkins Violations plugin. Default is
False
.- flake8-filesystem
If set to True, the flake8 code analysis step will write an output file. Ignored if jenkins is True. Default is
False
.- flake8
If set to True, run Flake8 code analysis. Default is
True
.- flake8-extensions
Flake8 now takes advantage of
flake8
extension system. Default is none. Ifflake8
is set to False, this option will be ignored. Example to supercharge with some extensions:
[code-analysis]
recipe = plone.recipe.codeanalysis
flake8 = True
flake8-extensions =
flake8-blind-except
flake8-coding
flake8-debugger
flake8-quotes
pep8-naming
All through flake8 extensions raised validation errors may be suppressed using the flake8-ignore
option.
- flake8-ignore
Skip errors or warnings. See Flake8 documentation for error codes. Default is none.
- flake8-exclude
Comma-separated filename and glob patterns default. Say you want to exclude bootstrap.py, setup.py and all collective.* and plone.* packages. Just set
flake8-exclude=bootstrap.py,docs,*.egg,setup.py,collective.*,plone.*
in your buildout configuration. Default isbootstrap.py,bootstrap-buildout.py,docs,*.egg
.- flake8-max-complexity
McCabe complexity threshold. Default is
10
.- flake8-max-line-length
Set maximum allowed line length. Default is
79
.
Note
You can add additional flake8 options as long as they are valid for flake8 itself or any of its plugins. Just prefix them with flake8-
.
For example, if you are using pep8-naming
and want to change the list of ignored names, add the following line on your buildout.cfg: flake8-ignore-names = setUp,tearDown,setUpClass,tearDownClass
Look at flake8 documentation and its plugins to see which options are available.
- check-manifest
If set to True,
check-manifest
will be run to check you MANIFEST.in file. Default isFalse
.- check-manifest-directory
Default is
.
which means check the current package where you included code-analysis in buildout.EXPERIMENTAL: For project buildouts where you use several source packages you may want to enter multiple directories or use
${buildout:develop}
to include all your development packages.- jshint
If set to True, jshint code analysis is run. Default is
False
. Note that plone.recipe.codeanalysis requires jshint >= 1.0.- jshint-bin
JSHint executable. Default is
jshint
. If you have JSHint installed on your system and in your path, there is nothing to do. To install JSHint in your buildout, use the following:
[jshint]
recipe = gp.recipe.node
npms = jshint
scripts = jshint
set jshint-bin to ${buildout:bin-directory}/jshint
.
- jshint-exclude
Allows you to specify directories which you don't want to be linted. Default is none. If you want JSHint to skip some files you can list them in a file named
.jshintignore
. See JSHint documentation for more details.- jshint-suppress-warnings
By default warnings of jshint are suppressed and not shown. You may disable this by setting to False, default is
True
for backward compatibility reasons.- jscs
If set to True, jscs code analysis is run. Default is
False
.JavaScript Code Style options should be configured using a
.jscs.json
file. You should align your javascript code to the next generation of Plone's javascript framework Mockup and take it's.jscs.json
file which is available here: https://github.com/plone/mockup/blob/master/mockup/.jscs.jsonAll configuration options are documented on the jscs website.
- jscs-bin
Set the path to a custom version of JSCS, e.g.
/usr/local/bin/jscs
.If you have Javascript Code Style Checker installed in your system and path, you have nothing to do. To install with Buildout, add the following section to your buildout and set jscs-bin to
{buildout:bin-directory}/jscs
:
[jscs]
recipe = gp.recipe.node
npms = jscs
scripts = jscs
- jscs-exclude
Allows you to specify directories and/or files which you don't want to be checked. Default is none. Note that these directories have to be given in absolute paths, use
${buildout:directory}/foo/bar/static/js-3rd-party
for example.- csslint
If set to True, CSS Lint code analysis is run. Default is
False
.CSS Lint options should be configured using a
.csslintrc
file. A typical.csslintrc
file will look like this:--format=compact --quiet --ignore=adjoining-classes,floats,font-faces,font-sizes,ids,qualified-headings,unique-headings --exclude-list=foo/bar/static/third-party.css
This typical configuration includes a list of CSS rules that will be ignored as they are considered useless.
See CSS Lint documentation and CSS Lint command-line interface for a detailed list and description of the rules.
- csslint-bin
Set the path to a custom version of CSS Lint, e.g.
/usr/local/bin/csslint
.If you have CSS Lint installed in your system and path, you have nothing to do. To install CSS Lint with Buildout, add the following section to your buildout and set csslint-bin to
{buildout:bin-directory}/csslint
:
[csslint]
recipe = gp.recipe.node
npms = csslint
scripts = csslint
- csslint-exclude
Allows you to specify directories and/or files which you don't want to be checked. Default is none.
- clean-lines
If set to True, any file containing trailing spaces or tabs anywhere on the lines will cause a warning. Default is
False
.- clean-lines-exclude
Allows you to specify directories and/or files which you don't want to be checked. Default is none.
- return-status-codes
If set to True, the
bin/code-analysis
script returns an error code that Continuous Integration servers (like Travis CI) can use to fail or pass a job, based on the code analyis output. Note that Jenkins usually does not need this option (this is better handled by the Jenkins Violations plugin). Note that this option does not have any effect on the other code analysis scripts. Default isFalse
.
To reduce the number of Zope/Plone direct dependencies, plone.recipe.codeanalysis no longer depends on i18ndude nor zptlint; in order to use the following options you have to install them on your system:
- find-untranslated
If set to True, scan Zope templates to find untranslated strings. Default is
False
. To use this you will need to set thei18ndude-bin
option.- find-untranslated-exclude
Allows you to specify directories and/or files which you don't want to be checked. Default is none.
- i18ndude-bin
Set the path to a custom version of i18ndude. Default is none.
- zptlint
If set to True, zptlint code analysis is run. Default is
False
. To use this you will need to set thezptlint-bin
option.- zptlint-bin
Set the path to a custom version of zptlint. Default is none.
- zptlint-exclude
Allows you to specify directories and/or files which you don't want to be checked. Default is none.
JSHint "ERROR: Unknown option --verbose":
JSHint [ OK ]
ERROR: Unknown option --verbose
Upgrade JSHint to latest version (>= 1.0) to fix this issue, e.g.:
$ sudo npm install -g jshint
JSHint "ERROR: Unknown option --exclude":
JSHint [ OK ]
ERROR: Unknown option --exclude
Upgrade JSHint to latest version (>= 2.1.6) to fix this issue, e.g.:
$ sudo npm install -g jshint