This directory contains an initial version of ridl, a tool for documenting python code. Ridl is a replacement for Sphinx. It is similar to Sphinx in that uses Python introspection to automatically extract signatures and docstrings for functions/methods. But it addresses some of the problems that made Sphinx a pain to use.
The next section explains a bit more about what ridl is and how it differs from sphinx, if you want to skip it and just run it, jump down to "running ridl" below.
Ridl addresses several things that make sphinx inconvenient to use, at least for our purposes with flyscript. First, Sphinx uses a text markup scheme called restructured text, which is awkward and clunky. Ridl uses markdown instead. At a high level, markdown and restructured text are similar but markdown has gained a lot more traction, mostly because its syntax is simpler and less cumbersome.
Another difference between sphinx and ridl is in the handling of
automatic documentation of python code. Sphinx is relatively rigid.
You can either ask it to document everything in a module or class or
you can write directives to enumerate individual functions, methods,
etc. In the first case you get no control over the order that items
are presented and no opporunity to insert additional text into the
documentation. Sphinx also includes the class name in the documentation
so there is no way to document the interface to a class without
exposing the name of the class, even if the class name isn't meant
to be public to your users. As an example to make this last point
concrete, suppose file objects were implemented in python and you
wanted to automatically document them. Since users only get file
objects as the return value from functions like open(), the full
module name should not be part of the object documentation, but
Sphinx always embeds the full name.
So, to get flexibilty we end up writing our own documentation container with embedded directives to document individual methods. But Sphinx has poor tools for tracking what has been documented and warning if any classes/methods/functions/etc are undocumented. Ridl uses the same structure but it tracks exactly what has and has not been documented and it emits a full list of all undocumented components after each run.
Finally, Sphinx treats python docstrings as literal markup. To create links in the generated documentation, we have to put restructured text markup into docstrings which compromises readability in an interactive python session (or with any other tool that reads docstrings such as eclipse or other IDEs). Ridl has several features for automatically creating links within docstrings, these features are described in detail below. But this approach lets us create links for things like common glossary terms, function names, etc. without polluting docstrings with markup.
To run ridl, you need to have the
python-markdown
library installed. If you're impatient, pip install python-markdown
should be enough.
Ridl takes a single mandatory command-line argument: the path to a
directory that holds the source files to be processed. The format
of these source files is described in the next section. This
directory must also contain a file called conf where a number of
options and parameters are specified. The format of this configuration
file is described later in this document.
XXX create a simple example to refer to here
As described above, a major goal of ridl is for python docstrings to be written in plain text, free of any markup but also to still have useful hyperlinks in the rendered html output. To do this, ridl provides two schemes for automatically inserting hyperlinks into python docstrings. These schemes are completely controlled by parameters specified at configuration time so you can use them as aggressively or as conservatively as you like.
Ridl treats any words inside docstrings that end with a pair
of parentheses (()) as a potential reference to another function
or method. When such a string is encountered, the ridl processor
extracts the string before the parentheses (e.g., from foo()
the string foo is extracted) and looks up that string as a symbol
first in the python class being currently documented, and then in
the python module currently being documented. If this string does
refer to a python function or method, the original word automatically
becomes a link to the documentation for that function or method.
The autolink directive in the ridl configuration file can be used
to force specific words or terms to automatically be turned into
hyperlinks. A typical use is for terms in a glossary. Take care
with auto-links, as using them for common words can easily lead to
links being created more frequently than you might intend...
The actual source files to be processed by ridl are written in markdown with a few additional directives available. For each source file, the directives described in this section are processed and then the resulting contents are fed directly into a markdown formatter.
Ridl directives are enclosed in curly braces (e.g., {foo}).
The following directives are recognized:
Insert the version string declared in the ridl configuration file
(see the configuration file version directive below).
Insert the date and time at which the documentation was formatted.
Create an "anchor" that can be the target of hyperlinks. This directive is followed by the name of the anchor, and then arbitrary markdown for the contents of the anchor. For example, the following directive:
{anchor intro ## Introduction}
logically translates into the following html:
<h2 id="intro">Introduction</h2>
(this is done by translating the original directive into the following fragment of markdown):
## Introduction{: #intro}
Note that in practice, additional markup is added for styling, etc.
Set the current python module being documented. This directive must be followed by the name of a python module, the given module will be immediately imported so it must be present somewhere in the python interpreter's path. Since ridl itself is a python script, it is not enough to spawn ridl from a directory that holds the python code you are documenting, if your code is not installed into the global system python path, you will probably need to set the PYTHONPATH environment variable.
After this directive, subsequent {class} and {function}
directives will refer to classes and functions within the named
module. In addition, while processing this directive, the module's
docstring will be emitted. The silent keyword can be added to
the directive to suppress printing the docstring.
Example:
{module mymodule.submodule}
Document the given function. This will print the signature for the function as well as its docstring.
Set the current python class being documented.
This directive must be preceded by a {module} directive to
set the current python module.
This directive establishes the python class referred to by subsequent
{method} directives. It also emits some documentation for the
class:
- The full class name (as a linkable anchor as described above)
- The docstring for the class itself
- The signature and docstring for the class constructor (
__init__())
As with the {module} directive, this directive can be followed by
the silent keyword to suppress printing the information above.
Suppressing the information above is useful for documenting classes
that have a public interface but that are never instantiated directly
by users of your library (in the standard python library,
file objects and regular expression match objects are examples of
classes that are not instantiated directly by users but that have
a set of documented methods).
Document the given method.
This directive must be preceded by a {class} directive to set
the current python class.
This directive documents the given method by printing
the signature for the method and its docstring. The method
signature is also a linkable anchor that can be referenced using
the {methodref} directive.
If instead of a method name, this directive appears as
{method *}
then this directive will look up the names of all public methods in the current class (i.e., those that do not begin with an underscore) and document each of them.
Create a link to the given method. XXX example
The behavior of ridl is controlled by a simple configuration file.
In the configuration file, blank lines and lines that begin with a
pound sign (#) are ignored. Valid directives for the
remaining lines are:
These directives each are followed by a directory path and
are mandatory. outdir specifies the output directory where
the rendered html files plus other support files will be created.
tempdir specifies a directory where intermediate files can be
stored during processing (XXX this will be become optional at some point).
For both of these directives, if the given path begins with a /
it is an absolute path, otherwise it is relative to the input directory.
These directives are optional. If present, each should be followed
by the name of a file (in the md/ subdirectory of the input directory)
to be prepended/appended to each file. This is useful for creating
a standard page layout. Note that these files are processed by ridl
and markdown so you can use the directives described here and other
markdown directives in your headers and footers.
XXX describe .md vs .html extension
This optional directive loads the named extension for
python-markdown.
Configuration options for the extension may be specified as a
series of additional arguments with the format name=value.
This directive lists the input files that ridl should process.
It may appear multiple times if the list of input files doesn't
fit neatly on a single line. Note that these files should be in
the md/ subdirectory of the input directory.
This directive enumerates files that should be simply copied from the input directory to the output directory. It is useful for custom images, style sheets, etc. The directive may appear multiple times if the list of static files does not feat neatly on one line.
This directive sets the string that will appear in the <title> tag in each rendered html page.
This directive sets the string that will be substituted for any
{version} directives in source files. If the contents of this
directive looks like a file path, that contents of that file will be
read and used.
This directive sets up auto-linking as described above. It may appear multiple times to create multiple auto-link rules. Each directive is followed by a regular expression (using the Python regular expression syntax and the relative url to link to.
This optional directive specifies the top-level python module to be documented. At the end of a run, ridl will only warn about undocumented objects with names that begin with this prefix.
This optional directive suppresses the ridl warning if the given python object (module, class, function, or method) is not documented. It may appear multiple times.