<?xml version="1.0" encoding="utf-8" ?> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en"> <head> <meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> <meta name="generator" content="Docutils 0.9: http://docutils.sourceforge.net/" /> <title>pycparser v2.05</title> <meta name="author" content="Eli Bendersky" /> <style type="text/css"> /* :Author: David Goodger (goodger@python.org) :Id: $Id: html4css1.css 7056 2011-06-17 10:50:48Z milde $ :Copyright: This stylesheet has been placed in the public domain. Default cascading style sheet for the HTML output of Docutils. See http://docutils.sf.net/docs/howto/html-stylesheets.html for how to customize this style sheet. */ /* used to remove borders from tables and images */ .borderless, table.borderless td, table.borderless th { border: 0 } table.borderless td, table.borderless th { /* Override padding for "table.docutils td" with "! important". The right padding separates the table cells. */ padding: 0 0.5em 0 0 ! important } .first { /* Override more specific margin styles with "! important". */ margin-top: 0 ! important } .last, .with-subtitle { margin-bottom: 0 ! important } .hidden { display: none } a.toc-backref { text-decoration: none ; color: black } blockquote.epigraph { margin: 2em 5em ; } dl.docutils dd { margin-bottom: 0.5em } object[type="image/svg+xml"], object[type="application/x-shockwave-flash"] { overflow: hidden; } /* Uncomment (and remove this text!) to get bold-faced definition list terms dl.docutils dt { font-weight: bold } */ div.abstract { margin: 2em 5em } div.abstract p.topic-title { font-weight: bold ; text-align: center } div.admonition, div.attention, div.caution, div.danger, div.error, div.hint, div.important, div.note, div.tip, div.warning { margin: 2em ; border: medium outset ; padding: 1em } div.admonition p.admonition-title, div.hint p.admonition-title, div.important p.admonition-title, div.note p.admonition-title, div.tip p.admonition-title { font-weight: bold ; font-family: sans-serif } div.attention p.admonition-title, div.caution p.admonition-title, div.danger p.admonition-title, div.error p.admonition-title, div.warning p.admonition-title { color: red ; font-weight: bold ; font-family: sans-serif } /* Uncomment (and remove this text!) to get reduced vertical space in compound paragraphs. div.compound .compound-first, div.compound .compound-middle { margin-bottom: 0.5em } div.compound .compound-last, div.compound .compound-middle { margin-top: 0.5em } */ div.dedication { margin: 2em 5em ; text-align: center ; font-style: italic } div.dedication p.topic-title { font-weight: bold ; font-style: normal } div.figure { margin-left: 2em ; margin-right: 2em } div.footer, div.header { clear: both; font-size: smaller } div.line-block { display: block ; margin-top: 1em ; margin-bottom: 1em } div.line-block div.line-block { margin-top: 0 ; margin-bottom: 0 ; margin-left: 1.5em } div.sidebar { margin: 0 0 0.5em 1em ; border: medium outset ; padding: 1em ; background-color: #ffffee ; width: 40% ; float: right ; clear: right } div.sidebar p.rubric { font-family: sans-serif ; font-size: medium } div.system-messages { margin: 5em } div.system-messages h1 { color: red } div.system-message { border: medium outset ; padding: 1em } div.system-message p.system-message-title { color: red ; font-weight: bold } div.topic { margin: 2em } h1.section-subtitle, h2.section-subtitle, h3.section-subtitle, h4.section-subtitle, h5.section-subtitle, h6.section-subtitle { margin-top: 0.4em } h1.title { text-align: center } h2.subtitle { text-align: center } hr.docutils { width: 75% } img.align-left, .figure.align-left, object.align-left { clear: left ; float: left ; margin-right: 1em } img.align-right, .figure.align-right, object.align-right { clear: right ; float: right ; margin-left: 1em } img.align-center, .figure.align-center, object.align-center { display: block; margin-left: auto; margin-right: auto; } .align-left { text-align: left } .align-center { clear: both ; text-align: center } .align-right { text-align: right } /* reset inner alignment in figures */ div.align-right { text-align: inherit } /* div.align-center * { */ /* text-align: left } */ ol.simple, ul.simple { margin-bottom: 1em } ol.arabic { list-style: decimal } ol.loweralpha { list-style: lower-alpha } ol.upperalpha { list-style: upper-alpha } ol.lowerroman { list-style: lower-roman } ol.upperroman { list-style: upper-roman } p.attribution { text-align: right ; margin-left: 50% } p.caption { font-style: italic } p.credits { font-style: italic ; font-size: smaller } p.label { white-space: nowrap } p.rubric { font-weight: bold ; font-size: larger ; color: maroon ; text-align: center } p.sidebar-title { font-family: sans-serif ; font-weight: bold ; font-size: larger } p.sidebar-subtitle { font-family: sans-serif ; font-weight: bold } p.topic-title { font-weight: bold } pre.address { margin-bottom: 0 ; margin-top: 0 ; font: inherit } pre.literal-block, pre.doctest-block, pre.math { margin-left: 2em ; margin-right: 2em } span.classifier { font-family: sans-serif ; font-style: oblique } span.classifier-delimiter { font-family: sans-serif ; font-weight: bold } span.interpreted { font-family: sans-serif } span.option { white-space: nowrap } span.pre { white-space: pre } span.problematic { color: red } span.section-subtitle { /* font-size relative to parent (h1..h6 element) */ font-size: 80% } table.citation { border-left: solid 1px gray; margin-left: 1px } table.docinfo { margin: 2em 4em } table.docutils { margin-top: 0.5em ; margin-bottom: 0.5em } table.footnote { border-left: solid 1px black; margin-left: 1px } table.docutils td, table.docutils th, table.docinfo td, table.docinfo th { padding-left: 0.5em ; padding-right: 0.5em ; vertical-align: top } table.docutils th.field-name, table.docinfo th.docinfo-name { font-weight: bold ; text-align: left ; white-space: nowrap ; padding-left: 0 } h1 tt.docutils, h2 tt.docutils, h3 tt.docutils, h4 tt.docutils, h5 tt.docutils, h6 tt.docutils { font-size: 100% } ul.auto-toc { list-style-type: none } </style> </head> <body> <div class="document" id="pycparser-v2-05"> <h1 class="title">pycparser v2.05</h1> <table class="docinfo" frame="void" rules="none"> <col class="docinfo-name" /> <col class="docinfo-content" /> <tbody valign="top"> <tr><th class="docinfo-name">Author:</th> <td><a class="first reference external" href="http://eli.thegreenplace.net">Eli Bendersky</a></td></tr> </tbody> </table> <div class="contents topic" id="contents"> <p class="topic-title first">Contents</p> <ul class="auto-toc simple"> <li><a class="reference internal" href="#introduction" id="id2">1 Introduction</a><ul class="auto-toc"> <li><a class="reference internal" href="#what-is-pycparser" id="id3">1.1 What is pycparser?</a></li> <li><a class="reference internal" href="#what-is-it-good-for" id="id4">1.2 What is it good for?</a></li> <li><a class="reference internal" href="#which-version-of-c-does-pycparser-support" id="id5">1.3 Which version of C does pycparser support?</a></li> <li><a class="reference internal" href="#what-grammar-does-pycparser-follow" id="id6">1.4 What grammar does pycparser follow?</a></li> <li><a class="reference internal" href="#how-is-pycparser-licensed" id="id7">1.5 How is pycparser licensed?</a></li> <li><a class="reference internal" href="#contact-details" id="id8">1.6 Contact details</a></li> </ul> </li> <li><a class="reference internal" href="#installing" id="id9">2 Installing</a><ul class="auto-toc"> <li><a class="reference internal" href="#prerequisites" id="id10">2.1 Prerequisites</a></li> <li><a class="reference internal" href="#installation-process" id="id11">2.2 Installation process</a></li> <li><a class="reference internal" href="#known-problems" id="id12">2.3 Known problems</a></li> </ul> </li> <li><a class="reference internal" href="#using" id="id13">3 Using</a><ul class="auto-toc"> <li><a class="reference internal" href="#interaction-with-the-c-preprocessor" id="id14">3.1 Interaction with the C preprocessor</a></li> <li><a class="reference internal" href="#what-about-the-standard-c-library-headers" id="id15">3.2 What about the standard C library headers?</a></li> <li><a class="reference internal" href="#basic-usage" id="id16">3.3 Basic usage</a></li> <li><a class="reference internal" href="#advanced-usage" id="id17">3.4 Advanced usage</a></li> </ul> </li> <li><a class="reference internal" href="#modifying" id="id18">4 Modifying</a></li> <li><a class="reference internal" href="#package-contents" id="id19">5 Package contents</a></li> <li><a class="reference internal" href="#contributors" id="id20">6 Contributors</a></li> <li><a class="reference internal" href="#changelog" id="id21">7 Changelog</a></li> </ul> </div> <div class="section" id="introduction"> <h1>1 Introduction</h1> <div class="section" id="what-is-pycparser"> <h2>1.1 What is pycparser?</h2> <p><tt class="docutils literal">pycparser</tt> is a parser for the C language, written in pure Python. It is a module designed to be easily integrated into applications that need to parse C source code.</p> </div> <div class="section" id="what-is-it-good-for"> <h2>1.2 What is it good for?</h2> <p>Anything that needs C code to be parsed. The following are some uses for <tt class="docutils literal">pycparser</tt>, taken from real user reports:</p> <ul class="simple"> <li>C code obfuscator</li> <li>Front-end for various specialized C compilers</li> <li>Static code checker</li> <li>Automatic unit-test discovery</li> <li>Adding specialized extensions to the C language</li> </ul> <p><tt class="docutils literal">pycparser</tt> is unique in the sense that it's written in pure Python - a very high level language that's easy to experiment with and tweak. To people familiar with Lex and Yacc, <tt class="docutils literal">pycparser</tt>'s code will be simple to understand.</p> </div> <div class="section" id="which-version-of-c-does-pycparser-support"> <h2>1.3 Which version of C does pycparser support?</h2> <p><tt class="docutils literal">pycparser</tt> aims to support the full C99 language (according to the standard ISO/IEC 9899). This is a new feature in the version 2.x series - earlier versions only supported C89. For more information on the change, read <a class="reference external" href="http://code.google.com/p/pycparser/wiki/C99support">this wiki page</a>.</p> <p><tt class="docutils literal">pycparser</tt> doesn't support any GCC extensions.</p> </div> <div class="section" id="what-grammar-does-pycparser-follow"> <h2>1.4 What grammar does pycparser follow?</h2> <p><tt class="docutils literal">pycparser</tt> very closely follows the C grammar provided in the end of the C99 standard document</p> </div> <div class="section" id="how-is-pycparser-licensed"> <h2>1.5 How is pycparser licensed?</h2> <p><a class="reference external" href="http://www.opensource.org/licenses/bsd-license.php">New BSD License</a></p> </div> <div class="section" id="contact-details"> <h2>1.6 Contact details</h2> <p>Drop me an email to <a class="reference external" href="mailto:eliben@gmail.com">eliben@gmail.com</a> for any questions regarding <tt class="docutils literal">pycparser</tt>. For reporting problems with <tt class="docutils literal">pycparser</tt> or submitting feature requests, the best way is to open an issue on the <a class="reference external" href="http://code.google.com/p/pycparser/">pycparser page at Google Code</a>.</p> </div> </div> <div class="section" id="installing"> <h1>2 Installing</h1> <div class="section" id="prerequisites"> <h2>2.1 Prerequisites</h2> <ul class="simple"> <li><tt class="docutils literal">pycparser</tt> was tested on Python 2.6 and 3.2, on both Linux and Windows. It should work on any later version (in both the 2.x and 3.x lines) as well.</li> <li><tt class="docutils literal">pycparser</tt> uses the PLY module for the actual lexer and parser construction. Install PLY from <a class="reference external" href="http://www.dabeaz.com/ply/">its website</a>.</li> </ul> </div> <div class="section" id="installation-process"> <h2>2.2 Installation process</h2> <p>Installing <tt class="docutils literal">pycparser</tt> is very simple. Once you download it from its <a class="reference external" href="http://code.google.com/p/pycparser/">website</a> and unzip the package, you just have to execute the standard <tt class="docutils literal">python setup.py install</tt>. The setup script will then place the <tt class="docutils literal">pycparser</tt> module into <tt class="docutils literal"><span class="pre">site-packages</span></tt> in your Python's installation library.</p> <p>Alternatively, since <tt class="docutils literal">pycparser</tt> is listed in the <a class="reference external" href="http://pypi.python.org/pypi/pycparser">Python Package Index</a> (PyPI), you can install it using your favorite Python packaging/distribution tool, for example with:</p> <pre class="literal-block"> > pip install pycparser </pre> <p>It's recommended to run <tt class="docutils literal">_build_tables.py</tt> in the <tt class="docutils literal">pycparser</tt> code directory after installation to make sure the parsing tables of PLY are pre-generated. This can make your code run faster.</p> </div> <div class="section" id="known-problems"> <h2>2.3 Known problems</h2> <ul class="simple"> <li>Some users who've installed a new version of <tt class="docutils literal">pycparser</tt> over an existing version ran into a problem using the newly installed library. This has to do with parse tables staying around as <tt class="docutils literal">.pyc</tt> files from the older version. If you see unexplained errors from <tt class="docutils literal">pycparser</tt> after an upgrade, remove it (by deleting the <tt class="docutils literal">pycparser</tt> directory in your Python's <tt class="docutils literal"><span class="pre">site-packages</span></tt>, or wherever you installed it) and install again.</li> </ul> </div> </div> <div class="section" id="using"> <h1>3 Using</h1> <div class="section" id="interaction-with-the-c-preprocessor"> <h2>3.1 Interaction with the C preprocessor</h2> <p>In order to be compilable, C code must be preprocessed by the C preprocessor - <tt class="docutils literal">cpp</tt>. <tt class="docutils literal">cpp</tt> handles preprocessing directives like <tt class="docutils literal">#include</tt> and <tt class="docutils literal">#define</tt>, removes comments, and does other minor tasks that prepare the C code for compilation.</p> <p>For all but the most trivial snippets of C code, <tt class="docutils literal">pycparser</tt>, like a C compiler, must receive preprocessed C code in order to function correctly. If you import the top-level <tt class="docutils literal">parse_file</tt> function from the <tt class="docutils literal">pycparser</tt> package, it will interact with <tt class="docutils literal">cpp</tt> for you, as long as it's in your PATH, or you provide a path to it.</p> <p>On the vast majority of Linux systems, <tt class="docutils literal">cpp</tt> is installed and is in the PATH. If you're on Windows and don't have <tt class="docutils literal">cpp</tt> somewhere, you can use the one provided in the <tt class="docutils literal">utils</tt> directory in <tt class="docutils literal">pycparser</tt>'s distribution. This <tt class="docutils literal">cpp</tt> executable was compiled from the <a class="reference external" href="http://www.cs.princeton.edu/software/lcc/">LCC distribution</a>, and is provided under LCC's license terms.</p> </div> <div class="section" id="what-about-the-standard-c-library-headers"> <h2>3.2 What about the standard C library headers?</h2> <p>C code almost always includes various header files from the standard C library, like <tt class="docutils literal">stdio.h</tt>. While, with some effort, <tt class="docutils literal">pycparser</tt> can be made to parse the standard headers from any C compiler, it's much simpler to use the provided "fake" standard includes in <tt class="docutils literal">utils/fake_libc_include</tt>. These are standard C header files that contain only the bare necessities to allow valid parsing of the files that use them. As a bonus, since they're minimal, it can significantly improve the performance of parsing C files.</p> <p>See the <tt class="docutils literal">using_cpp_libc.py</tt> example for more details.</p> </div> <div class="section" id="basic-usage"> <h2>3.3 Basic usage</h2> <p>Take a look at the <tt class="docutils literal">examples</tt> directory of the distribution for a few examples of using <tt class="docutils literal">pycparser</tt>. These should be enough to get you started.</p> </div> <div class="section" id="advanced-usage"> <h2>3.4 Advanced usage</h2> <p>The public interface of <tt class="docutils literal">pycparser</tt> is well documented with comments in <tt class="docutils literal">pycparser/c_parser.py</tt>. For a detailed overview of the various AST nodes created by the parser, see <tt class="docutils literal">pycparser/_c_ast.cfg</tt>.</p> <p>There's also a <a class="reference external" href="http://code.google.com/p/pycparser/wiki/FAQ">FAQ available here</a>. In any case, you can always drop me an <a class="reference external" href="mailto:eliben@gmail.com">email</a> for help.</p> </div> </div> <div class="section" id="modifying"> <h1>4 Modifying</h1> <p>There are a few points to keep in mind when modifying <tt class="docutils literal">pycparser</tt>:</p> <ul class="simple"> <li>The code for <tt class="docutils literal">pycparser</tt>'s AST nodes is automatically generated from a configuration file - <tt class="docutils literal">_c_ast.cfg</tt>, by <tt class="docutils literal">_ast_gen.py</tt>. If you modify the AST configuration, make sure to re-generate the code.</li> <li>Make sure you understand the optimized mode of <tt class="docutils literal">pycparser</tt> - for that you must read the docstring in the constructor of the <tt class="docutils literal">CParser</tt> class. For development you should create the parser without optimizations, so that it will regenerate the Yacc and Lex tables when you change the grammar.</li> </ul> </div> <div class="section" id="package-contents"> <h1>5 Package contents</h1> <p>Once you unzip the <tt class="docutils literal">pycparser</tt> package, you'll see the following files and directories:</p> <dl class="docutils"> <dt>README.txt/html:</dt> <dd>This README file.</dd> <dt>setup.py:</dt> <dd>Installation script</dd> <dt>examples/:</dt> <dd>A directory with some examples of using <tt class="docutils literal">pycparser</tt></dd> <dt>pycparser/:</dt> <dd>The <tt class="docutils literal">pycparser</tt> module source code.</dd> <dt>tests/:</dt> <dd>Unit tests.</dd> <dt>utils/cpp.exe:</dt> <dd>A Windows executable of the C pre-processor suitable for working with pycparser</dd> <dt>utils/fake_libc_include:</dt> <dd>Minimal standard C library include files that should allow to parse any C code.</dd> <dt>utils/internal/:</dt> <dd>Internal utilities for my own use. You probably don't need them.</dd> </dl> </div> <div class="section" id="contributors"> <h1>6 Contributors</h1> <p>Some people have contributed to <tt class="docutils literal">pycparser</tt> by opening issues on bugs they've found and/or submitting patches. The list of contributors is at <a class="reference external" href="http://code.google.com/p/pycparser/wiki/Contributors">this pycparser Wiki page</a>.</p> </div> <div class="section" id="changelog"> <h1>7 Changelog</h1> <ul class="simple"> <li>Version 2.05 (16.10.2011)<ul> <li>Added support for the C99 <tt class="docutils literal">_Bool</tt> type and <tt class="docutils literal">stdbool.h</tt> header file</li> <li>Expanded <tt class="docutils literal">examples/explore_ast.py</tt> with more details on working with the AST</li> <li>Relaxed the rules on parsing unnamed struct members (helps parse <tt class="docutils literal">windows.h</tt>)</li> <li>Bug fixes:<ul> <li>Fixed spacing issue for some type declarations</li> <li>Issue 47: display empty statements (lone ';') correctly after parsing</li> </ul> </li> </ul> </li> <li>Version 2.04 (21.05.2011)<ul> <li>License changed from LGPL to BSD</li> <li>Bug fixes:<ul> <li>Issue 31: constraining the scope of typedef definitions</li> <li>Issues 33, 35: fixes for the c-to-c.py example</li> </ul> </li> <li>Added C99 integer types to fake headers</li> <li>Added unit tests for the c-to-c.py example</li> </ul> </li> <li>Version 2.03 (06.03.2011)<ul> <li>Bug fixes:<ul> <li>Issue 17: empty file-level declarations</li> <li>Issue 18: empty statements and declarations in functions</li> <li>Issue 19: anonymous structs & union fields</li> <li>Issue 23: fix coordinates of Cast nodes</li> </ul> </li> <li>New example added (<tt class="docutils literal"><span class="pre">examples/c-to-c.py</span></tt>) for translating ASTs generated by <tt class="docutils literal">pycparser</tt> back into C code.</li> <li><tt class="docutils literal">pycparser</tt> is now on PyPI (Python Package Index)</li> <li>Created <a class="reference external" href="http://code.google.com/p/pycparser/wiki/FAQ">FAQ</a> on the <tt class="docutils literal">pycparser</tt> project page</li> <li>Removed support for Python 2.5. <tt class="docutils literal">pycparser</tt> supports Python 2 from 2.6 and on, and Python 3.</li> </ul> </li> <li>Version 2.02 (10.12.2010)<ul> <li>The name of a <tt class="docutils literal">NamedInitializer</tt> node was turned into a sequence of nodes instead of an attribute, to make it discoverable by the AST node visitor.</li> <li>Documentation updates</li> </ul> </li> <li>Version 2.01 (04.12.2010)<ul> <li>Removed dependency on YAML. Parsing of the AST node configuration file is done with a simple parser.</li> <li>Fixed issue 12: installation problems</li> </ul> </li> <li>Version 2.00 (31.10.2010)<ul> <li>Support for C99 (read <a class="reference external" href="http://code.google.com/p/pycparser/wiki/C99support">this wiki page</a> for more information).</li> </ul> </li> <li>Version 1.08 (09.10.2010)<ul> <li>Bug fixes:<ul> <li>Correct handling of <tt class="docutils literal">do{} ... while</tt> statements in some cases</li> <li>Issues 6 & 7: Concatenation of string literals</li> <li>Issue 9: Support for unnamed bitfields in structs</li> </ul> </li> </ul> </li> <li>Version 1.07 (18.05.2010)<ul> <li>Python 3.1 compatibility: <tt class="docutils literal">pycparser</tt> was modified to run on Python 3.1 as well as 2.6</li> </ul> </li> <li>Version 1.06 (10.04.2010)<ul> <li>Bug fixes:<ul> <li>coord not propagated to FuncCall nodes</li> <li>lexing of the ^= token (XOREQUALS)</li> <li>parsing failed on some abstract declarator rules</li> </ul> </li> <li>Linux compatibility: fixed end-of-line and <tt class="docutils literal">cpp</tt> path issues to allow all tests and examples run on Linux</li> </ul> </li> <li>Version 1.05 (16.10.2009)<ul> <li>Fixed the <tt class="docutils literal">parse_file</tt> auxiliary function to handle multiple arguments to <tt class="docutils literal">cpp</tt> correctly</li> </ul> </li> <li>Version 1.04 (22.05.2009)<ul> <li>Added the <tt class="docutils literal">fake_libc_include</tt> directory to allow parsing of C code that uses standard C library include files without dependency on a real C library.</li> <li>Tested with Python 2.6 and PLY 3.2</li> </ul> </li> <li>Version 1.03 (31.01.2009)<ul> <li>Accept enumeration lists with a comma after the last item (C99 feature).</li> </ul> </li> <li>Version 1.02 (16.01.2009)<ul> <li>Fixed problem of parsing struct/enum/union names that were named similarly to previously defined <tt class="docutils literal">typedef</tt> types.</li> </ul> </li> <li>Version 1.01 (09.01.2009)<ul> <li>Fixed subprocess invocation in the helper function parse_file - now it's more portable</li> </ul> </li> <li>Version 1.0 (15.11.2008)<ul> <li>Initial release</li> <li>Support for ANSI C89</li> </ul> </li> </ul> </div> </div> </body> </html>