===========================================================
| |
| __ __ |
| ____ _____ _____________ / / ____ / /_ |
| / __ \/ __ / ___/ ___/ _ \/ / / __ `/ __ \ |
| / /_/ / /_/ / / (__ ) __/ /___/ /_/ / /_/ / |
| / ,___/\__,_/_/ /____/\___/_____/\__,_/_.___/ |
| /_/ |
| |
===========================================================
"parseLab" is the name of a tool/framework created by Lockheed Martin ATL which is designed to generate protocol parsers, fuzz protocol messages, and provide the services necessary for building custom protocol parser generators. The tool portion of parseLab is what allows the user to:
- generate parsers with native parser generators and parsing libraries
- generate valid messages according to a target protocol
- generate invalid messages according to a target protocol
The framework portion of parseLab is what allows the user to build custom protocol generators for parsing libraries that are not yet defined by parseLab.
Currently, parseLab ships with support for generating parsers using the Hammer C-library, and the Daedalus Data Definition Language. Note: The Daedalus generator module is very much a limited portion of the Daedalus language. The Hammer generator module can, and should, be used as an example for any custom generator modules that a user might want to create.
A guide for walking through the process of building a custom generator module can be found here.
The generator modules are really the heart of the parseLab system, without them, very little can be done with parseLab.
Each of the parseLab Generator Modules are to be derived from the ParselabGenerator class in order to properly function inside of the parseLab framework.
This ParselabGenerator class provides the interface functions that the rest of parseLab will leverage.
When creating a custom generator, the user will overload these interface functions with their own logic.
These generator modules are what hold all of the logic for generating the source code to make parsers for a target parsing library/language along with the logic for generating and running source code which passes in message data into the generated parser.
The interface functions that are defined in the generator modules have a very loose set of requirements. Since we are not capable of knowing all of the quirks of every target parsing language/library, we have simply written a system that provides the user with as much information necessary to go forth and build a parser code generator.
There are only a few dependencies for parselab-proper, while the dependencies for the natively supported generator modules are slightly more involved.
As a result, the install guide splits up the installation process based on the desired support.
For example, if you would like to only leverage the message generation aspect of parseLab, you don't need to install anything for the generator modules. Similarly, if you would like to only generate parsers for the Hammer C-library, you can just follow the parseLab and Hammer installation steps.
For a full understanding of parseLab, it is reccomended to follow the parseLab guides in the following sequence:
- Install Guide
- Successful installation allows for following the rest of the guides
- Creation of a Protocol Specification File with UDP
- Data is passed into parseLab through protocol specification files
- The UDP format is a simple format that is easily understood as a first attempt at creating a protocol specification file
- Using the UDP Protocol Specification to Generate UDP Messages
- With a completed UDP protocol specification, it is now possible to generate valid and invalid instances of UDP messages
- Creation of a Protocol Specification File with MAVLink
- Learn more about the functionality of the protocol specification file while creating a new specification for the MAVLink protocol
- Using Custom Structs in a Protocol Specification File
- Learn how to expand the capabaility of the protocol specification through the use of custom struct definitions.
- Creating a Custom Generator Module with Hammer's Python Bindings
- With the understanding of what parseLab is capable of with existing generator modules, creating one for a new parsing library is now possible
- Understanding the Inner-Mechanisms of parseLab and Protocol Specification Files
- When creating a new generator module, it is important to understand the data that is being passed into the modules, this document goes into more details about this effort
- Creating a Semantic Parser with parseLab
- Go through the steps to generate a parseLab syntax parser and learn how to convert it to support semantic parsing
For this section, it is assumed that the Install Guide was successfully completed for at least parseLab and Hammer.
# Start where parseLab was cloned
cd bin
# Ensure that the Hammer generator is passing all unit tests
./unit_tests.py --module hammer
# Generate a protocol directory for CAN with the files for Hammer
./setup.py --protocol ../protocols/can --module hammer
# Observe the new directory and its contents
ls ../protocols/can
# Copy the protocol specification from can example in the examples/ directory
cp ../examples/can/protocol.json ../protocols/can/protocol.json
# Remove the mission specification from the new can protocol directory
rm ../protocols/can/mission.json
# Generate a CAN parser using Hammer
./generate_parser.py --protocol ../protocols/can --module hammer
# Observe the generated code
less ../protocols/can/hammer/out/src/parser.c
# Generate a testcase consisting of 10 valid CAN messages
./generate_testcase.py --protocol ../protocols/can --msg_count 10 --valid --name myTest
# Observe the generated CAN messages
less ../protocols/can/testcases/myTest/myTest.xxd
# Generate test code for the new testcase
./generate_test.py --protocol ../protocols/can --module hammer --testcase ../protocols/can/testcases/myTest
# Observe the generated test code
less ../protocols/can/hammer/out/tests/myTest/src/test.c
# Run the generated parser against the generated testcase code
./run_test.py --protocol ../protocols/can --module hammer --testcase ../protocols/can/testcases/myTestIf everything went well, all 10 test messages should have successfully been "correctly accepted by parser".
ACKNOWLEDGMENT This material is based upon work supported by the Defense Advanced Research Projects Agency (DARPA) under Contract No. HR0011-19-C-0077. Any opinions, findings and conclusions or recommendations expressed in this material arethose of the author(s) and do not necessarily reflect the views of the Defense Advanced Research Projects Agency (DARPA).
Distribution Statement "A" (Approved for Public Release, Distribution Unlimited).