asn1tools: A Python repository from qwertyquerty

About

A Python package for ASN.1 parsing, encoding and decoding.

This project is under development and does only support a subset of the ASN.1 specification syntax.

Supported codecs:

Basic Encoding Rules (BER)
Distinguished Encoding Rules (DER)
Generic String Encoding Rules (GSER)
JSON Encoding Rules (JER)
Basic Octet Encoding Rules (OER)
Aligned Packed Encoding Rules (PER)
Unaligned Packed Encoding Rules (UPER)
XML Encoding Rules (XER)

Miscellaneous features:

C source code generator for OER and UPER (with some limitations).

Project homepage: https://github.com/eerimoq/asn1tools

Documentation: http://asn1tools.readthedocs.org/en/latest

Known limitations

The CLASS keyword (X.681) and its friends are not yet supported.
Parametrization (X.683) is not yet supported.
The EMBEDDED PDV type is not yet supported.
The ANY and ANY DEFINED BY types are not supported. They were removed from the ASN.1 standard 1994.
WITH COMPONENT and WITH COMPONENTS constraints are ignored, except for OER REAL.
The DURATION type is not yet supported.

Installation

pip install asn1tools

Example Usage

This is an example ASN.1 specification defining the messages of a fictitious Foo protocol (based on the FooProtocol on Wikipedia).

Foo DEFINITIONS ::= BEGIN

    Question ::= SEQUENCE {
        id        INTEGER,
        question  IA5String
    }

    Answer ::= SEQUENCE {
        id        INTEGER,
        answer    BOOLEAN
    }

END

Scripting

Compile the ASN.1 specification, and encode and decode a question using the default codec (BER).

>>> import asn1tools
>>> foo = asn1tools.compile_files('tests/files/foo.asn')
>>> encoded = foo.encode('Question', {'id': 1, 'question': 'Is 1+1=3?'})
>>> encoded
bytearray(b'0\x0e\x02\x01\x01\x16\x09Is 1+1=3?')
>>> foo.decode('Question', encoded)
{'id': 1, 'question': 'Is 1+1=3?'}

The same ASN.1 specification, but using the PER codec.

>>> import asn1tools
>>> foo = asn1tools.compile_files('tests/files/foo.asn', 'per')
>>> encoded = foo.encode('Question', {'id': 1, 'question': 'Is 1+1=3?'})
>>> encoded
bytearray(b'\x01\x01\tIs 1+1=3?')
>>> foo.decode('Question', encoded)
{'id': 1, 'question': 'Is 1+1=3?'}

See the examples folder for additional examples.

Command line tool

The shell subcommand

Use the command line shell to convert data between given formats. The default input codec is BER and output codec is GSER (produces human readable text).

> asn1tools shell

Welcome to the asn1tools shell!

$ help
Commands:
  compile
  convert
  exit
  help
$ compile tests/files/foo.asn
$ convert Question 300e0201011609497320312b313d333f
question Question ::= {
    id 1,
    question "Is 1+1=3?"
}
$ compile --output-codec xer tests/files/foo.asn
$ convert Question 300e0201011609497320312b313d333f
<Question>
    <id>1</id>
    <question>Is 1+1=3?</question>
</Question>
$ compile -o uper tests/files/foo.asn
$ convert Question 300e0201011609497320312b313d333f
01010993cd03156c5eb37e
$ exit
>

The convert subcommand

Convert given encoded Question from BER to GSER (produces human readable text).

> asn1tools convert tests/files/foo.asn Question 300e0201011609497320312b313d333f
question Question ::= {
    id 1,
    question "Is 1+1=3?"
}
>

Convert given encoded Question from UPER to XER (xml).

> asn1tools convert -i uper -o xer tests/files/foo.asn Question 01010993cd03156c5eb37e
<Question>
    <id>1</id>
    <question>Is 1+1=3?</question>
</Question>
>

Convert given encoded Question from UPER to JER (json).

> asn1tools convert -i uper -o jer tests/files/foo.asn Question 01010993cd03156c5eb37e
{
    "id": 1,
    "question": "Is 1+1=3?"
}
>

Continuously convert encoded Questions read from standard input. Any line that cannot be converted is printed as is, in this example the dates.

> cat encoded.txt
2018-02-24 11:22:09
300e0201011609497320312b313d333f
2018-02-24 11:24:15
300e0201021609497320322b323d353f
> cat encoded.txt | asn1tools convert tests/files/foo.asn Question -
2018-02-24 11:22:09
question Question ::= {
    id 1,
    question "Is 1+1=3?"
}
2018-02-24 11:24:15
question Question ::= {
    id 2,
    question "Is 2+2=5?"
}
>

The convert subcommand with a cache

Convert given encoded PCCH-Message from UPER to GSER with the --cache-dir option set to my_cache. Using a cache significantly reduces the command execution time after the first call.

> time asn1tools convert --cache-dir my_cache -i uper tests/files/3gpp/rrc_8_6_0.asn PCCH-Message 28
pcch-message PCCH-Message ::= {
    message c1 : paging : {
        systemInfoModification true,
        nonCriticalExtension {
        }
    }
}

real    0m2.090s
user    0m1.977s
sys     0m0.032s
> time asn1tools convert --cache-dir my_cache -i uper tests/files/3gpp/rrc_8_6_0.asn PCCH-Message 28
pcch-message PCCH-Message ::= {
    message c1 : paging : {
        systemInfoModification true,
        nonCriticalExtension {
        }
    }
}

real    0m0.276s
user    0m0.197s
sys     0m0.026s
>

The parse subcommand

Parse given ASN.1 specification and write it as a Python dictionary to given file. Use the created file to convert given encoded Question from BER to GSER (produces human readable text). The conversion is significantly faster than passing .asn-file(s) to the convert subcommand, especially for larger ASN.1 specifications.

> asn1tools parse tests/files/foo.asn foo.py
> asn1tools convert foo.py Question 300e0201011609497320312b313d333f
question Question ::= {
    id 1,
    question "Is 1+1=3?"
}
>

The generate C source subcommand

Generate OER or UPER C source code from an ASN.1 specification.

No dynamic memory is used in the generated code. To achieve this all types in the ASN.1 specification must have a known maximum size, i.e. INTEGER (0..7), OCTET STRING (SIZE(12)), etc.

Below is an example generating OER C source code from tests/files/c_source/c_source.asn.

> asn1tools generate_c_source --namespace oer tests/files/c_source/c_source.asn
Successfully generated oer.h and oer.c.

The same as above, but generate UPER C source code instead of OER.

> asn1tools generate_c_source --codec uper --namespace uper tests/files/c_source/c_source.asn
Successfully generated uper.h and uper.c.

The same as the first example, but also generate fuzz testing C source code for libFuzzer.

> asn1tools generate_c_source --namespace oer --generate-fuzzer tests/files/c_source/c_source.asn
Successfully generated oer.h and oer.c.
Successfully generated oer_fuzzer.c and oer_fuzzer.mk.

Run "make -f oer_fuzzer.mk" to build and run the fuzzer. Requires a
recent version of clang.

See oer.h, oer.c, uper.h, uper.c, oer_fuzzer.c and oer_fuzzer.mk for the contents of the generated files.

Limitations by design:

Only the types BOOLEAN, INTEGER, NULL, OCTET STRING, BIT STRING, ENUMERATED, SEQUENCE, SEQUENCE OF, and CHOICE are supported. The OER generator also supports REAL.
All types must have a known maximum size, i.e. INTEGER (0..7), OCTET STRING (SIZE(12)).
INTEGER must be 64 bits or less.
REAL must be IEEE 754 binary32 or binary64. binary32 is generated as float and binary64 as double.
Recursive types are not supported.

Known limitations:

Extension additions (...) are only supported in the OER generator. See compact_extensions_uper for how to make UPER CHOICE and SEQUENCE extendable without using ....
Named numbers in ENUMERATED are not yet supported.

Other OER and/or UPER C code generators:

See the benchmark example for a comparison of asn1c, asn1scc and asn1tools.

Contributing

Fork the repository.
Install prerequisites.
```
pip install -r requirements.txt
```
Implement the new feature or bug fix.
Implement test case(s) to ensure that future changes do not break legacy.
Run the tests.
```
make test
```
Create a pull request.

Specifications

ASN.1 specifications released by ITU and IETF.

qwertyquerty/asn1tools