Specifications for better computing.
- Compact Float Format: Compresses floating point values.
- Compact Time Format: Compresses dates, times, and timestamps (to the nanosecond).
- Safe Text Encoding: Binary to text encoding for modern systems.
- Smalltime: Binary date & time format in 64 bits.
- Variable Bit Padding: Arbitrary length variable bit-level padding.
- Variable Length Quantity: Compresses unsigned integers.
- Varpad: Unlimited padding with an embedded length field.
- Concise Binary and Text Encoding: General purpose, compact representations of semi-structured hierarchical data, in binary and text formats.
- Streamux: A minimalist, asynchronous, multiplexing, request-response protocol.
Compact float format is an encoding scheme to store a decimal floating point value in as few bytes as possible for data transmission.
Compact float can store all of the kinds of values that the IEEE754 decimal types can, without data loss:
- ±0
- ±infinity
- Signaling and quiet NaNs
- IEEE754 Floating point
An encoding scheme to store dates, times, and timestamps, down to the nanosecond, in as few bytes as possible.
- Encodes a date into as few as 3 bytes.
- Encodes a time into as few as 4 bytes.
- Encodes a timestamp into as few as 5 bytes.
- Supports unlimited positive and negative year values.
- Supports time units down to the nanosecond.
- Supports leap years and leap seconds.
- Maintenance-free (no leap second tables to update).
- Efficient conversion to/from human readable fields (no multiplication or division).
- Time zones are location-based.
CBE and CTE are general purpose, compact representations of semi-structured hierarchical data, in binary and text formats. They support all common data types, including dates, URIs, and decimal floating point values.
Concise Encoding is the next step in the evolution of ad-hoc hierarchical data formats, aiming to address the shortfalls of the current generation:
- It is split into two formats: binary-based CBE and text-based CTE, which serve different but parallel purposes.
- 1:1 type compatiblility between the binary and text formats. Converting between CBE and CTE is transparent, allowing you to use the much smaller and energy efficient binary format for data interchange and storage, converting to/from text only when and where a human needs to be involved.
- Native support for the most commonly used data types. Concise Encoding aims to support 80% of data use cases natively.
- Support for metadata and comments.
- Completely redesigned from the ground up to balance user readability, encoded size, and codec complexity.
- The formats are fully specified, eliminating ambiguities and covering edge cases, thus facilitating compatibility between implementations and reducing complexity.
- Documents and specifications are versioned to support future expansion.
CBE:
CTE:
- C Implementation: TODO
- Go Implementation
- JSON
- XML
- BSON
- CBOR
- MessagePack
- Protobuf
Binary data encoding schemes that are safe to be passed through modern processing systems that expect human readable text, without requiring escaping.
- Safe for use in JSON, SGML formats, source code strings, without escaping
- Safe for use in path, query, and fragment components of URIs
- Safe for use in formatted documents
- Safe for use in legacy text processing systems
- Support for length fields
- Liberal whitespace rules
- No padding characters
- Safe for use in filenames on POSIX file systems
- Sortable in generic text sorting algorithms (such as file listings)
- Safe for use in filenames on Windows file systems
- Safe for use in all URI components without escaping
- Useful for human input situations such as activation codes
- Easily confusable characters & digits are interchangeable
- Uppercase and lowercase characters are interchangeable
- Safe16 Specification
- Safe32 Specification
- Safe64 Specification
- Safe80 Specification
- Safe85 Specification
- Base16
- Base32
- Base64
- Ascii85
Smalltime offers a simple and convenient way to encode date & time values into signed 64-bit integers that are comparable.
- Encodes a complete date & time into a 64-bit signed integer.
- Fields (including year) are compatible with ISO-8601.
- Maintenance-free (no leap second tables to update).
- Easily converts to human readable fields.
- Supports hundreds of thousands of years.
- Supports time units to the microsecond.
- Supports leap years and leap seconds.
- Values are in timezone zero by default.
- Encoded values are comparable.
A minimalist, asynchronous, multiplexing, request-response protocol.
A minimalist, asynchronous, multiplexing, request-response protocol.
Streamux is designed as a low level, point-to-point, bidirectional protocol for you to build a messaging layer on top of. It handles the nitty gritty things like initialization, multiplexing, asynchronous operation, and packetization of your messages.
The only additional components required are:
- A reliable communication transport (TCP, pipes, RS-232, etc)
- A message encoding format & marshaling scheme (for example CBE)
- Endpoints to receive the messages.
- Minimal Overhead (1 to 4 bytes per message chunk, depending on configuration)
- Multiplexing (multiple data streams can be sent across a single channel)
- Asynchronous (client is informed asynchronously upon completion or error)
- Interruptible (requests may be canceled)
- Floating roles (both peers can operate as client and server at the same time)
- Quick init mode for faster initialization
A scheme for encoding an arbitrary number of padding bits in a field or bit stream.
- Padding can be decoded from the left or right side.
- Padding can be 0-based (filled with
0bits) or 1-based (filled with1bits).
- TODO
Variable Length Quantity (VLQ) encoding is an unsigned integer compression scheme originally designed for the MIDI file format. This specification expands upon it slightly by allowing encoding from the "left" or "right" side of the quantity (the original MIDI version is "right" oriented).
- Left or right oriented encoding
- Supports progressive decoding
Varpad is a padding encoding scheme that allows unambiguous detection of the padding length without requiring an external field. It is similar to the scheme described in PKCS#7, but doesn't suffer from its limitations (PKCS#7 padding has a length limit of 255).
- Doesn't require a separate length field (the length is embedded in the padding itself).
- Padding length can be unambiguously detected by examining the first or last byte.
- No upper limit on padding length.