spack
spack is a way for you to serialize structured data.
Binary format
Float values are stored in little endian.
The various datatypes it supports as well as their encoding are as follows:
| Types | encoding |
|---|---|
| integer (encoded as signed leb-128) | 0x01 |
| float32 (ieee single precision) | 0x02 |
| float64 (ieee double precision) | 0x03 |
| byte | 0x04 |
| utf-8 string | 0x05 for start, followed by size in bytes |
| a non-zero array of a single type | 0x10 for start, followed by type, followed by num elems |
Arrays can only be one-dimensional. They cannot contain strings.
To create a byte array, pass :byte-array to spush.
The packet created by out will contain a header consisting of
three elements: the sha-256 hash of the packet (256 bits, 32
bytes), followed by the size in bytes of the type information
(leb-128), followed by the size in bytes of the data (leb-128),
followed by the type information, followed by the data.
API
The API is designed for languages which have runtime polymorphism of functions and/or powerful type systems and/or macro systems. It is implemented here for Common Lisp, but is easily reimplemented.
class: spack
A spack object only has one slot, a vector of spack-elem
objects. You don’t really use spack-elem s directly, but instead
interact with them through spush.
class: spack-elem
Has two slots: elem-type and val. You should’t ever use these
directly, and instead interface through it as shown below:
function: spush
Takes three arguments. The value to push, the type of the value,
and the spack to push it onto. Types can be one of: integer,
float32, float64, string (can either pass a utf-8 encoded
~’(unsigned-byte 8)~ buffer, or a string, which will be encoded as
utf-8), array, or byte-array.
function: out
Takes a single spack object, and will output the serialization
of that object.
function: parse
Takes the serialization produced by out, and turns it into the
object once more
macro: make-and-push
Takes a list of tuples (obj type), such as (2 :integer) and
create a spack object with these elements
macro: destructuring-elements
Like destructuring-bind, takes a list of symbols and a spack object
and then assigns to each symbol a corresponding value in the same way as
destructuring-bind does (it actually uses it under the hood)
Short Tutorial
SPACK> (setf *spack* (make-instance 'spack))
#<SPACK {1002D27413}>
SPACK> (spush 1234567890 :integer *spack*)
0
SPACK> (spush 12345.2 :float32 *spack*)
1
SPACK> (spush 12345.2d0 :float64 *spack*)
2
SPACK> (spush "testing" :string *spack*)
3
SPACK> (spush #(1 2 3 4) :array *spack*)
4
SPACK> (spush #(1 2 3 4) :byte-array *spack*)
5
SPACK> (spush #(1.5 2.3 3.2 4.7) :array *spack*)
6
SPACK> (defparameter *spack-out* (out *spack*))
*SPACK-OUT*
SPACK> *spack-out*
#(236 190 244 174 238 150 103 34 19 24 108 195 2 80 74 122 149 5 144 198 121
204 63 111 120 0 25 83 155 19 144 246 14 48 1 2 3 5 7 16 1 4 16 4 4 16 2 4
210 133 216 204 4 205 228 64 70 154 153 153 153 153 28 200 64 116 101 115 116
105 110 103 1 2 3 4 1 2 3 4 0 0 192 63 51 51 19 64 205 204 76 64 102 102 150
64)
SPACK> (parse *spack-out*)
#<SPACK {10020CD8B3}>
SPACK> (loop for i across (elements (parse *spack-out*)) do
(format t "Type: ~A~%Value: ~A~%" (elem-type i) (val i)))
Type: INTEGER
Value: 1234567890
Type: FLOAT32
Value: 12345.2
Type: FLOAT64
Value: 12345.2
Type: STRING
Value: #(116 101 115 116 105 110 103)
Type: (ARRAY INTEGER)
Value: #(1 2 3 4)
Type: (ARRAY BYTE)
Value: #(1 2 3 4)
Type: (ARRAY FLOAT32)
Value: #(1.5 2.3 3.2 4.7)
NIL
CL-USER> (spack:destructuring-elements (a b c d)
(spack:make-and-push (1 :integer)
(2 :byte)
(5.3 :float32)
("test" :string))
(list a b c d))
(1 2 5.3 "test")