nimble install jsony
This library has no dependencies other than the Nim standard library.
Real world json is never what you want. It might have extra fields that you don't care about. It might have missing fields requiring default values. It might change or grow new fields at any moment. Json might use camelCase or snake_case. It might use inconsistent naming.
With this library you can use json your way, from the mess you get to the objects you want.
@[1, 2, 3].toJson() -> "[1,2,3]"
"[1,2,3]".fromJson(seq[int]) -> @[1, 2, 3]Currently, the Nim standard module first parses or serializes json into JsonNodes and then turns the JsonNodes into your objects with the to() macro. This is slower and creates unnecessary work for the garbage collector. This library skips the JsonNodes and creates the objects you want directly.
Another speed up comes from not using StringStream. Stream has a function dispatch overhead because it has to be able to switch between StringStream or FileStream at runtime. Jsony skips the overhead and just directly reads or writes to memory buffers.
Another speed up comes from parsing and readings its own numbers directly from memory buffer. This allows it to bypass string allocations that parseInt or $ create.
name ............................... min time avg time std dv times
treeform/jsony ..................... 1.317 ms 1.365 ms ±0.054 x100
status-im/nim-json-serialization ... 2.043 ms 3.448 ms ±0.746 x100
planetis-m/eminim .................. 5.951 ms 9.305 ms ±3.210 x100
disruptek/jason .................... 6.858 ms 7.043 ms ±0.125 x100
nim std/json ....................... 8.222 ms 8.510 ms ±0.123 x100
name ............................... min time avg time std dv times
treeform/jsony ..................... 4.134 ms 4.196 ms ±0.052 x100
status-im/nim-json-serialization ... 7.119 ms 14.276 ms ±2.033 x100
planetis-m/eminim .................. 7.761 ms 8.001 ms ±0.277 x100
nim std/json ...................... 14.326 ms 14.473 ms ±0.113 x100
Note: If you find a faster nim json parser or serializer let me know!
- numbers and strings
- seq and arrays
- objects and ref objects
- options
- enums
- tuples
- characters
HashTables andOrderedTablesHashSets andOrderedSets- json nodes
- and
parseHook()enables you to parse any type!
Extra json fields are ignored and missing json fields keep their default values.
type Entry1 = object
color: string
var s = """{"extra":"foo"}"""
var v = s.fromJson(Entry1)
doAssert v.color == ""Nim usually uses camelCase for its variables, while a bunch of json in the wild uses snake_case. This library will convert snake_case to camelCase for you when reading json.
type Entry4 = object
colorBlend: string
var v = """{"colorBlend":"red"}""".fromJson(Entry4)
doAssert v.colorBlend == "red"
v = """{"color_blend":"red"}""".fromJson(Entry4)
doAssert v.colorBlend == "red"Hooks are a powerful concept that allows you to parse json "your way" and is the main idea behind jsony!
- Note: that hooks need to be exported to where you are parsing the json so that the parsing system can pick them up.
Sometimes the absence of a field means it should have a default value. Normally this would just be Nim's default value for the variable type but that isn't always what you want. With the newHook() you can initialize the object's defaults before the main parsing happens.
type
Foo5 = object
visible: string
id: string
proc newHook*(foo: var Foo5) =
# Populates the object before its fully deserialized.
foo.visible = "yes"
var s = """{"id":"123"}"""
var v = s.fromJson(Foo5)
doAssert v.id == "123"
doAssert v.visible == "yes"Sometimes we need run some code after the object is created. For example to set other values based on values that were set but are not part of the json data. Maybe to sanitize the object or convert older versions to new versions. Here I need to retain the original size as I will be messing with the object's regular size:
type Sizer = object
size: int
originalSize: int
proc postHook*(v: var Sizer) =
v.originalSize = v.size
var sizer = """{"size":10}""".fromJson(Sizer)
doAssert sizer.size == 10
doAssert sizer.originalSize == 10In the wild json enum names almost never match to Nim enum names which usually have a prefix. The enumHook*() allows you to rename the enums to your internal names.
type Color2 = enum
c2Red
c2Blue
c2Green
proc enumHook*(v: string): Color2 =
case v:
of "RED": c2Red
of "BLUE": c2Blue
of "GREEN": c2Green
else: c2Red
doAssert """ "RED" """.fromJson(Color2) == c2Red
doAssert """ "BLUE" """.fromJson(Color2) == c2Blue
doAssert """ "GREEN" """.fromJson(Color2) == c2GreenIn the wild json field names can be reserved words such as type, class, or array. With the renameHook*() you can rename fields to what you want.
type Node = ref object
kind: string
proc renameHook*(v: var Node, fieldName: var string) =
if fieldName == "type":
fieldName = "kind"
var node = """{"type":"root"}""".fromJson(Node)
doAssert node.kind == "root"Json can't store dates, so they are usually stored as strings. You can use
parseHook*() to override default parsing and parse DateTime as a string:
import jsony, times
proc parseHook*(s: string, i: var int, v: var DateTime) =
var str: string
parseHook(s, i, str)
v = parse(str, "yyyy-MM-dd hh:mm:ss")
var dt = """ "2020-01-01 00:00:00" """.fromJson(DateTime)Sometimes json gives you an object of entries with their id as keys, but you might want it as a sequence with ids inside the objects. You can handle this and many other scenarios with parseHook*():
type Entry = object
id: string
count: int
filled: int
let data = """{
"1": {"count":12, "filled": 11},
"2": {"count":66, "filled": 0},
"3": {"count":99, "filled": 99}
}"""
proc parseHook*(s: string, i: var int, v: var seq[Entry]) =
var table: Table[string, Entry]
parseHook(s, i, table)
for k, entry in table.mpairs:
entry.id = k
v.add(entry)
let s = data.fromJson(seq[Entry])Gives us:
@[
(id: "1", count: 12, filled: 11),
(id: "2", count: 66, filled: 0),
(id: "3", count: 99, filled: 99)
]"""
Just like reading custom data types you can also write data types with dumpHook*().
The dumpHook() will receive the incomplete string representation of a given serialization (here s).
You will need to add the serialization of your data type (here v) to that string.
type Fraction = object
numerator: int
denominator: int
proc dumpHook*(s: var string, v: Fraction) =
## Output fraction type as a string "x/y".
s.add '"'
s.add $v.numerator
s.add '/'
s.add $v.denominator
s.add '"'
var f = Fraction(numerator: 10, denominator: 13)
let s = f.toJson()Gives us:
"10/13"
If you want to skip some fields when serializing an object you can declare a skipHook*()
type
Conn = object
id: int
Foo = object
a: int
password: string
b: float
conn: Conn
proc skipHook*(T: typedesc[Foo], key: static string): bool =
key in ["password", "conn"]
var v = Foo(a:1, password: "12345", b:0.5, conn: Conn(id: 1))
let s = v.toJson()Gives us:
"{"a":1,"b":0.5}"
Sometimes you have some json, and you want to write it in a static way. There is a special function for that:
thing.toStaticJson()Make sure thing is a static or a const value, and you will get a compile time string with your JSON.
Case variant objects like this are fully supported:
type RefNode = ref object
case kind: NodeNumKind # The ``kind`` field is the discriminator.
of nkInt: intVal: int
of nkFloat: floatVal: floatThe discriminator does not have to come first, if they do come in the middle this library will scan the object, find the discriminator field, then rewind and parse the object normally.
Sometimes your json objects could contain arbitrary json structures, maybe event user defined, that could only be walked as json nodes. This library allows you to parse json-in-json were you parse some of the structure as real nim objects but leave some parts of it as Json Nodes to be walked later with code:
import jsony, json
type Entry = object
name: string
data: JsonNode
"""
{
"name":"json-in-json",
"data":{
"random-data":"here",
"number":123,
"number2":123.456,
"array":[1,2,3],
"active":true,
"null":null
}
}""".fromJson(Entry)Sometimes you don't need to parse the json, but just send it or store it in the database. You can speed this up by using RawJson type. What it does is prevents full parsing of that json tree and instead returns it is a RawJson (distinct string) type. You can then do anything you want with that. Store it in a database or pass it along to some other API. Or maybe parse it later again with jsony.
import jsony
type
Message = object
id: uint64
data: RawJson
let
messageData = """{"id":123,"data":{"page":"base64","arr":[1,2,3]}}"""
message = messageData.fromJson(Message)
# make sure raw json was not parsed
doAssert message.data.string == """{"page":"base64","arr":[1,2,3]}"""
# make sure that dumping raw json produces same result
doAssert message.toJson() == messageDataYou can also wait to parse the json later or maybe even with different types:
message.data.string.fromJson(DataPayload)