EgorBo/SimdJsonSharp

Native Bindings is x10 Slower

Alois-xx opened this issue · 7 comments

Alois-xx commented

I have tried to add SimdJsonSharp to my serializer perfomrance test suite. See https://github.com/Alois-xx/SerializerTests

When you compile it and let it run on .NET Core Preview 7

D:\Source\git\SerializerTests\bin\Debug\netcoreapp3.0>SerializerTests.exe -test combined -serializer Utf8JsonSerializer,SimdJsonSharpSerializer,SimdJsonSharpSerializerN

Serializer      Objects "Time to serialize in s"        "Time to deserialize in s"      "Size in bytes" FileVersion     Framework       ProjectHome     DataFormat      FormatDetails   Supports Versioning
SimdJsonSharpSerializer 1000000 0.1310  0.116   35777803        1.5.0.0 .NET Core 3.0.0-preview7-27912-14       https://github.com/EgorBo/SimdJsonSharp based on https://github.com/lemire/simdjson     Text    Json    No
SimdJsonSharpSerializerN        1000000 0.1117  3.139   35777803        1.7.0.0 .NET Core 3.0.0-preview7-27912-14       https://github.com/EgorBo/SimdJsonSharp based on https://github.com/lemire/simdjson     Text    Json    No
Utf8JsonSerializer      1000000 0.1135  0.330   35777803        1.3.7   .NET Core 3.0.0-preview7-27912-14       https://github.com/neuecc/Utf8Json      Text    Json    Yes

I find the native version needs 3s vs 0.1s for the managed version.
grafik

When lookint at the data in PerfView I find most time is spent in paserJson. Why this is so costly I cannot tell but it looks wasteful. Can you take a look why this is so much slower? Am I using the library the wrong way?

EgorBo commented

@Alois-xx there is also a managed version which doesn't have pinvoke overhead (but has some C# vs C overhead).
Anyway simdjsonsharp is good for large files (>200kb)

Alois-xx commented

Yes I know about the managed version. Why should one use the native bindings if they are 10x times slower? Does it make sense to interop into such a slow solution if the interop overhead is so huge? The test with 1 million objects results in a ca. 30 MB Json file. I would have expected to get a much higher speedup compared to a normal serializer like Utf8Json.

EgorBo commented

@Alois-xx can you share some steps how to test that 30Mb json file, a repro project would be perfect.

Alois-xx commented

Clone https://github.com/Alois-xx/SerializerTests. Compile it and see the command line help. It is a serializer performance testing framework to make it easy to compare different serializers.

D:\Source\git\SerializerTests\bin\Release\netcoreapp3.0>SerializerTests.exe
SerializerTests is a serializer performance testing framework to evaluate and compare different serializers for .NET by Alois Kraus
SerializerTests [-Runs dd] -test [serialize, deserialize, combined, firstCall] [-reftracking] [-maxobj dd]
 -N 1,2,10000    Set the number of objects to de/serialize which is repeated -Runs times to get stable results.
 -Runs           Default is 5. The result is averaged where the first run is excluded from the average
 -test xx        xx can be serialize, deserialize, combined or firstcall to test a scenario for many different serializers
 -reftracking    If set a list with many identical references is serialized.
 -serializer xxx Execute the test only for a specific serializer with the name xxx where multiple ones can be used with ,
 -list           List all registered serializers
 -notouch        Do not touch the deserialized objects to test lazy deserialization
                 To execute deserialize you must first have called the serialize to generate serialized test data on disk to be read during deserialize
Examples
Compare protobuf against MessagePackSharp for serialize and deserialize performance
 SerializerTests -Runs 1 -test combined -serializer protobuf,MessagePackSharp
Test how serializers perform when reference tracking is enabled. Currently that are BinaryFormatter,Protobuf_net and DataContract
 Although Json.NET claim to have but it is not completely working.
 SerializerTests -Runs 1 -test combined -reftracking
Test SimdJsonSharpSerializer serializer with 3 million objects for serialize and deserialize.
 SerializerTests -test combined -N 3000000 -serializer SimdJsonSharpSerializer

I have added SimdJsonSharpSerializer with 3 million objects which creates a 100 MB Json file with Utf8JsonSerializer and deserialized it with your parsers.

For code see https://github.com/Alois-xx/SerializerTests/blob/master/Serializers/SimdJsonSharpSerializer.cs
and
https://github.com/Alois-xx/SerializerTests/blob/master/Serializers/SimdJsonSharpSerializerN.cs

Alois-xx commented

@EgorBo: Any update on this?

Alois-xx commented
Alois-xx commented