English | 中文
A very fast dynamic Thrift serializer & deserializer based on just-in-time compilation.
WARNING: This is still in very early alpha stage, use with care !!
Traditional Thrift serializer and deserializer are based on generated code which is no longer needed since we can use JIT compilation to dynamically generate machine code.
Thanks to JIT compilation, Frugal can generate better machine code than Go language compiler. In multi-core scenarios, Frugal's performance is about 5 times higher than that of traditional serializer and deserializer.
name old time/op new time/op delta
MarshalAllSize_Parallel/small-16 78.8ns ± 0% 14.9ns ± 0% -81.10%
MarshalAllSize_Parallel/medium-16 1.34µs ± 0% 0.32µs ± 0% -76.32%
MarshalAllSize_Parallel/large-16 37.7µs ± 0% 9.4µs ± 0% -75.02%
UnmarshalAllSize_Parallel/small-16 368ns ± 0% 30ns ± 0% -91.90%
UnmarshalAllSize_Parallel/medium-16 11.9µs ± 0% 0.8µs ± 0% -92.98%
UnmarshalAllSize_Parallel/large-16 233µs ± 0% 21µs ± 0% -90.99%
name old speed new speed delta
MarshalAllSize_Parallel/small-16 7.31GB/s ± 0% 38.65GB/s ± 0% +428.84%
MarshalAllSize_Parallel/medium-16 12.9GB/s ± 0% 54.7GB/s ± 0% +322.10%
MarshalAllSize_Parallel/large-16 11.7GB/s ± 0% 46.8GB/s ± 0% +300.26%
UnmarshalAllSize_Parallel/small-16 1.56GB/s ± 0% 19.31GB/s ± 0% +1134.41%
UnmarshalAllSize_Parallel/medium-16 1.46GB/s ± 0% 20.80GB/s ± 0% +1324.55%
UnmarshalAllSize_Parallel/large-16 1.89GB/s ± 0% 20.98GB/s ± 0% +1009.73%
name old alloc/op new alloc/op delta
MarshalAllSize_Parallel/small-16 112B ± 0% 0B -100.00%
MarshalAllSize_Parallel/medium-16 112B ± 0% 0B -100.00%
MarshalAllSize_Parallel/large-16 779B ± 0% 57B ± 0% -92.68%
UnmarshalAllSize_Parallel/small-16 1.31kB ± 0% 0.10kB ± 0% -92.76%
UnmarshalAllSize_Parallel/medium-16 448B ± 0% 3022B ± 0% +574.55%
UnmarshalAllSize_Parallel/large-16 1.13MB ± 0% 0.07MB ± 0% -93.54%
name old allocs/op new allocs/op delta
MarshalAllSize_Parallel/small-16 1.00 ± 0% 0.00 -100.00%
MarshalAllSize_Parallel/medium-16 1.00 ± 0% 0.00 -100.00%
MarshalAllSize_Parallel/large-16 1.00 ± 0% 0.00 -100.00%
UnmarshalAllSize_Parallel/small-16 6.00 ± 0% 1.00 ± 0% -83.33%
UnmarshalAllSize_Parallel/medium-16 6.00 ± 0% 30.00 ± 0% +400.00%
UnmarshalAllSize_Parallel/large-16 4.80k ± 0% 0.76k ± 0% -84.10%
Use Frugal as Kitex serializer and deserializer
No more massive serialization and deserialization code, leads to a more tidy project. No more meaningless diff of generated code in code review.
Serialized and Deserialize struct generated by Thriftgo
If you have a Thrift file, and all you need is using Frugal to do serialization and deserialization. You can use thriftgo to generate Go struct, then you can use Frugal.
If you don't want any Thrift files, and you want serialize or deserialize a customized Go struct. You can add some struct field tag to the Go struct, then you can use Frugal.
go get github.com/cloudwego/kitex@latestExample:
kitex -thrift frugal_tag -service a.b.c my.thriftIf you don't need codec code, you can use -thrift template=slim option.
kitex -thrift frugal_tag,template=slim -service a.b.c my.thriftClient example:
package client
import (
"context"
"github.com/cloudwego/kitex/client"
"example.com/kitex_test/client/kitex_gen/a/b/c/echo"
"github.com/cloudwego/kitex/pkg/remote/codec/thrift"
)
func Echo() {
code := thrift.NewThriftCodecWithConfig(thrift.FastRead | thrift.FastWrite | thrift.FrugalRead | thrift.FrugalWrite)
cli := echo.MustNewClient("a.b.c", client.WithPayloadCodec(codec))
...
}Server example:
package main
import (
"log"
"code.byted.org/kite/kitex/server"
c "example.com/kitex_test/kitex_gen/a/b/c/echo"
"github.com/cloudwego/kitex/pkg/remote/codec/thrift"
)
func main() {
code := thrift.NewThriftCodecWithConfig(thrift.FastRead | thrift.FastWrite | thrift.FrugalRead | thrift.FrugalWrite)
svr := c.NewServer(new(EchoImpl), server.WithPayloadCodec(code))
err := svr.Run()
if err != nil {
log.Println(err.Error())
}
}We can define a struct in Thrift file like below:
my.thrift:
struct MyStruct {
1: string msg
2: i64 code
}Now we have thrift file, we can use Thriftgo with frugal_tag option to generate Go code.
Example:
thriftgo -r -o thrift -g go:frugal_tag,package_prefix=example.com/kitex_test/thrift my.thriftIf you don't need codec code, you can use template=slim option
thriftgo -r -o thrift -g go:frugal_tag,template=slim,package_prefix=example.com/kitex_test/thrift my.thriftNow we can use Frugal to serialize or deserialize the struct defined in thrift file.
Example:
package main
import (
"github.com/cloudwego/frugal"
"example.com/kitex_test/thrift"
)
func main() {
ms := &thrift.MyStruct{
Msg: "my message",
Code: 1024,
}
...
buf := make([]byte, frugal.EncodedSize(ms))
frugal.EncodeObject(buf, nil, ms)
...
got := &thrift.MyStruct{}
frugal.DecodeObject(buf, got)
...
}We can define a struct like this:
type MyStruct struct {
Msg string
Code int64
Numbers []int64
}Frugal tag is like frugal:"1,default,string", 1 is field ID, default is field requiredness, string is field type. Field ID and requiredness is always required, but field type is only required for list, set and enum.
You can add Frugal tag to MyStruct like below:
type MyStruct struct {
Msg string `frugal:"1,default"`
Code int64 `frugal:"2,default"`
Numbers []int64 `frugal:"3,default,list<i64>"`
}All types example:
type MyEnum int64
type Example struct {
MyOptBool *bool `frugal:"1,optional"`
MyReqBool bool `frugal:"2,required"`
MyOptByte *int8 `frugal:"3,optional"`
MyReqByte int8 `frugal:"4,required"`
MyOptI16 *int16 `frugal:"5,optional"`
MyReqI16 int16 `frugal:"6,required"`
MyOptI32 *int32 `frugal:"7,optional"`
MyReqI32 int32 `frugal:"8,required"`
MyOptI64 *int64 `frugal:"9,optional"`
MyReqI64 int64 `frugal:"10,required"`
MyOptString *string `frugal:"11,optional"`
MyReqString string `frugal:"12,required"`
MyOptBinary []byte `frugal:"13,optional"`
MyReqBinary []byte `frugal:"14,required"`
MyOptI64Set []int64 `frugal:"15,optional,set<i64>"`
MyReqI64Set []int64 `frugal:"16,required,set<i64>"`
MyOptI64List []int64 `frugal:"17,optional,list<i64>"`
MyReqI64List []int64 `frugal:"18,required,list<i64>"`
MyOptI64StringMap map[int64]string `frugal:"19,optional"`
MyReqI64StringMap map[int64]string `frugal:"20,required"`
MyOptEnum *MyEnum `frugal:"21,optional,i64"`
MyReqEnum *MyEnum `frugal:"22,optional,i64"`
}Example:
package main
import (
"github.com/cloudwego/frugal"
)
func main() {
ms := &thrift.MyStruct{
Msg: "my message",
Code: 1024,
Numbers: []int64{0, 1, 2, 3, 4},
}
...
buf := make([]byte, frugal.EncodedSize(ms))
frugal.EncodeObject(buf, nil, ms)
...
got := &thrift.MyStruct{}
frugal.DecodeObject(buf, got)
...
}