/kissrpc

Ultra high performance RPC

Primary LanguageDApache License 2.0Apache-2.0

kiss-rpc-flatbuffer features:

  1. Lightweight and easy to use. There are two ways to support IDL and manually write protocols. Analog function call, more in line with the RPC remote call logic, simple, transparent.

  2. Easy to change, easy to use, existing code can be used directly

  3. The data format supports downward compatibility and uses the flatbuffer protocol, with better compatibility and faster speed.

  4. Support multi valued return feature, support timeout mechanism, analog grpc, thrift, Dubbo fast several times or even dozens of times.

  5. Support snappy compression algorithm, compression speed, superior performance.

  6. Support pipeline data compression, dynamic data compression, request data compression, flexible use of a wide range of scenarios.

development environment

IDL introduction and instructions for use:

Setup:

  1. install flatbuffer for dlang (https://github.com/huntlabs/google-flatbuffers)
  2. install google snappy
  3. dub Compiler

About the compression mode used by kiss-rpc

  • data compression: using Google snappy compression technology to support forced compression and dynamic compression, flexible compression methods can be applied to a variety of scenarios.

  • dynamic compression technique: when data packets larger than 200 bytes or set the threshold, compressed packet, otherwise not compressed packets, use and help to improve the performance of space, response will be based on whether the request needs dynamic data compression.

  • single request compression: compression of a single request request, and response compression based on the way data packets are pressed.

  • pipeline compression: packet compression can be performed on the specified pipeline.

enable RPC debug logs

  1. show debug logs info(add dub config, versions:["RpcDebug"])

  2. close all logs info(add dub config, versions:["UltraHigh"])

  3. write logs to file, call tunction:setOutputLogPath("./info.log")

IDL Example

  1. client test: "IDL-Example/client/"

  2. server test: "IDL-Example/server/"

  3. idl protocol: "IDL-Example/kiss-idl"

Performance test code

  1. 100W QPS synchronous testing takes time: 20 seconds, average 5w QPS per second

  2. 100W QPS asynchronous testing takes 5 seconds, with an average of 20W QPS per second

  3. 1000 concurrent, 100wQPS asynchronous testing takes time: 5 seconds, average QPS:20W per second

  • server test code:"test/server"

  • client test code: "test/client"

What is IDL?

1. IDL is the kiss RPC interface code generation protocol, the preparation of IDL protocol, you can generate the corresponding server and client common RPC code call interface.
2. Standardize the unity, interface unification, simple to use.

IDL usage

1. [idl file path]    [output file name]    [output file path，default current dir]  E."/root/home/kiss-rpc.idl"	kiss-rpc	"/root/home/rpc/"
2. module name output, module path is ".": E."/root/home/kiss-rpc.idl"	module.test.kiss-rpc	 "/root/home/rpc/"	
3. At the same time output client and server file code, only need to copy to the corresponding client and server directory.
4. Message type names must be uppercase first, type members must be marked with serial numbers, otherwise they cannot be compiled 	
5. The function parameter list can only be one, or else it cannot be compiled

IDL Supported type

IDL D lang
bool bool
byte byte
ubyte ubyte
short short
ushort ushort
int int
uint uint
long long
ulong ulong
float float
double double
char char
string string
[] DynamicArrayList
@message struct

IDL code usage

1. IDL code usage, server-side as long as the server directory to fill the service file function interface code.

2. The client only needs to call the function of the interface of the service file under the client directory.

Kiss-rpc IDL writing examples

	//kiss rpc idl demo

	@message:UserInfo
	{
		string phone:3;
		string userName:1;
		int age:2;
		double wiget:4;
		
		string[] addressList:5;
	}

	@message:Contacts
	{
		int number:1;
		UserInfo[] userInfoList:2;		
	}

	@message:User
	{
		string name:1;
	}


	@service:AddressBook	//class interface
	{
		Contacts getContactList(User userName);
	}

Client remote call(demo path:IDL-Example/client/source/app.d):

IDL generates both synchronous and asynchronous interfaces, and asynchronous interfaces are all parameter callbacks.
  • import hander files
import kissrpc.IDL.kissidlService;
import kissrpc.IDL.kissidlMessage;
import kissrpc.Unit;
  • Client synchronous invocation
			try{
				auto c = addressBookService.getContactList(name);
				foreach(v; c.userInfoList)
				{
					writefln("sync number:%s, name:%s, phone:%s, age:%s", c.number, v.name, v.widget, v.age);
					
				}

			}catch(Exception e)
			{
				writeln(e.msg);
			}
  • Client asynchronous call
			try{

				addressBookService.getContactList(name, delegate(Contacts c){
						
						foreach(v; c.userInfoList)
						{
							writefln("async number:%s, name:%s, phone:%s, age:%s", c.number, v.name, v.widget, v.age);
						}
					}
				);
			}catch(Exception e)
			{
				writeln(e.msg);
			}
Call in compression (support for dynamic compression and forced compression)
  • Bind socket mode compression
RpcClient.setSocketCompress(RPC_PACKAGE_COMPRESS_TYPE.RPCT_DYNAMIC); //The dynamic compression mode defaults to more than 200 bytes of compression

RpcClient.setSocketCompress(RPC_PACKAGE_COMPRESS_TYPE.RPCT_COMPRESS); //forced compression method
  • Single request compression, synchronous call, forced compression
			//use compress demo
			try{
				auto c = addressBookService.getContactList(name, RPC_PACKAGE_COMPRESS_TYPE.RPCT_COMPRESS);

				foreach(v; c.userInfoList)
				{
					writefln("compress test: sync number:%s, name:%s, phone:%s, age:%s", c.number, v.name, v.widget, v.age);
				}
				
			}catch(Exception e)
			{
				writeln(e.msg);
			}
  • Single request compression, asynchronous call, 100 bytes of dynamic compression, request timeout 30 seconds
			//use dynamic compress and set request timeout


			try{
				RPC_PACKAGE_COMPRESS_DYNAMIC_VALUE = 100; //reset compress dynamaic value 100 byte, default:200 byte

				addressBookService.getContactList(name, delegate(Contacts c){
						
						foreach(v; c.userInfoList)
						{
							writefln("dynamic compress test: sync number:%s, name:%s, phone:%s, age:%s", c.number, v.name, v.widget, v.age);
						}

					}, RPC_PACKAGE_COMPRESS_TYPE.RPCT_DYNAMIC, 30
				);

			}catch(Exception e)
			{
				writeln(e.msg);
			}

Server service file code rpc_address_book_service(file path:IDL-Example/server/source/IDL/kissidlInterface.d):

The server interface can handle asynchronous events.
  • RpcAddressBookService.getContactList
	Contacts getContactList(AccountName accountName){

		Contacts contactsRet;
		//input service code for Contacts class
		contactsRet.number = accountName.count;

		for(int i = 0; i < 10; i++)
		{
			UserInfo userInfo;
			userInfo.age = 18+i;
			userInfo.name = accountName.name ~ to!string(i);
			userInfo.widget = 120+i;
			contactsRet.userInfoList ~= userInfo;
		}

		return contactsRet;
	}