custom subjects
Closed this issue · 11 comments
I am currently evaluating nRPC for xbus.
One thing we rely a lot on is the permission system, and we have subjects that contains some ids so that only some clients may post on them, and the server does a wildcard sub so it gets all the requests from all the clients.
So I would like to specify, in the API, the exact subject to use, and it may contain some parameters that I would like the client to pass to the wrapping function, and the handler function on the server to get as a parameter.
Since protobuf allows custom options, I was thinking we could add a "nats_subject" method options.
The default would be function name, and it would always be prefixed by the service name and the package name if applicable.
A proto using this option would look like:
package demo;
service myservice {
rpc function1(string) returns (bool) {
option (nrpc.nats_subject) = "{clientid:string}.function1";
}
}In this case, the actual subject used by the client "c1" would be: "demo.myservice.c1.function1".
The wrapper function for the client would be something like:
func (myservice) function1(clientid string, request string) (bool, error) {
// ...
}On the server side, the handler interface would be:
type myserviceHandler interface {
function1(clientid string, request string) (bool, error)
}A side effect is that we would have 1 function = 1 subject, unlike the current implementation that seems to have one subject per service (unless I missed something).
Is it something that could make its way into nRPC if contributed ?
Having the subject same as the service name facilitates load balancing. We don't want to jeopardize/confuse this story.
Complying with this restriction, you can implement your feature by (1) including the client ID as a field in the incoming protobuf request message and (2) subscribing on the wildcard (outside of nRPC).
One possible enhancement that can be included is to let the handler know the actual subject name (in case the subscription was a wildcard one). It should be enough to stuff this into the Context object that is passed to the handler. A helper method (in nrpc package) to extract the value from the context would also be helpful. Something like:
// file tmpl.go
handlerCtx, cancel := context.WithValue(h.ctx, nrpc.SubjectKey, msg.Subject)
defer cancel()
// ..
innerResp, err := h.server.{{.GetName}}(handlerCtx, innerReq)package nrpc
func Subject(ctx context.Context) string {...}
type ContextKey string
var SubjectKey = ContextKey("subject")I do not think having 1 subject=1 service is a requirement for load balancing.
The same effect can be obtained with wildcard subscription, so that the actual function is the tail of the subject instead of an additional attribute of the message. Morevover it would allow partial implementation of a service, which is impossible with 1 subject=1 service.
For example, a service "hello" with 2 functions "world" and "me" would use the following subjects:
demo.hello.world and demo.hello.me, and could subscribe to demo.hello.*. The extraction of the actual function would be done on the subject, and would not necessitate decoding the message until just before calling the implementation. It could also subscribe separately to each subject without loosing anything load-balancing wise.
A big win with this approach is that the message is the request and only that, no additional information needed. Thanks to that simple clients that cannot use the whole auto-generated code and the nrpc lib would have an easier job using the API.
Aside from that, the solution (1) you propose would not work because in NATS permissions are entirely based on the subjects, and I rely on that, and only that. The whole subject list is designed to reflect the permission model and I do not want to implement an extra layer of security based on the message content (plus, having a client ID in the message would not guaranty anything as anyone can fake it).
Solution (2) could work, but I would appreciate something more included so that the whole actual protocol, including subject construction, is described in the .proto files. It allows code generator to provide automatically nice wrappers. Even with (2), I think having 1 subject=1 function would be better.
Another nice thing with using the subject to carry the function name is that we avoid the double encoding of the request.
#9 is just related to this issue as it is only a first step toward per-function custom subjects, which is a more complicated thing.
Unless you consider this feature (custom subjects) should not land in nrpc, could you reopen the issue ?
Ah yes, sorry about that, reopening.
Thinking again about the original use case, I think there should just be package-level or service-level option to generate client-identity-aware servers and clients. (I prefer to think of "clientid" as "API keys" -- would that be better terminology and make the concept easier to grasp?)
Exposing nats subject implies that users could use it as a general-purpose "extra parameters in the request" feature, which does not seem to be a good idea. It'd be tempting to add, say, an OpenTracing ID to it. Let the subject-encoding logic remain private to nrpc.
How about something like this:
// caller
c := NewGreeterClient(nc)
c.SetClientID("...")
c.SayHello(...)
// service
func (myservice) SayHello(ctx context.Context, req HelloRequest) (resp HelloReply, err error) {
clientID := nrpc.ClientID(ctx)
// ...
}Thoughts?
I agree that we could have package-level or service-level options, as it fits nicely for some of the use-case. But I still think it should be a custom-subject option, with maybe an optional subject-param, that would limit the number of parameters in subjects to 3 (package-level, service-level, method-level).
I disagree that subject-encoding logic should be completely private. It can be auto-handled in most case, but an easy access to it is important if we want to provide a way to formalize any (decent) NATS-based API. If we don't do that, NATS becomes only an implementation detail, and in my case it is a too limiting approach.
Maybe I should describe some of my use cases more precisely:
Package-level custom subjects.
The API of xbus is separated in at least 3 big parts, each one containing one or more services: Client/Actor, Control, and Core. The natural thing to represent them is to have 3 .proto files, with a different package name. But they should all have a common subject prefix. With a package-level custom subject, I can set the package subjects to "xbus.client", "xbus.control" and "xbus.core".
Package-level parameter.
The long term goal is to have several bus sharing the same NATS cluster. And of course the various services of each should not eavesdrop on the other xbus. The idea here is to add a bus id in the subject, and use that in the permission lists, so even the clients that have a wide access can be restricted with a permission like: "xbus.BUSID.>". In this case, the id in the subject is not a clientid.
Service-level custom subjects
Same as package-level custom subject, we can regroup services by their subject prefix, so permissions can be given on the whole group at once.
Service-level parameter
Some services are restricted depending on the client. The parameter would carry the client id. It is the case we discussed in the precedent messages.
Method level parameter
In some case I tweak the subject so the client can catch all a stream or only a selected part.
Example:
service receiver {
rpc ReceiveFirst(Fragment) returns (Ack);
rpc ReceiveNext(Fragment) returns (Ack) {
option (nrpc.subject_param) = "streamID";
}
}The fragments of a stream contains the streamID. Once the first fragment is received, the client can subscribe to the ad-hoc custom-subject for receiving the remaining of this stream, and only it. This pattern proves useful when several clients are queue-subscribed, but we want a same client instance to receive a complete stream.
But if the clients are state-less (which is better), they can do a queue+wildcard subscription so they receive any fragment from any stream.
This particular use-case could be implemented using 2 services, but then we would need 2 parameters at the service-level (one for the client id, the other for the streamID), and the final subject structure would be less natural (function name after the streamID).
Let me also put down a use-case that I wanted to achieve using nRPC. I think it makes sense to discuss that also in this thread, as that also probably involves manipulating the nats subject.
There is a component, like an agent, that provides a certain service (a SaltStack minion is a good example of the kind of agent I'm talking about).
I want to be able to send a request to all such agents within a deployment, and collect the responses -- like the "survey" communication pattern in zeromq. This will have to be done by publishing a nats request, collecting all responses that are received into a temporary inbox within a user-specified timeout.
I want to be able to target a specific instance of the service (a particular agent), and send it a regular request-reply.
The natural implementation is to tack on a "service instance id" to the end of the subject when the services subscribe -- in our example, this can be the node hostname that is unique within the deployment. The service then creates 2 subscriptions -- one with a wildcard subscription ("minion.>") and responds to "survey" messages on that and one with a full subscription including the "service instance id" ("minion.host42") and response to instance-specific messages on that.
Do you have similar a use case for xbus?
If I get this straight, you want to call the same method on either all the agents or only a specific one.
Yes I will have a similar use case in xbus.
Indeed the client would need to do two subscriptions, but I don't see the need for a wildcard one. 'minion' for the survey, and 'minion.host42' for the instance-specific one. The wildcard would make the client respond to the request targeted to specific clients.
One way would be to have two methods : one with a parameter, the other not, and they could share the same base subject:
We could also have a notion of "optional" parameter, which would imply subscribing to 2 subjects. One with the parameter, one without. The service declaration could look like:
service minion {
rpc Ping() returns (Pong) {
option (nrpc.subject_param) = "clientid";
option (nrpc.subject_param_optional) = true;
}
}The new thing here is handling the survey: it is a special case of request, not covered directly by the basic nats.Conn api (unless I missed something). But it should not be difficult to implement anyway.
So overall, does this look like what we're talking about?
package pkg;
option (nrpc.subject_param) = "pkglvlparam1 string";
service service1 {
option (nrpc.subject_param) = "svclvlparam1 string";
rpc method1() returns (bool) {}
rpc method2() returns (bool) {
option (nrpc.subject_param) = "methodlvlparam string";
}
}
// {pkglvlparam1}.service1.{svclvlparam1}.method1
// {pkglvlparam1}.service1.{svclvlparam1}.method2.{methodlvlparam}
How about the generated client?
Something like that indeed, although I am unsure about letting the type an option. Maybe let the user provide a specific type for the generated code would be enough, and that would be language specific.
In that case any type implementing the TextMarshal interface would be alright.
As for the generated client, I am not really decided, but my guess is that we would have the package and service subject values attached to the client service instance, and the function subject param would be passed as the first parameter of the method.