natefinch/pie

Pie throws panic if iop.proc is nil

lonelycode opened this issue · 17 comments

Not sure why this is, it could just be that the process is never assigned properly when it is created, this is using a python script as an external RPC plugin, everything works until it tries to close the process and then it explodes at this point:

if err := iop.proc.Signal(os.Interrupt); err != nil {
    return err
}

Not sure if this is just the way I am handling it (I'm trying to use an external, non-golang plugin writen in python, gist is here

Thanks for the bug report. I'll try to figure out why the process is nil. I wonder if I have a logic error somewhere. I don't think just avoiding the panic on cleanup is the right approach, instead we should figure out why proc is nil at that point, and try to prevent it from being nil in the first place.

Also, thanks a ton for providing an example python script to use with this. I'm not a python expert, but I really wanted to try using a python script as a plugin to show how this package supports plugins from different languages.

Actually, looks like this is happening with my example programs as well...

run go get -u github.com/natefinch/pie (or git pull) and try again... there was a bug I introduced when refactoring for tests, ironically. I'm going to add more tests to catch similar problems in the future.

Allready did - as soon as the fix went in :-)

Works like a charm.

Since this uses stdio though, it's prone to blocking if one RPC call takes too long, I think that's why most RPC implementations instantly asume that you are going to run a TCP server, since you can then fork out to handle load and long-running replies.

Unless I've totally set this up wrong :-/

It's actually not blocking. The RPC server starts a goroutine for each request. See here: https://github.com/golang/go/blob/release-branch.go1.4/src/net/rpc/server.go#L476

You can test it out by adding a long wait in your plugin's function, and using rpc.Client.Go() to make a few requests in parallel.

The code for the master process would look something like this:

package main

import (
    "log"
    "net/rpc/jsonrpc"

    "github.com/natefinch/pie"
)

func main() {
    path := "plugin.py"
    client, err := pie.StartProviderCodec(jsonrpc.NewClientCodec, os.Stderr, path)
    if err != nil {
        log.Fatalf("Error running plugin: %s", err)
    }
    defer client.Close()

    var reply1 string
    call1 := client.Go("Plugin.SayHi", "one", &reply1, nil)

    var reply2 string
    call2 := client.Go("Plugin.SayHi", "two", &reply2, nil)

    var reply3 string
    call3 := client.Go("Plugin.SayHi", "three", &reply3, nil)

    count := 0
    for count < 3 {
        select {
        case <-call1.Done:
            log.Printf("#1 is done, reply: %s", reply1)
            count++
        case <-call2.Done:
            log.Printf("#2 is done, reply: %s", reply2)
            count++
        case <-call3.Done:
            log.Printf("#3 is done, reply: %s", reply3)
            count++
        }
    }
}

You'll see that the total runtime is approximately equal to one of the waits, not 3.

Oh sorry... you're talking about using python's JSON RPC... I don't know about that... I'm not much of a pythonista. I would imagine there's a way to do it such that the calls don't have to be blocking.

There's one and only one in and out, but that's no different than the one and only one TCP connection to a web-based RPC server, too. I presume pretty much every RPC server implementation has a way to accept more than one connection at a time, or otherwise force serialization... I'm honestly not an RPC expert... but I trust the go implementation is doing the right thing.

Right, I have tested this just to be sure.

RPCServer can handle out-of-order responses over STDIO.

Which is great, I've created a new python demo which uses threaded handlers for RPC calls over STDIN with a random timer variable to make sure that the responses are printed out-of-order.

The new gist for this demo plugin is here with the matching golang master here. It might help others :-)

Thanks for all the help, we may be using this in our application soon!

That's awesome. I'm glad it's working out so well for you. I'd love to see the end result, if it's open sourced.

Would you care to make a PR with the python code and matching master code? I moved the examples into a single subdirectory, so you could just add those two files to examples/python/master_python. I could do this myself, but I figured I'd offer you the opportunity to do a PR, to get proper credit (I'd credit you anyway, but this way it's is more "official"). I'd be super happy to have an example of a python plugin working with a go program in the actual repo.

Awesome. Let me know if there's anything I can do to help. Actually, here's one thing, typo on the front page of tyk.io: "Swagger and API Blubrint suport". I went to go make a PR against the website, but couldn't find the code for it (perhaps it's private?).

It is 😊

Thanks for pointing it out, will make the change…

@lonelycode - just a ping on the PR :)

Awesome, thanks!

On Fri, Jun 5, 2015 at 3:45 AM Martin Buhr notifications@github.com wrote:

@natefinch https://github.com/natefinch Done :-)


Reply to this email directly or view it on GitHub
#1 (comment).