Criticisms of RPC
Closed this issue · 8 comments
Recently on IRC, we discussed an article in which Alan Kay explained OO. In his explanation, he derided RPC as a failed premise which OO solved. We were unsure what he meant specifically, so I posed the question to Hacker News.
The most interesting response I got was this:
That might be not what Alan Kay meant, but here's one description of fundamental problems with RPC: [1]
[...] the fundamental problem is that RPC tries to make a distributed invocation look like a local one. This can't work because the failure modes in distributed systems are quite different from those in local systems, so you find yourself having to introduce more and more infrastructure that tries to hide all the hard details and problems that lurk beneath. That's how we got Apollo NCS and Sun RPC and DCE and CORBA and DSOM and DCOM and EJB and SOAP and JAX-RPC, to name a few off the top of my head, each better than what came before in some ways but worse in other ways, especially footprint and complexity. But it's all for naught because no amount of infrastructure can ever hide those problems of distribution. Network partitions are real, timeouts are real, remote host and service crashes are real, the need for piecemeal system upgrade and handling version differences between systems is real, etc. [...]
[1] http://erlang.org/pipermail/erlang-questions/2008-May/035209.html
I recommend you read the actual email on the erlang list. (It's written by one of the architects at Basho.) He describes what a couple of the responses on HN described, which is that RPC tries to map the messaging onto a language, whereas messaging includes no strict definition. Why?
Ultimately, RPC is a leaky abstraction. It can't hide what it tries to
hide, and because of that, it can easily make the overall problem more
difficult to deal with by adding a lot of accidental complexity.
I recommend you read the links in that email, especially "RPC Under Fire." Which of these criticisms apply to MuxRPC? How might this compare to go channels or erlang's messaging?
Just to offer some of my thoughts, MuxRPC I think dodges the complexity criticism because it's relatively simple: there are a few core network primitives in packet-stream, the IDLs are replaced by very lightweight manifests, and its messages are plaintext JSON. MuxRPC does map very cleanly to Javascript, and I doubt another language would be too burdened to implement it, but I haven't looked into it.
I do wonder about the issue of network conditions and errors, though I think Vinoski's solution is simply to have the developer handle them explicitly instead of trying to abstract them away.
If you really wanted to get rid of IDLs/manifests, you could implement MuxRPC like this:
rpc.async('follow', { id: bob.id }, function(err, data) { ... })
pull(rpc.source('createHistoryStream'), pull.drain(...))In which case, you're not truly that far away from REST:
http.request('post', url, { id: bob.id }, function(err, res) { ... })
pull(http.request('get', historyStreamUrl), pull.drain(...))The primary difference being that HTTP initiates a separate connection for each request.
This is all absolutely correct.
The problem with rpc, is that rpc has indeed tried to pave over certain truths about distributed computing.
such as: disconnections happen.
muxrpc does not do this. It doesn't have retries or, pretend there is a connection when there isn't.
If the connection fails, all current requests and streams emit an error, that the application must deal with.
Thus applications must be designed to recover from errors (which is why the replication protocol is designed so that whenever a failure occurs, it will be in a valid state, and can continue)
If you strip it back to messages, pretty much the first thing you'll do is build abstractions about certain types of messages on top again. you probably want messages that represent request/response. And maybe streams... now you have RPC again. The important thing, though, is not to pretend you have a local system, and still design for failure. At the end of the day, this comes down to the discipline of the programmer.
Agreed.
MuxRPC is a fairly light RPC design. Core principles seem to be:
- Use text, base64, and JSON
- Exchange function calls and directional streams
- Write connection-aware application code
- Name and version RPC APIs
to clarify one thing: JSON isn't really a principle here. you could drop in another encoding like msgpack and the semantics would be the same. json is just the most accessable encoding format.
At some point, we'll probably switch to a length delimited encoding at the top level, so that it's possible to have binary streams without base64 expanding the length 4/3 times.
Good to know
update: muxrpc now uses a length delimited encoding, which allows streams of buffers without having to encode them as base64.
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.