doRedis Error setting TCP_NODELAY

[mlist]

I have trouble spawning local workers with doRedis, where I get the following error

Error setting TCP_NODELAY
Calls: redisWorker -> redisConnect -> .openConnection -> .Call
Execution halted

I'm trying to use doRedis in a docker image (linked to a redis docker container) and I'm not sure if this is docker related. However, I tried this both with redis 2.8 and 3 and with the latest CRAN version of rredis and doRedis as well as with the current github versions of both packages. R is version 3.2.2.

To reproduce the problem install docker (I use version 1.8.1) and then run the following commands

sudo docker pull rocker/r-base:latest
sudo docker pull redis:latest
sudo docker run -d --name redis redis
sudo docker run -it -P --name rbase --link redis:redis r-base

This will start the R container in interactive mode, then just run

install.packages("doRedis")
doRedis::registerDoRedis('test', 'redis')
doRedis::startLocalWorkers(2, 'test', 'redis')

The last command will produce the error message.

[bwlewis]

Can you try using the latest dev version instead of the one on CRAN? Replace:

install.packages("doRedis")

with

devtools::install_github("bwlewis/doRedis")

There are many changes in doRedis recently that I hope to finalize for a new CRAN version soon.

[mlist]

I already tried using this. Same result, unfortunately. I also tried using the development version of rredis, still the same.

[bwlewis]

OK, well I'll try to replicate this. I don't have a docker setup handy however, so it may take me a while.

What OS are you using?

[bwlewis]

Oh yes, what OS and R version?

[mlist]

The host system is Ubuntu 14.04, but I guess what is more interesting is that the docker image I use (r-base) is based on Debian testing. I think I'll try if the error also occurs on an ubuntu image. R is 3.2.2.

[mlist]

I didn't get a ubuntu image to work yet, but I now exchanged debian testing with debian jessie and it still gave the same error message.

[bwlewis]

I added pass-through options to redisConnect where used as a hack-y way to maybe get around this problem until I can replicate it (I can't repro the problem on the usual non-docker OS/R platforms).

Try this:

devtools::install_github("bwlewis/doRedis")
library(doRedis)
registerDoRedis("jobs", nodelay=FALSE)

seperately, start workers similarly...

library(doRedis)
redisWorker("jobs", timeout=2, nodelay=FALSE)

-or-

library(doRedis)
startLocalWorkers(1, "jobs", timeout=2, nodelay=FALSE)

Note the added nodelay parameter.

Maybe this will work around the problem?

[mlist]

Awesome! This solves the problem for me. Thanks!

[trcook]

I was having a similar problem -- i'm trying to do almost the exact same thing with docker and R. I found that, for whatever reason you could get this to work if you ran the registration command twice.

i.e.:

try(redisWorker(host='redis',queue='jobs'),silent=T)
redisWorker(host='redis',queue='jobs')

Not sure if this helps track down the source of the issue.
(project -- for reference: github with images: worker and master )

[bwlewis]

Tom, did the above fix also solve your problem? It's in the dev version:

devtools::install_github("bwlewis/doRedis")

Try that out, I'm trying to make sure that version is ready for CRAN release...would like to release it soon...

[trcook]

It does seem to work -- I tried it when accessing a redis controlled set of workers from outside of docker.

To the extent that it fixes the (probably few) number of cases where this is a problem, I don't think it's too hacky. After looking at it, it appears that the nodelay issue drips down at least into the C++ in the rredis package.

From the looks of it, it would minimize errors to just setup the function so that nodelay is off by default. Though, I'm not sure what implications this might have efficiency or speed-wise -- the intricacies of tcp elude me...

I happened upon my solution specifically because I wasn't in a good way to install the dev version in my docker image and needed a workaround. Given the docker-centric nature of the issue, and the fact that it's not always convenient to rebuild an image, I thought I'd put it up here.

[bwlewis]

This is fixed in the 2.0.0 branch (via the redux package and the C hiredis interface to Redis).

Will appear on CRAN soon.