bwlewis/doRedis

subscript out of bounds error

Closed this issue · 4 comments

ksens commented

While going through your vignette, I tried Listing 1:

registerDoRedis("jobs")
startLocalWorkers(n=2, queue = "jobs")
foreach ( icount (10) ,.combine = sum , .multicombine = TRUE ,.inorder = FALSE ) %dopar%
  4* sum (( runif (1000000)^2 + runif (1000000)^2) <1)/10000000
removeQueue("jobs")

I got Listing 1 to give an approximation of Pi as expected (so foreach with doRedis backend is working), but from time to time I keep getting the following output:

> Error in work[[1]] : subscript out of bounds
Calls: redisWorker -> paste
Execution halted
Error in work[[1]] : subscript out of bounds
Calls: redisWorker -> paste
Execution halted

Any idea as to why this is coming up?

EDIT:
On further testing, I noticed that if I just do:

registerDoRedis("jobs")
startLocalWorkers(n=2, queue = "jobs")

... I get the same error i.e. there is some problem with the spawning of the local R workers.

bwlewis commented

I'm not sure what this is! Perhaps I've introduced a bug recently.

Can you include the output of

packageDescription("doRedis")

packageDescription("rredis")
On May 13, 2016 3:10 PM, "Kriti Sen Sharma" notifications@github.com
wrote:

While going through your vignette, I tried Listing 1:

registerDoRedis("jobs")
startLocalWorkers(n=2, queue = "jobs")
foreach ( icount (10) ,.combine = sum , .multicombine = TRUE ,.inorder = FALSE ) %dopar%
4* sum (( runif (1000000)^2 + runif (1000000)^2) <1)/10000000
removeQueue("jobs")

I got Listing 1 to give an approximation of Pi as expected (so foreach
with doRedis backend is working), but from time to time I keep getting the
following output:

Error in work[[1]] : subscript out of bounds
Calls: redisWorker -> paste
Execution halted
Error in work[[1]] : subscript out of bounds
Calls: redisWorker -> paste
Execution halted

Any idea as to why this is coming up?

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub
#36

bwlewis commented

Hi Kriti,

I think that it might be related to some TCP timeout options that I
recently introduced (see the commit log).

Those are important for issue #34.

I've got a new commit that better manages and documents that change. I'll
push it tonight.
On May 13, 2016 4:09 PM, "Bryan W. Lewis" blewis@illposed.net wrote:

I'm not sure what this is! Perhaps I've introduced a bug recently.

Can you include the output of

packageDescription("doRedis")

packageDescription("rredis")
On May 13, 2016 3:10 PM, "Kriti Sen Sharma" notifications@github.com
wrote:

While going through your vignette, I tried Listing 1:

registerDoRedis("jobs")
startLocalWorkers(n=2, queue = "jobs")
foreach ( icount (10) ,.combine = sum , .multicombine = TRUE ,.inorder = FALSE ) %dopar%
4* sum (( runif (1000000)^2 + runif (1000000)^2) <1)/10000000
removeQueue("jobs")

I got Listing 1 to give an approximation of Pi as expected (so foreach
with doRedis backend is working), but from time to time I keep getting the
following output:

Error in work[[1]] : subscript out of bounds
Calls: redisWorker -> paste
Execution halted
Error in work[[1]] : subscript out of bounds
Calls: redisWorker -> paste
Execution halted

Any idea as to why this is coming up?

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub
#36

ksens commented

This is what I had:

> packageDescription("doRedis")
Package: doRedis
Type: Package
Title: "Foreach" Parallel Adapter Using the "Redis" Database
Version: 1.2.0
Date: 2015-08-06
Author: B. W. Lewis <blewis@illposed.net>
Maintainer: B. W. Lewis <blewis@illposed.net>
Description: A parallel back end for the "foreach" package using the "Redis" database.
BugReports: https://github.com/bwlewis/doRedis/issues
Depends: R (>= 2.5), rredis (>= 1.7.0), foreach(>= 1.3.0), iterators(>= 1.0.0), utils
Imports: parallel, stats
Suggests: knitr
VignetteBuilder: knitr
License: GPL-2
LazyLoad: yes
RoxygenNote: 5.0.0
Built: R 3.2.2; x86_64-pc-linux-gnu; 2016-05-13 17:56:36 UTC; unix
RemoteType: github
RemoteHost: api.github.com
RemoteRepo: doRedis
RemoteUsername: bwlewis
RemoteRef: master
RemoteSha: 6c086bf81cff09be4a2ea79af59f9f25fe938112
GithubRepo: doRedis
GithubUsername: bwlewis
GithubRef: master
GithubSHA1: 6c086bf81cff09be4a2ea79af59f9f25fe938112

-- File: /usr/local/lib/R/library/doRedis/Meta/package.rds

and...

> packageDescription("rredis")
Package: rredis
Type: Package
Title: "Redis" Key/Value Database Client
Version: 1.7.1
Date: 2015-07-04
Author: B. W. Lewis
Maintainer: B. W. Lewis <blewis@illposed.net>
Description: R client interface to the "Redis" key-value database.
License: Apache License (>= 2.0)
LazyLoad: yes
Built: R 3.2.2; x86_64-pc-linux-gnu; 2016-05-13 17:56:31 UTC; unix
RemoteType: github
RemoteHost: api.github.com
RemoteRepo: rredis
RemoteUsername: bwlewis
RemoteRef: master
RemoteSha: 6df5427ecb4849051914cad00c3d866f7f068919
GithubRepo: rredis
GithubUsername: bwlewis
GithubRef: master
GithubSHA1: 6df5427ecb4849051914cad00c3d866f7f068919

-- File: /usr/local/lib/R/library/rredis/Meta/package.rds

I updated to the latest commit on doRedis, but I still find the problem. Also I added an EDIT to the original issue.

ksens commented

After discussion with @bwlewis, here is the solution:

While following Listing 1, I was also testing Listing 2. So I was typing the following command:

startLocalWorkers(n=4,queue = "jobs", timeout=1)

The correct solution is to use the linger argument (timeout is not used in the startLocalWorkers function of doRedis anymore, as it clashed with timeout option in rredis package):

startLocalWorkers(n=4,queue = "jobs", linger=1)

Vignette fixed in e1522eb