uswitch/blueshift

java.lang.StackOverflowError problems at startup

Closed this issue · 5 comments

Deleted user commented

I'm on Java 7 (1.0.79) on Linux Ubuntu 14.4 (Mint). I built Blueshift a few weeks ago and set it up. It ran fine.

But, now it does not. At startup I get this:
[2015-08-28 20:36:41,430] INFO uswitch.blueshift.main - Starting Blueshift with configuration elb.config.edn
[2015-08-28 20:36:41,447] INFO uswitch.blueshift.telemetry - Loading reporter uswitch.blueshift.telemetry/log-metrics-reporter for registry #<MetricRegistry com.codahale.metrics.MetricRegistry@c909572>
[2015-08-28 20:36:41,474] INFO uswitch.blueshift.s3 - Starting BucketWatcher. Polling edmodo.hadoop.export every 30 seconds for keys matching #"elblogs.*"
[2015-08-28 20:36:41,480] INFO uswitch.blueshift.s3 - Starting KeyWatcherSpawner
[2015-08-28 20:36:41,489] INFO uswitch.blueshift.telemetry - Starting Slf4j metrics reporter

This happens without any JVM stack memory arguments. Also, have tested various combinations of memory arguments. Here is a sample startup line:

java -Dlogback.configurationFile=./etc/logback.xml -Xss325k -Xmx10g -jar target/blueshift-0.1.0-SNAPSHOT-standalone.jar --config elb.config.edn

Do you have a time-tested combination of memory flags that will support 5-10 polled directories?

(I am filing an issue as there is no mailing list/google groups etc.)

ragnard commented

Hi,

Thanks for the report. Can you provide a complete stacktrace? I can't see a StackOverflowException in the log output above.

ragnard commented

That said, I'm going to take a wild guess and say that the number of leaf directories in your bucket has grown sufficiently large to trigger a concat bug in one of:

https://github.com/uswitch/blueshift/blob/master/src/uswitch/blueshift/s3.clj#L39
https://github.com/uswitch/blueshift/blob/master/src/uswitch/blueshift/s3.clj#L40
https://github.com/uswitch/blueshift/blob/master/src/uswitch/blueshift/s3.clj#L58

But a stacktrace would help confirm that. We can have a look at this next week.

Deleted user commented

The bug does not happen when I run as root. I was running it as a regular
user.

What is a "key"? Is it (in Posix terms) a leaf file name, a directory name,
or a complete path?

On Fri, Aug 28, 2015 at 3:06 PM, Ragnar Dahlén notifications@github.com
wrote:

That said, I'm going to take a wild guess and say that the number of leaf
directories in your bucket has grown sufficiently large to trigger a
concat bug in one of:

https://github.com/uswitch/blueshift/blob/master/src/uswitch/blueshift/s3.clj#L39

https://github.com/uswitch/blueshift/blob/master/src/uswitch/blueshift/s3.clj#L40

https://github.com/uswitch/blueshift/blob/master/src/uswitch/blueshift/s3.clj#L58

But a stacktrace would help confirm that. We can have a look at this next
week.

—
Reply to this email directly or view it on GitHub
#13 (comment).

Deleted user commented

The error message did not give a stacktrace, only the single line shown in
the example.

On Fri, Aug 28, 2015 at 3:55 PM, Lance Norskog lance@edmodo.com wrote:

The bug does not happen when I run as root. I was running it as a regular
user.

What is a "key"? Is it (in Posix terms) a leaf file name, a directory
name, or a complete path?

On Fri, Aug 28, 2015 at 3:06 PM, Ragnar Dahlén notifications@github.com
wrote:

That said, I'm going to take a wild guess and say that the number of leaf
directories in your bucket has grown sufficiently large to trigger a
concat bug in one of:

https://github.com/uswitch/blueshift/blob/master/src/uswitch/blueshift/s3.clj#L39

https://github.com/uswitch/blueshift/blob/master/src/uswitch/blueshift/s3.clj#L40

https://github.com/uswitch/blueshift/blob/master/src/uswitch/blueshift/s3.clj#L58

But a stacktrace would help confirm that. We can have a look at this next
week.

—
Reply to this email directly or view it on GitHub
#13 (comment).

Deleted user commented

"Fixed" by running as root.