brettwooldridge/NuProcess

How to increase the buffer size of the builder

VEDANTDOKANIA opened this issue · 4 comments

I am getting very large data from the plugin side and i am getting only half data because of length of buffer. How can I increase the length of the buffer in stdout and stderr

@VEDANTDOKANIA,

You can't increase the buffer sizes; they're hard-coded. The expectation for large output (or for sending in large input) is that you'll receive multiple callbacks. It's not scalable for NuProcess to assume it's safe to try and load the full stdout or stderr and hand it to the callback in a single buffer. If you're getting >64K in output, you'll need to do something with it between callbacks. (Note that you cannot leave it in the buffer NuProcess passes to onStdout or onStderr, as doing so will cause processing to fail. You need to return the buffer to NuProcess with at least one byte available for writing.) One option is to allocate your own buffer (be it a ByteBuffer, ByteArrayOutputStream or whatever) and use that to aggregate data across onStdout/onStderr callbacks (where you'd probably need 2 buffers if you're using both).

@VEDANTDOKANIA,

I'm not entirely sure what you're trying to ask, or what your code is trying to show. (One thing I can tell you for sure, you shouldn't call handler.onStart(process); NuProcess will make all the handler callbacks itself automatically, in the correct places for the process's lifecycle. It will call onStart after onPreStart at some point after you call processBuilder.start().)

I don't have any direct familiarity with VertX, so I can't really offer any advice/insight into anything that involves it. That said, if you use NuProcessBuilder.start to start your process, you're essentially telling NuProcess to run your process asynchronously, and NuProcess handles the threading. That can be very useful if your application starts a number of long-running processes that sporadically produce output, since a large number of processes can efficiently be pumped by a small number of threads. In that model, all of your NuProcessHandler.onStdin, onStdout and onStderr callbacks will happen from a NuProcess-managed thread (and it will always be the same NuProcess thread), regardless of your surrounding environment. NuProcess always calls onPreStart and onStart from the thread where you call NuProcessBuilder.start. Your handler's onExit callback could be invoked from either thread--the starting thread or a NuProcess thread--depending on how the process exits.

If you always want to block and wait for the process to complete, you may want to consider using NuProcessBuilder.run instead. If you use run, all NuProcessHandler callbacks will always be made on the thread you called run on, since it's used to do the actual process handling. If your goal is for NuProcess's processing to happen on some work thread you control, run should do it.