slow with very large files

[michaelkirk]

I'm using koa-range via local-web-server to serve up small slices (e.g. 12kb) of a large (12GB) file. The response time is very slow - on the order of 30+ seconds.

I'm not at all familiar with node, but I'm somewhat suspicious of this line, which seems to read and discard from the very beginning of the file.

rawBody = rawBody.pipe(slice(start, end + 1));

That rawBody seems to come from this line in koa-send:

ctx.body = fs.createReadStream(path)

It might be better if we can find a way to stream only the requested portion of the data , rather than streaming the entire stream through the slice function.

e.g. somewhere in koa-send:

ctx.body = fs.createReadStream(path, { start, end });

[michaelkirk]

I hacked a POC and verified that the responses are now super fast.

diff -r node_modules/koa-range/index.js hacks-node-modules/koa-range/index.js
25a26,31
>   // avoid multi ranges
>   var firstRange = ranges[0];
>   var start = firstRange[0];
>   var end = firstRange[1];
>   ctx.req.foo_range = { start, end };
> 
33c39,40
<   var first = ranges[0];
---
> 
> 
37,42c44
<   // avoid multi ranges
<   var firstRange = ranges[0];
<   var start = firstRange[0];
<   var end = firstRange[1];
< 
<   if (!Buffer.isBuffer(rawBody)) {
---
>     if (!Buffer.isBuffer(rawBody)) {
45c47,48
<       rawBody = rawBody.pipe(slice(start, end + 1));
---
>       // redundant, since it's been sliced in koa-send
>       // rawBody = rawBody.pipe(slice(start, end + 1));

diff -r node_modules/koa-send/index.js hacks-node-modules/koa-send/index.js
153c153,161
<   ctx.body = fs.createReadStream(path)
---
>   if (ctx.req.foo_range != undefined) {
>     let start = ctx.req.foo_range.start;
>     let end = ctx.req.foo_range.end;
>     console.log(`range requested: ${start}-${end}`);
>     ctx.body = fs.createReadStream(path, { start, end })
>   } else {
>     console.log("no range requested");
>     ctx.body = fs.createReadStream(path)
>   }

[masx200]

I also encountered the same problem

[masx200]

#24