Ability to handle ridiculously large input

Question

Ability to handle ridiculously large input

Closed this issue 7 years ago · 3 comments

While I use these tools to convert some otf to ttf.

otfccdump NotoSerifCJKtc-Regular.otf | otfcc-c2q | otfccbuild -o NotoSerifCJKtc-Regular.ttf

An Error prompts out and says

/usr/lib/node_modules/otfcc-c2q/c2q_cli.js:7
process.stdin.on('data', function (buf) { glyfsource += buf.toString(); });
                                                            ^

RangeError: Invalid string length
    at Socket.<anonymous> (/usr/lib/node_modules/otfcc-c2q/c2q_cli.js:7:61)
    at emitOne (events.js:96:13)
    at Socket.emit (events.js:188:7)
    at readableAddChunk (_stream_readable.js:176:18)
    at Socket.Readable.push (_stream_readable.js:134:10)
    at Pipe.onread (net.js:548:20)
otfccbuild : Parse into JSON : [ERROR] Cannot parse JSON file "(null)". Exit.

However, I have no problem converting other fonts like Source Han Sans.
Does node version matter? Node version of my computer is v6.10.0.

Answer 1 · 2017-04-07T13:30:43.000Z

It means that your source file is too large... Will test tomorrow using some internal build. 发自我的 iPhone

…

在 2017年4月7日，21:29，limelightful ***@***.***> 写道： While I use these tools to convert some otf to ttf. otfccdump NotoSerifCJKtc-Regular.otf | otfcc-c2q | otfccbuild -o NotoSerifCJKtc-Regular.ttf An Error prompts out and says /usr/lib/node_modules/otfcc-c2q/c2q_cli.js:7 process.stdin.on('data', function (buf) { glyfsource += buf.toString(); }); ^ RangeError: Invalid string length at Socket.<anonymous> (/usr/lib/node_modules/otfcc-c2q/c2q_cli.js:7:61) at emitOne (events.js:96:13) at Socket.emit (events.js:188:7) at readableAddChunk (_stream_readable.js:176:18) at Socket.Readable.push (_stream_readable.js:134:10) at Pipe.onread (net.js:548:20) otfccbuild : Parse into JSON : [ERROR] Cannot parse JSON file "(null)". Exit. However, I have no problem converting other fonts like Source Han Sans. Does node version matter? Node version of my computer is v6.10.0. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

Answer 2 · 2017-04-25T12:19:33.000Z

For some reason (bug 1, bug 2), Node.js cannot handle input longer than 512 MiB (256 mebi UTF-16 code units).

According to a Stack Overflow post, JSONStream might be useful to load and store this kind of large JSON object.
I don't have time to modify the source code and test right now. Hope someone could do this for us.

Answer 3 · 2017-08-21T18:45:59.000Z

I think it is fixed now. Currently c2q depends on Megaminx backwards and can read arbitrary-sized font as input.