parallel-js/parallel.js

Can't require libs or other files in node?

blakewest opened this issue · 6 comments

Hey, I'm trying to use this for a local node project I have. It doesn't seem like I can easily require in a lib file to use (like lodash, for example). I tried passing strings, and absolute paths to it, but no luck. The specs in the source code make it look like this is not possible in Node. I'm curious if there's some way around this, or some reason for it? It makes the project basically unusable if I can't require in a basic library like lodash.
Because the best I could otherwise do is some absurdly hacky thing like create an array of function names for everything in the lodash exports, and require in each one to Parallel, and then rename all my uses of lodash to just do the function name, instead of lodash.

I feel like I'm missing something... this must be possible.

Can you provide a code example?

@mathiasrw thanks for getting back quickly.
Sure.. to make things a bit more concrete, this snippet would be representative of what I want to acheive

var myLocalExports = require('../myLocalExports.js');
var _ = require('lodash');

function myDataManipulationFunction(data) {
  var intermediateValue = myLocalExports.sweetFunction(data);
  var otherValue = _.zip(intermediateValue, [1,2,3])
  return otherValue;
}

var p = new Parallel(myDataArray, {env: myEnvVars})
  .require(__dirName + '../myLocalExports.js')
  .require(__dirName + '../node_modules/lodash/lodash.js')

p.map(myDataManipulationFunction) // throws error, "myLocalExports" is not defined

Which... kinda makes sense. Even if the exports from myLocalExports made it into the scope of the worker.... where exactly did I define the variable "myLocalExports"

That code example is contrived, but hopefully my question is clear. What should I be doing to require in exported modules in node? And how could I assign those modules to a variable?

Essentially I need an equivalent for
var myLocalExports = require('../myLocalExports.js') and/or for var _ = require('lodash')

I was hoping there'd be something like...

var myLocalExports = require('../myLocalExports.js');
var _ = require('lodash');
var p = new Parallel(myData).require({myLocalExports: myLocalExports, _: _})

Is there a way to accomplish this kind of thing in Parallel.js?

Thanks!!! The library looks so sweet. I really want to use it in my project.

@mathiasrw any thoughts on this? I've been considering some hacks to make this possible in a not too gross syntax. But still would love to hear your thoughts.

SOrry long response time. I am traveling, so will only be able to get really back to you after christmas...

stale commented

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

Has anybody figured out if external dependencies can be used with Parallel.js?

I'm looking to abstract file parsing to web workers and the function would require external dependencies.