josephreisinger/vowpal_porpoise

GridSearchCV with n_jobs > 1 (Parallelized) with VW classfier results in a Broken Pipe error

viveksck opened this issue · 0 comments

127         with self.vw_.training():
128             for instance in examples:
129                 self.vw_.push_instance(instance) <-----
130 
131         # learning done after "with" statement
132         return self
133 

...........................................................................
/usr/local/lib/python2.7/dist-packages/vowpal_porpoise-0.3-py2.7.egg/vowpal_porpoise/vw.pyc in push_instance_stdin(self=<vowpal_porpoise.vw.VW instance>, instance='2 | 42:2.000000 29:16.000000 60:13.000000 61:16....3:16.000000 52:16.000000 33:7.000000 37:16.000000')
204 if self.vw_process.wait() != 0:
205 raise Exception("vw_process %d (%s) exited abnormally with return code %d" %
206 (self.vw_process.pid, self.vw_process.command, self.vw_process.returncode))
207
208 def push_instance_stdin(self, instance):
--> 209 self.vw_process.stdin.write(('%s\n' % instance).encode('utf8'))
210
211 def start_predicting(self):
212 model_file = self.get_model_file()
213 # Be sure that the prediction file has a unique filename, since many processes may try to

IOError: [Errno 32] Broken pipe

To reproduce: Just pass the parameter n_jobs = 10 to GridSearchCV in example_sklearn.py