On entry to DLASCL parameter number 4 had an illegal value

[yihuang995]

When I am building the conceptnet, it reports this error:

[Fri Jun 7 05:22:25 2019]
rule miniaturize:
input: data/vectors/numberbatch-biased.h5, data/vectors/w2v-google-news-converted.h5
output: data/vectors/mini.h5
jobid: 9
resources: ram=4

Building prefix dict from /home/yhuang/Software/miniconda3/envs/ConceptNet/lib/python3.6/site-packages/wordfreq/data/jieba_zh.txt ...
Loading model from cache /tmp/jieba.ucd6ec2997f3667ea5b4295344599a98f.cache
Loading model cost 0.079 seconds.
Prefix dict has been built succesfully.
/home/yhuang/Datasets/conceptnet5-master/conceptnet5/vectors/miniaturize.py:83: FutureWarning:
Passing list-likes to .loc or [] with any missing label will raise
KeyError in the future, you can use .reindex() as an alternative.

See the documentation here:
https://pandas.pydata.org/pandas-docs/stable/indexing.html#deprecate-loc-reindex-listlike
vocab[shard_start:shard_end]
Traceback (most recent call last):
File "/home/yhuang/Software/miniconda3/envs/ConceptNet/bin/cn5-vectors", line 11, in
load_entry_point('ConceptNet', 'console_scripts', 'cn5-vectors')()
File "/home/yhuang/Software/miniconda3/envs/ConceptNet/lib/python3.6/site-packages/click/core.py", line 764, in call
return self.main(*args, **kwargs)
File "/home/yhuang/Software/miniconda3/envs/ConceptNet/lib/python3.6/site-packages/click/core.py", line 717, in main
rv = self.invoke(ctx)
File "/home/yhuang/Software/miniconda3/envs/ConceptNet/lib/python3.6/site-packages/click/core.py", line 1137, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/yhuang/Software/miniconda3/envs/ConceptNet/lib/python3.6/site-packages/click/core.py", line 956, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/yhuang/Software/miniconda3/envs/ConceptNet/lib/python3.6/site-packages/click/core.py", line 555, in invoke
return callback(*args, **kwargs)
File "/home/yhuang/Datasets/conceptnet5-master/conceptnet5/vectors/cli.py", line 294, in run_miniaturize
mini = miniaturize(input_filename, other_vocab=other_vocab, k=k)
File "/home/yhuang/Datasets/conceptnet5-master/conceptnet5/vectors/miniaturize.py", line 86, in miniaturize
U, _S, _Vt = np.linalg.svd(smaller, full_matrices=False)
File "/home/yhuang/Software/miniconda3/envs/ConceptNet/lib/python3.6/site-packages/numpy/linalg/linalg.py", line 1612, in svd
u, s, vh = gufunc(a, signature=signature, extobj=extobj)
ValueError: On entry to DLASCL parameter number 4 had an illegal value
[Fri Jun 7 05:27:42 2019]
Error in rule miniaturize:
jobid: 9
output: data/vectors/mini.h5
shell:
cn5-vectors miniaturize data/vectors/numberbatch-biased.h5 data/vectors/w2v-google-news-converted.h5 data/vectors/mini.h5
(exited with non-zero exit code)

[Fri Jun 7 10:18:40 2019]
Finished job 4.
473 of 475 steps (100%) done
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message

[rspeer]

Hmm that's exciting.

We tried to change the code recently to run in fewer than 15 GB of RAM. I'd been able to successfully build it. But I'm familiar with this error message, which means that we tried to take an SVD of a matrix that includes NaN as a value. I'll have to see if I can reproduce it.

[rspeer]

In the meantime: if you check out the released version5.7 branch instead of the master branch, delete at least the contents of data/vectors, and rebuild, does that work?

[yihuang995]

In the meantime: if you check out the released version5.7 branch instead of the master branch, delete at least the contents of data/vectors, and rebuild, does that work?

it succeeded when I follow your suggestions. Thank you!