Generating biases

[ncoronges]

There are two files used for generation referenced in config.py:
caption_style.npy'
romance_style.npy'

these style vectors are used by generate.py
bneg = numpy.load(config.paths['negbias'])
bpos = numpy.load(config.paths['posbias'])

While supposedly you can create your decoder using the skip-thoughts decoder there does not seem to be a step where the style vectors are generated. If you have trained on your own corpus, and have dictionary and model, how do you then generate the style vectors and biases?

[naotokui]

Any info on this topic?? I'd love to know how to generate the style vectors.

[koga73]

I would like to know the same. How are the .npy bias files generated?

[Pantsworth]

Based on code in generate.py, in the story function, the sentences are encoded, then the mean is taken using numpy.mean before applying the pos and neg biases.

svecs = skipthoughts.encode(z['stv'], sentences, verbose=False)
shift = svecs.mean(0) - z['bneg'] + z['bpos']

To train new biases, encode your sentences and then save the mean of the encoded vector – numpy.save("path", <encoded_vector_array.mean(0)>). This will create a compatible bias file, which will be a simple numpy array.

The romance_style bias file was "the mean of the skip-thought vectors for romance novel passages that are of length > 100". Based on my limited testing of new bias vectors, this line culling is important to getting good results (although the overall effect may be more subtle than expected). Apologies in advance if any of the above is incorrect, I am not an authoritative source. Best of luck!