Type-based NCE

[SRL94]

Hi,

I found you used type-based noise-constrained estimation (NCE) for negative sampling, which is very interesting. Could you please tell me which part of your code is for the NCE?

Best regards
Sirui

[guolingbing]

In fact, we create two different weight matrices for entity prediction and relation prediction, respectively. When the target is a relation, we only need to call the standard NCE function tf.nn.nce_loss with the corresponding relation weight matrix as input.

[SRL94]

Thanks for the explaination but I am still confused about the NCE-based negative sampling. My understanding is: in the entity prediction, the input is the entity embedding matrix and the tf.nn.nce_loss output is the negative samples, is it right?

[guolingbing]

Almost right. tf.nn.nce_loss is a high-level API that computes (include negative sampling) and returns the loss. If you want to customize the sampling algorithm, I suggest you have a look at log_uniform_candidate_sampler. tf.nn.nce_loss has such an augment sampled_values to receive sampled entities:

a tuple of (sampled_candidates, true_expected_count, sampled_expected_count) returned by a *_candidate_sampler function.

Thanks for your attention to our work.