guxd/deep-code-search

What role do '<s>' and '</s>' label play in the vocabulary?

Closed this issue · 1 comments

I understand the '<pad>' and '<unk>' label in the vocabulary. But what's the use of '<s>' and '</s>' label? Do you add '<s>' and '</s>' label at the beginning and the end of each sentence in the training set? I found there exit '</s>' label in the api_seq sentence.

guxd commented

Yes, they represent the beginning and the end of each sequence respectively.