TypeError : Nonetype object not iterable

[ayush2051]

I have been getting a typerror Nonetype object not iterable in the below line,
ctxts[i], ans_pos[i] = preprocessing.filter_context(ctxts[i], ans_pos[i], filter_window_size_before, filter_window_size_after, filter_max_tokens)

After having populated the answers & passing onto the generation model batch-wise, it's showing the error, "Couldn't find the char position in the filter_context block of preprocessing"

[tomhosking]

Hi, what dataset are you using? The error implies that the answer string couldn't be found in the context.

[ayush2051]

Hi Tom, I have used a custom dataset where I have generated custom answers which are then fed to the generation model. In one of the contexts, it is failing even if the answer heads are present inside the context. Unable to figure out the issue why it is failing.

[tomhosking]

Does the context or answer contain any non ascii characters? The tokenizer can struggle with tracking character offsets for dual-width characters.