OpenPipe/ART

Add error message if log_probs not found and allow_training_without_logprobs=False

Opened this issue · 2 comments

Observed in ART 0.4.4
If logprobs is not present in messages_and_choices of a trajectory, art.local.tokenize.tokenize_trajectory returns None, which causes art.local.backend._train_model to throw the warning "Skipping tuning as there is no suitable data. This can happen when all the trajectories in the same group have the same reward and thus no advantage to train on." which is a rather misleading warning.

Suggestion:
Either add a detailed validation of trajectories given the configs, or add an extra warning patching this issue.

Hey, @mattiasutancykeln!
Do you want to open a PR? If not, we will look into it.

This isn't a critical bug in any way. I can attempt a PR in a week or so when my schedule clears up :)