Citation request

[222464]

Hi,

I sent an email and received no response, so I am trying the issues section as a way to contact the authors.

I would like to request a citation from the "Decision Transformers" paper. Our work is very relevant I believe - the novelty presented in the "Decision Transformers" paper is identical to ours that we introduced nearly 2 years ago.

It's a blog post and not a paper, but I don't think that matters. The source code has also been public for a long time. Here is the blog post in question: https://ogma.ai/2019/08/acting-without-rewards/

The idea of "RL as a sequence prediction/generation problem" is identical to ours. The use of the Transformer is not, but that is not the novelty being presented so I don't think it matters.

We used slightly different language, as we do not use Transformers but rather a bio-inspired system (that avoids backpropagation). Still, it does the whole process of predicting a sequence and performing "goal relabeling". We took it a step further and did so hierarchically as well. As in decision transformers, we do not use any classic RL algorithm (no dynamic programming), but rather we learn to predict the sequences in such a way that they can be "prompted" and generate desired trajectories. We invented it specifically as a way to avoid rewards, but rewards can be used as well. Decision Transformers also do not require rewards necessarily, as shown in one of the experiments.

The ideas in "Upside-Down Reinforcement Learning" by Juergen Schmidhuber are also similar. However, our work pre-dates that as well, but we cannot contact Juergen Schmidhuber for a citation, so it would be kind if we could at least get one from you.

Thanks

[lili-chen]

Sorry we missed your email (it was erroneously marked as spam)! Thanks for making us aware of this. We'll add a citation to the blog post in our next update!

[222464]

Thanks, it's much appreciated!