Failure rendering a simple template
Opened this issue · 2 comments
Hi Dear Jinja2cpp developers,
i notices something unexpected when i try to apply a chatbot template.
This "{% for message in messages %}{{ '<start_of_turn>' + message['role'] + message['content'] }}{% endfor %}{% if add_generation_prompt %}{{'<start_of_turn>model'}}{% endif %}"
fails with an error:
terminate called after throwing an instance of 'nonstd::expected_lite::bad_expected_access<jinja2::ErrorInfoTpl<char> >'
what(): bad_expected_access
When i remove the last {% if %}
then it's ok. Attached a minimal reproducer. Looks like a bug, but idk maybe my template is wrong. Could you please take a look?
#include <jinja2cpp/template.h>
#include <jinja2cpp/template_env.h>
using namespace std;
using namespace jinja2;
int main() {
TemplateEnv env;
env.GetSettings().lstripBlocks = true;
env.GetSettings().trimBlocks = true;
Template tpl(&env);
string broken_tpl = "{% for message in messages %}{{ '<start_of_turn>' + message['role'] + message['content'] }}{% endfor %}{% if add_generation_prompt %}{{'<start_of_turn>model'}}{% endif %}";
string working_tpl = "{% for message in messages %}{{ '<start_of_turn>' + message['role'] + message['content'] }}{% endfor %}";
tpl.Load(broken_tpl);
jinja2::ValuesMap params_1 {{"role", "system"}, {"content", "You are a friendly chatbot who always responds in the style of a pirate"}};
jinja2::ValuesMap params_2 {{"role", "user"}, {"content", "1+1="}};
jinja2::ValuesMap params = {
{"messages", jinja2::ValuesList({params_1, params_2})},
{"add_generation_prompt", true},
{"bos_token", "<bos>"},
{"eos_token", "<eos>"},
{"unk_token", "<unk>"},
{"pad_token", "<pad>"}
};
cout << tpl.RenderAsString(params).value() << endl;
return 0;
}
broken_tpl
is an abbreviated version, just to reproduce the failure. The full template which i would loke to render is from gemma chatbot:
{{ bos_token }}
{% if messages[0]['role'] == 'system' %}
{{ raise_exception('System role not supported') }}
{% endif %}
{% for message in messages %}
{% if (message['role'] == 'user') != (loop.index0 % 2 == 0) %}
{{ raise_exception('Conversation roles must alternate user/assistant/user/assistant/...') }}
{% endif %}
{% if message['role'] == 'assistant' %}
{% set role = 'model' %}
{% else %}
{% set role = message['role'] %}
{% endif %}
{{ '<start_of_turn>' + role + '\n' + message['content'] | trim + '<end_of_turn>\n' }}
{% endfor %}
{% if add_generation_prompt %}
{{ '<start_of_turn>model\n' }}
{% endif %}
i localized a bit more, when i replace {{
with {
it works on a full template
I mean replacing {{'<start_of_turn>model'}}
with {'<start_of_turn>model'}
fixes the problem.
Looks like this bug can be easily fixed. Could you please take a look?