huggingface/chat-ui

Standardize Chat Prompt Templates to Use Jinja Format

calycekr opened this issue · 8 comments

calycekr commented

Describe your feature request

Currently, the chatPromptTemplate for each model that can be set in env uses Handlebars format. However, the chat_prompt in the actual model's tokenizer_config.json uses Jinja format. This inconsistency is causing significant inconvenience. Since Jinja is widely used and preferred, it would be beneficial to standardize on Jinja format for both chatPromptTemplate and chat_prompt. This will improve consistency and ease of use for developers.

Screenshots (if relevant)

Implementation idea

To implement this change, the following steps can be taken:

  1. Update Codebase: Update the codebase to handle Jinja templates for chatPromptTemplate.

  2. Documentation: Update the documentation to reflect this change and provide examples of how to use Jinja templates.

  3. Testing: Thoroughly test the changes to ensure compatibility and that all existing templates work correctly with the new format.

nsarrazin commented

This is a good idea! It would be a breaking change though so we could keep handlebars as a fallback if the jinja evaluator fails, wdyt?

calycekr commented

@nsarrazin Sure, I agree with your suggestion. Keeping Handlebars as a fallback in case the Jinja evaluator fails is a good approach to ensure backward compatibility and smooth transition.

julien-c commented

yep, seems like the field has standardized on Jinja in the past year 😁 cc @Rocketknight1 😮

Rocketknight1 commented

Please don't fire me, I know you liked Handlebars! 😰

(but yes transitioning to Jinja is definitely the right decision)

julien-c commented

cements that we're a Python-first company (and community) i guess 😁

varad0309 commented

hey guys, did you add this feature?

nsarrazin commented

@varad0309 the recommended way for now would be to set "tokenizer": to point to the model's repo on the hub so it can pick up the chat template (using Jinja) there.

varad0309 commented

@nsarrazin , thanks for the fast reply. Is there a way to do the same if we have a local model?
also, just tried "tokenizer" : Qwen/Qwen2.5-Coder-14B-Instruct. It is not working.