ianarawjo/ChainForge

100% Racket Code Generation using "Dehumanized Prompts"

kfern opened this issue · 2 comments

Hello,

I want to share a 100% success rate on "Measure impact of system message on response" example by applying the "Dehumanize Prompt" technique found at https://github.com/kfern/dehumanize-prompt

You can view the working result directly here: https://chainforge.ai/play/?f=37wlj5rychogs

To reproduce the result, follow these steps:

  1. Go to the Chainforge.ai playground: https://chainforge.ai/play/
  2. Click on the "Example Flows" button.
  3. Select the "Measure impact of system message on response" example.
  4. Add a new OpenAI > GPT 3.5 Model with the following details:
    • Nickname: dehumanize-prompt
    • Model Version: gpt-3.5-turbo (default)
  5. Paste the following dehumanized system message prompt:
You are a software generator machine that creates programs in the Racket programming language. 
Your response has to be **CODE ONLY**, because it will be used by a machine that only knows this syntax.
  1. Click the "Submit" button with the default values.
  2. Run the "Prompt Node".
  3. Run the "JavaScript Evaluator".

image

We can also go one step further and remove from the acceptance criteria that the code has a single text response that looks like a codeblock https://chainforge.ai/play/?f=3blmo5an97ok0

image

I can do a PR if you want to update this example :-)

Sure, please add this to the example and make a PR and I will update it! Great idea!! :)