SystemMessage too large.
Opened this issue · 4 comments
@SystemMessage(value = "", fromResource = "large_prompt.txt")
String chat(@MemoryId String cid, @UserMessage String userMessage);
If the file is large (mine is around 87kb) I get the following exception.
io.vertx.core.impl.NoStackTraceException
Caused by: java.lang.RuntimeException: io.quarkus.builder.BuildException: Build failure: Build failed due to errors
[error]: Build step io.quarkus.deployment.steps.MainClassBuildStep#build threw an exception: java.lang.RuntimeException: Failed to record call to method public void io.quarkiverse.langchain4j.runtime.AiServicesRecorder.setMetadata(java.util.Map)
at io.quarkus.deployment.recording.BytecodeRecorderImpl.writeBytecode(BytecodeRecorderImpl.java:480)
at io.quarkus.deployment.steps.MainClassBuildStep.writeRecordedBytecode(MainClassBuildStep.java:501)
at io.quarkus.deployment.steps.MainClassBuildStep.build(MainClassBuildStep.java:201)
at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
at java.base/java.lang.reflect.Method.invoke(Method.java:580)
at io.quarkus.deployment.ExtensionLoader$3.execute(ExtensionLoader.java:849)
at io.quarkus.builder.BuildContext.run(BuildContext.java:256)
at org.jboss.threads.ContextHandler$1.runWith(ContextHandler.java:18)
at org.jboss.threads.EnhancedQueueExecutor$Task.doRunWith(EnhancedQueueExecutor.java:2516)
at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2495)
at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1521)
at java.base/java.lang.Thread.run(Thread.java:1570)
at org.jboss.threads.JBossThread.run(JBossThread.java:483)
Caused by: java.lang.RuntimeException: java.lang.RuntimeException: java.lang.RuntimeException: java.lang.RuntimeException: String too large to record:
We should fix this, but I'll just note that using such a huge prompt as a system or user message is almost certainly not a good idea
Why is this being recorded at all, shouldn't the system message be overridable at runtime?
I got around it by setting the SystemPrompt to a variable.
@SystemMessage="{system_prompt}"
And then in my @PostConstruct I'm setting the prompt as a global variable.
Not the usual approach for doing thing however this specific prompt is dictated by a massive schema definition.
Why is this being recorded at all, shouldn't the system message be overridable at runtime?
No, it's not