Issue with using @SystemMessage(fromResource=...) in dev mode.
Opened this issue · 9 comments
fmatar commented
I have moved my prompt content to a file named my_prompt.txt to resources/dev/langchain4j/services
My service looks as follows:
interface MyService {
@SystemMessage(fromResource = "my_prompt.txt")
String chat();
}The following error happens in dev mode only, I can get around it by building and running it in prod, not fun!
2024-05-23 09:36:27,410 ERROR [io.qua.dep.dev.IsolatedDevModeMain] (Aesh InputStream Reader) Failed to start quarkus: java.lang.RuntimeException: io.quarkus.builder.BuildException: Build failure: Build failed due to errors
[error]: Build step io.quarkiverse.langchain4j.deployment.devui.LangChain4jDevUIProcessor#cardPage threw an exception: java.lang.NullPointerException: Cannot invoke "org.jboss.jandex.AnnotationValue.asStringArray()" because the return value of "org.jboss.jandex.AnnotationInstance.value()" is null
at io.quarkiverse.langchain4j.deployment.devui.LangChain4jDevUIProcessor.lambda$addChatPage$4(LangChain4jDevUIProcessor.java:98)
at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:212)
at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:194)
at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1709)
at java.base/java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:782)
at java.base/java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:291)
at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:212)
at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1709)
at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:556)
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:546)
at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:622)
at java.base/java.util.stream.AbstractPipeline.evaluateToArrayNode(AbstractPipeline.java:291)
at java.base/java.util.stream.ReferencePipeline.toArray(ReferencePipeline.java:631)
at java.base/java.util.stream.ReferencePipeline.toArray(ReferencePipeline.java:637)
at java.base/java.util.stream.ReferencePipeline.toList(ReferencePipeline.java:642)
at io.quarkiverse.langchain4j.deployment.devui.LangChain4jDevUIProcessor.addChatPage(LangChain4jDevUIProcessor.java:99)
at io.quarkiverse.langchain4j.deployment.devui.LangChain4jDevUIProcessor.cardPage(LangChain4jDevUIProcessor.java:51)
at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
at java.base/java.lang.reflect.Method.invoke(Method.java:580)
at io.quarkus.deployment.ExtensionLoader$3.execute(ExtensionLoader.java:849)
at io.quarkus.builder.BuildContext.run(BuildContext.java:256)
at org.jboss.threads.ContextHandler$1.runWith(ContextHandler.java:18)
at org.jboss.threads.EnhancedQueueExecutor$Task.doRunWith(EnhancedQueueExecutor.java:2516)
at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2495)
at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1521)
at java.base/java.lang.Thread.run(Thread.java:1570)
at org.jboss.threads.JBossThread.run(JBossThread.java:483)
at io.quarkus.runner.bootstrap.AugmentActionImpl.runAugment(AugmentActionImpl.java:337)
at io.quarkus.runner.bootstrap.AugmentActionImpl.reloadExistingApplication(AugmentActionImpl.java:267)
at io.quarkus.runner.bootstrap.AugmentActionImpl.reloadExistingApplication(AugmentActionImpl.java:60)
at io.quarkus.deployment.dev.IsolatedDevModeMain.restartApp(IsolatedDevModeMain.java:190)
at io.quarkus.deployment.dev.IsolatedDevModeMain.restartCallback(IsolatedDevModeMain.java:173)
at io.quarkus.deployment.dev.RuntimeUpdatesProcessor.doScan(RuntimeUpdatesProcessor.java:542)
at io.quarkus.deployment.console.ConsoleStateManager.forceRestart(ConsoleStateManager.java:175)
at io.quarkus.deployment.console.ConsoleStateManager.lambda$installBuiltins$0(ConsoleStateManager.java:112)
at io.quarkus.deployment.console.ConsoleStateManager$1.accept(ConsoleStateManager.java:77)
at io.quarkus.deployment.console.ConsoleStateManager$1.accept(ConsoleStateManager.java:49)
at io.quarkus.deployment.console.AeshConsole.lambda$setup$1(AeshConsole.java:278)
at org.aesh.terminal.EventDecoder.accept(EventDecoder.java:118)
at org.aesh.terminal.EventDecoder.accept(EventDecoder.java:31)
at org.aesh.terminal.io.Decoder.write(Decoder.java:133)
at org.aesh.readline.tty.terminal.TerminalConnection.openBlocking(TerminalConnection.java:216)
at org.aesh.readline.tty.terminal.TerminalConnection.openBlocking(TerminalConnection.java:203)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
at java.base/java.lang.Thread.run(Thread.java:1570)
Caused by: io.quarkus.builder.BuildException: Build failure: Build failed due to errors
[error]: Build step io.quarkiverse.langchain4j.deployment.devui.LangChain4jDevUIProcessor#cardPage threw an exception: java.lang.NullPointerException: Cannot invoke "org.jboss.jandex.AnnotationValue.asStringArray()" because the return value of "org.jboss.jandex.AnnotationInstance.value()" is null
at io.quarkiverse.langchain4j.deployment.devui.LangChain4jDevUIProcessor.lambda$addChatPage$4(LangChain4jDevUIProcessor.java:98)
at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:212)
at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:194)
at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1709)
at java.base/java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:782)
at java.base/java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:291)
at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:212)
at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1709)
at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:556)
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:546)
at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:622)
at java.base/java.util.stream.AbstractPipeline.evaluateToArrayNode(AbstractPipeline.java:291)
at java.base/java.util.stream.ReferencePipeline.toArray(ReferencePipeline.java:631)
at java.base/java.util.stream.ReferencePipeline.toArray(ReferencePipeline.java:637)
at java.base/java.util.stream.ReferencePipeline.toList(ReferencePipeline.java:642)
at io.quarkiverse.langchain4j.deployment.devui.LangChain4jDevUIProcessor.addChatPage(LangChain4jDevUIProcessor.java:99)
at io.quarkiverse.langchain4j.deployment.devui.LangChain4jDevUIProcessor.cardPage(LangChain4jDevUIProcessor.java:51)
at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
at java.base/java.lang.reflect.Method.invoke(Method.java:580)
at io.quarkus.deployment.ExtensionLoader$3.execute(ExtensionLoader.java:849)
at io.quarkus.builder.BuildContext.run(BuildContext.java:256)
at org.jboss.threads.ContextHandler$1.runWith(ContextHandler.java:18)
at org.jboss.threads.EnhancedQueueExecutor$Task.doRunWith(EnhancedQueueExecutor.java:2516)
at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2495)
at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1521)
at java.base/java.lang.Thread.run(Thread.java:1570)
at org.jboss.threads.JBossThread.run(JBossThread.java:483)
at io.quarkus.builder.Execution.run(Execution.java:123)
at io.quarkus.builder.BuildExecutionBuilder.execute(BuildExecutionBuilder.java:79)
at io.quarkus.deployment.QuarkusAugmentor.run(QuarkusAugmentor.java:160)
at io.quarkus.runner.bootstrap.AugmentActionImpl.runAugment(AugmentActionImpl.java:333)
... 18 more
Caused by: java.lang.NullPointerException: Cannot invoke "org.jboss.jandex.AnnotationValue.asStringArray()" because the return value of "org.jboss.jandex.AnnotationInstance.value()" is null
at io.quarkiverse.langchain4j.deployment.devui.LangChain4jDevUIProcessor.lambda$addChatPage$4(LangChain4jDevUIProcessor.java:98)
at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipelin212)va:
at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:194)
at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1709)
at java.base/java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:782)
at java.base/java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:291)
at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:212)
at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1709)
at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:556)
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:546)
at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:622)
at java.base/java.util.stream.AbstractPipeline.evaluateToArrayNode(AbstractPipeline.java:291)
at java.base/java.util.stream.ReferencePipeline.toArray(ReferencePipeline.java:631)
at java.base/java.util.stream.ReferencePipeline.toArray(ReferencePipeline.java:637)
at java.base/java.util.stream.ReferencePipeline.toList(ReferencePipeline.java:642)
at io.quarkiverse.langchain4j.deployment.devui.LangChain4jDevUIProcessor.addChatPage(LangChain4jDevUIProcessor.java:99)
at io.quarkiverse.langchain4j.deployment.devui.LangChain4jDevUIProcessor.cardPage(LangChain4jDevUIProcessor.java:51)
at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
at java.base/java.lang.reflect.Method.invoke(Method.java:580)
at io.quarkus.deployment.ExtensionLoader$3.execute(ExtensionLoader.java:849)
at io.quarkus.builder.BuildContext.run(BuildContext.java:256)
at org.jboss.threads.ContextHandler$1.runWith(ContextHandler.java:18)
at org.jboss.threads.EnhancedQueueExecutor$Task.doRunWith(EnhancedQueueExecutor.java:2516)
at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2495)
at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1521)
at java.base/java.lang.Thread.run(Thread.java:1570)
at org.jboss.threads.JBossThread.run(JBossThread.java:483)
fmatar commented
Looking at the issue I think it's quite an easy fix:
The filtering could include the non null values. Possibly update
private void addChatPage(CardPageBuildItem card, List<DeclarativeAiServiceBuildItem> aiServices) {
List<String> systemMessages = aiServices.stream()
.map(s -> s.getServiceClassInfo())
.flatMap(c -> c.annotations().stream()) //This includes method annotations
.filter(a -> a.name().equals(LangChain4jDotNames.SYSTEM_MESSAGE))
.map(a -> String.join("", a.value().asStringArray()))
.toList();
...
}
to:
private void addChatPage(CardPageBuildItem card, List<DeclarativeAiServiceBuildItem> aiServices) {
List<String> systemMessages = aiServices.stream()
.map(s -> s.getServiceClassInfo())
.flatMap(c -> c.annotations().stream()) //This includes method annotations
.filter(a -> a.name().equals(LangChain4jDotNames.SYSTEM_MESSAGE) && a.value()!=null)
.map(a -> String.join("", a.value().asStringArray()))
.toList();
...
}
Thoughts?
gsmet commented
I think you would have to resolve the system message instead and reading it from the file. Not sure where it's done for runtime but you need something similar (or a way to propagate the value) in Dev UI.
fmatar commented
I'm looking a bit more into it, thank you for the suggestion
fmatar commented
found a super hacky way to make it work.
@SystemMessage(value="", fromResource="my_prompt.txt")
fromResource still overrides the value.
fmatar commented
That also uncovered another issue. In dev mode a message changed from an external file won't be changed unless I rebuild the app
jmartisk commented
That also uncovered another issue. In dev mode a message changed from an external file won't be changed unless I rebuild the app
Yeah the extension will have to declare that file to be watched for changes, this should be doable.
Sorry for the delay in answering, I'm traveling all week and on wifi that mostly doesn't work, but I'll have a look into this as soon as I can.
geoand commented
this should be doable.
Definitely doable and easy 😜