Sweep: Using the Llama 3:8b model, checking internet speed with Crew and Ollama throws an exception
melandz opened this issue · 5 comments
Details
Hi,
I switched to using the Llama 3:8b model in the example "Checking internet speed with Crew and Ollam", and it threw the following exception. How should I handle this? Thank you.
Begin! This is VERY important to you, your job depends on it!
Current Task: What is the ping to Google's main DNS server?
This is the context you are working with:
Thought: Do I need to use a tool? Yes
Action: ping
Action Input: 8.8.8.8
Observation: PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
64 bytes from 8.8.8.8: icmp_seq=1 ttl=108 time=6.12 ms
64 bytes from 8.8.8.8: icmp_seq=2 ttl=108 time=4.21 ms
64 bytes from 8.8.8.8: icmp_seq=3 ttl=108 time=4.42 ms
64 bytes from 8.8.8.8: icmp_seq=4 ttl=108 time=5.46 ms
64 bytes from 8.8.8.8: icmp_seq=5 ttl=108 time=4.66 ms
--- 8.8.8.8 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4006ms
rtt min/avg/max/mdev = 4.209/4.973/6.120/0.713 ms
Thought: Do I need to use a tool? No
Final Answer: The connection speed is 4.973 milliseconds.
Here's the solution:
```python
# Thought: Do I need to use a tool? Yes
Action: ping
Action Input: 8.8.8.8
Unhandled exception. LangChain.Chains.HelperChains.Exceptions.StackableChainException: Error occured in ReActParserChain with inputs
text= Do I need to use a tool? No
Final Answer: The connection speed is 4.973 milliseconds.
Here's the solution:
# Thought: Do I need to use a tool? Yes
Action: ping
Action Input: 8.8.8.8
.
---> LangChain.Schema.OutputParserException: Parsing LLM output produced both a final answer and a parse-able action:: Do I need to use a tool? No
Final Answer: The connection speed is 4.973 milliseconds.
Here's the solution:
# Thought: Do I need to use a tool? Yes
Action: ping
Action Input: 8.8.8.8
at LangChain.Chains.StackableChains.ReAct.ReActParserChain.Parse(String text) in /_/src/Core/src/Chains/StackableChains/ReAct/ReActParserChain.cs:line 47
at LangChain.Chains.StackableChains.ReAct.ReActParserChain.InternalCall(IChainValues values) in /_/src/Core/src/Chains/StackableChains/ReAct/ReActParserChain.cs:line 78
at LangChain.Chains.HelperChains.BaseStackableChain.CallAsync(IChainValues values, ICallbacks callbacks, IReadOnlyList`1 tags, IReadOnlyDictionary`2 metadata) in /_/src/Core/src/Chains/StackableChains/BaseStackableChain.cs:line 99
--- End of inner exception stack trace ---
at LangChain.Chains.HelperChains.BaseStackableChain.CallAsync(IChainValues values, ICallbacks callbacks, IReadOnlyList`1 tags, IReadOnlyDictionary`2 metadata) in /_/src/Core/src/Chains/StackableChains/BaseStackableChain.cs:line 118
at LangChain.Chains.HelperChains.StackChain.InternalCall(IChainValues values) in /_/src/Core/src/Chains/StackableChains/StackChain.cs:line 95
at LangChain.Chains.HelperChains.StackChain.InternalCall(IChainValues values) in /_/src/Core/src/Chains/StackableChains/StackChain.cs:line 82
at LangChain.Chains.HelperChains.StackChain.InternalCall(IChainValues values) in /_/src/Core/src/Chains/StackableChains/StackChain.cs:line 95
at LangChain.Chains.HelperChains.BaseStackableChain.Run[T](String resultKey, StackableChainHook hook) in /_/src/Core/src/Chains/StackableChains/BaseStackableChain.cs:line 193
at LangChain.Chains.StackableChains.Agents.Crew.CrewAgent.InternalCall(IChainValues values) in /_/src/Core/src/Chains/StackableChains/Agents/Crew/CrewAgent.cs:line 196
at LangChain.Chains.HelperChains.StackChain.InternalCall(IChainValues values) in /_/src/Core/src/Chains/StackableChains/StackChain.cs:line 95
at LangChain.Chains.HelperChains.BaseStackableChain.Run(String resultKey, StackableChainHook hook) in /_/src/Core/src/Chains/StackableChains/BaseStackableChain.cs:line 174
at LangChain.Chains.StackableChains.Agents.Crew.Tools.AskQuestionTool.ToolTask(String input, CancellationToken token) in /_/src/Core/src/Chains/StackableChains/Agents/Crew/Tools/AskQuestionTool.cs:line 41
at LangChain.Chains.StackableChains.Agents.Crew.CrewAgent.InternalCall(IChainValues values) in /_/src/Core/src/Chains/StackableChains/Agents/Crew/CrewAgent.cs:line 210
at LangChain.Chains.HelperChains.StackChain.InternalCall(IChainValues values) in /_/src/Core/src/Chains/StackableChains/StackChain.cs:line 95
at LangChain.Chains.HelperChains.BaseStackableChain.Run(String resultKey, StackableChainHook hook) in /_/src/Core/src/Chains/StackableChains/BaseStackableChain.cs:line 174
at LangChain.Chains.StackableChains.Agents.Crew.AgentTask.ExecuteAsync(String context, CancellationToken cancellationToken) in /_/src/Core/src/Chains/StackableChains/Agents/Crew/AgentTask.cs:line 45
at LangChain.Chains.StackableChains.Agents.Crew.CrewChain.InternalCall(IChainValues values) in /_/src/Core/src/Chains/StackableChains/Agents/Crew/CrewChain.cs:line 45
at LangChain.Chains.HelperChains.StackChain.InternalCall(IChainValues values) in /_/src/Core/src/Chains/StackableChains/StackChain.cs:line 95
at LangChain.Chains.HelperChains.BaseStackableChain.Run(String resultKey, StackableChainHook hook) in /_/src/Core/src/Chains/StackableChains/BaseStackableChain.cs:line 174
at Program.<Main>$(String[] args) in /home/melandz/projects/blazorTest/Program.cs:line 53
at Program.<Main>(String[] args)
### Branch
_No response_
<details open>
<summary>Checklist</summary>
- [X] Modify `src/Core/src/Chains/StackableChains/ReAct/ReActParserChain.cs` ✓ https://github.com/tryAGI/LangChain/commit/5870abdc36cd5b2e60bc203852c2fcc938ccae73 [Edit](https://github.com/tryAGI/LangChain/edit/sweep/using_the_llama_38b_model_checking_inter/src/Core/src/Chains/StackableChains/ReAct/ReActParserChain.cs)
</details>
Got the same results. The llama3 trending to explain things a lot. So, when the final answer is given, it quotes itself, which may break the parser. Also, this model may have tags it it's response, so they must be added to stop words.
@melandz can you try this:
- add
<|eot_id|>
tag to stop words:
var model = new OllamaLanguageModelInstruction("llama3:8b",
"http://localhost:11434",
options: new OllamaLanguageModelOptions()
{
Stop = new[] { "Observation", "[END]", "<|eot_id|>" }, // add <|eot_id|> here
Temperature = 0
}).UseConsoleForDebug();
- explain the manager that his answers need to be short
var manager = new CrewAgent(model, "manager", "assign task to one of your co-workers and return the result. your answers are always very short.");
this solved the problem for me
Thank you, this method can be successful, but there is a 50% chance it will fail. Llama3 indeed provides extensive explanations. We may need to adjust the prompt content.
Human: You are manager.
Your personal goal is: assign task to one of your co-workers and return the result. your answers are always very short.
You have access to the following tools:
-
question, Useful to ask a question, opinion or take from on
of the following co-workers: ['pinger'].
The input to this tool should be a pipe (|) separated text of length
three, representing the co-worker you want to ask a question to,
the question and all actual context you have for the question.
For example,coworker|question|context
. -
delegate, Useful to delegate a specific task to one of the
following co-workers: ['pinger'].
The input to this tool should be a pipe (|) separated text of length
three, representing the co-worker you want to assign to,
the task and all actual context you have for the task.
For example,coworker|task|context
To use a tool, please use the exact following format:
Thought: Do I need to use a tool? Yes
Action: the action to take, should be one of [question, delegate], just the name.
Action Input: the input to the action
Observation: the result of the action
When you have a response for your task, or if you do not need to use a tool, you MUST use the format:
Thought: Do I need to use a tool? No
Final Answer: [your response here. should be a simple sentence without explanations.]
Begin! This is VERY important to you, your job depends on it!
Current Task: What is my ping to google's main dns server?
Thought: Do I need to use a tool? Yes
Action: question
Action Input: pinger|What is my ping to google's main dns server?|none
Human: You are pinger.
you using ping command and analyzing it's result. After this you print a single number as your final answer(the connection speed in milliseconds).
Your personal goal is: checks the speed of internet connection
You have access to the following tools:
- ping, executes ping on specified ip address
To use a tool, please use the exact following format:
Thought: Do I need to use a tool? Yes
Action: the action to take, should be one of [ping], just the name.
Action Input: the input to the action
Observation: the result of the action
When you have a response for your task, or if you do not need to use a tool, you MUST use the format:
Thought: Do I need to use a tool? No
Final Answer: [your response here. should be a simple sentence without explanations.]
Begin! This is VERY important to you, your job depends on it!
Current Task: What is my ping to google's main dns server?
This is the context you are working with:
none
Thought: Do I need to use a tool? Yes
Action: ping
Action Input: 8.8.8.8
Human: You are pinger.
you using ping command and analyzing it's result. After this you print a single number as your final answer(the connection speed in milliseconds).
Your personal goal is: checks the speed of internet connection
You have access to the following tools:
- ping, executes ping on specified ip address
To use a tool, please use the exact following format:
Thought: Do I need to use a tool? Yes
Action: the action to take, should be one of [ping], just the name.
Action Input: the input to the action
Observation: the result of the action
When you have a response for your task, or if you do not need to use a tool, you MUST use the format:
Thought: Do I need to use a tool? No
Final Answer: [your response here. should be a simple sentence without explanations.]
Begin! This is VERY important to you, your job depends on it!
Current Task: What is my ping to google's main dns server?
This is the context you are working with:
none
Thought: Do I need to use a tool? Yes
Action: ping
Action Input: 8.8.8.8
Observation: PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
64 bytes from 8.8.8.8: icmp_seq=1 ttl=108 time=4.47 ms
64 bytes from 8.8.8.8: icmp_seq=2 ttl=108 time=3.86 ms
64 bytes from 8.8.8.8: icmp_seq=3 ttl=108 time=5.92 ms
64 bytes from 8.8.8.8: icmp_seq=4 ttl=108 time=4.25 ms
64 bytes from 8.8.8.8: icmp_seq=5 ttl=108 time=8.77 ms
--- 8.8.8.8 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4006ms
rtt min/avg/max/mdev = 3.855/5.455/8.773/1.799 ms
Thought: Do I need to use a tool? No
Final Answer: The connection speed is 5.46 milliseconds.
This problem can be solved by using the `ping` command and analyzing its output. The final answer is the average of the round-trip times (RTTs) in the output, which represents the connection speed.
Here's a breakdown of how to solve this problem:
1. Use the `ping` command with the Google DNS server IP address (`8.8.8.8`) as input.
2. Analyze the output of the `ping` command, which includes the RTTs for each packet.
3. Calculate the average RTT by summing up all the RTTs and dividing by the number of packets received.
In this case, the output shows 5 packets transmitted with 5 received, and the RTTs are:
* 4.47 ms
* 3.86 ms
* 5.92 ms
* 4.25 ms
* 8.77 ms
The average RTT is calculated as follows:
(4.47 + 3.86 + 5.92 + 4.25 + 8.77) / 5 = 5.46 ms
Therefore, the final answer is: The connection speed is 5.46 milliseconds.Human: You are manager.
Your personal goal is: assign task to one of your co-workers and return the result. your answers are always very short.
You have access to the following tools:
- question, Useful to ask a question, opinion or take from on
of the following co-workers: ['pinger'].
The input to this tool should be a pipe (|) separated text of length
three, representing the co-worker you want to ask a question to,
the question and all actual context you have for the question.
For example, `coworker|question|context`.
- delegate, Useful to delegate a specific task to one of the
following co-workers: ['pinger'].
The input to this tool should be a pipe (|) separated text of length
three, representing the co-worker you want to assign to,
the task and all actual context you have for the task.
For example, `coworker|task|context`
To use a tool, please use the exact following format:
Thought: Do I need to use a tool? Yes
Action: the action to take, should be one of [question, delegate], just the name.
Action Input: the input to the action
Observation: the result of the action
When you have a response for your task, or if you do not need to use a tool, you MUST use the format:
Thought: Do I need to use a tool? No
Final Answer: [your response here. should be a simple sentence without explanations.]
Begin! This is VERY important to you, your job depends on it!
Current Task: What is my ping to google's main dns server?
Thought: Do I need to use a tool? Yes
Action: question
Action Input: pinger|What is my ping to google's main dns server?|none
Observation: The connection speed is 5.46 milliseconds.
This problem can be solved by using the ping
command and analyzing its output. The final answer is the average of the round-trip times (RTTs) in the output, which represents the connection speed.
Here's a breakdown of how to solve this problem:
- Use the
ping
command with the Google DNS server IP address (8.8.8.8
) as input. - Analyze the output of the
ping
command, which includes the RTTs for each packet. - Calculate the average RTT by summing up all the RTTs and dividing by the number of packets received.
In this case, the output shows 5 packets transmitted with 5 received, and the RTTs are:
- 4.47 ms
- 3.86 ms
- 5.92 ms
- 4.25 ms
- 8.77 ms
The average RTT is calculated as follows:
(4.47 + 3.86 + 5.92 + 4.25 + 8.77) / 5 = 5.46 ms
Therefore, the final answer is: The connection speed is 5.46 milliseconds.
Thought: Do I need to use a tool? No
Final Answer: The connection speed is 5.46 milliseconds.
This solution uses the `ping` command and analyzes its output to calculate the average RTT, which represents the connection speed. It does not require any additional tools or co-workers.The connection speed is 5.46 milliseconds.
This solution uses the ping
command and analyzes its output to calculate the average RTT, which represents the connection speed. It does not require any additional tools or co-workers.
I use the prompt below, and it works well:
var manager = new CrewAgent(model, "manager", "assign task to one of your co-workers and return the result. you should add [END] tag at the end of your final answer.");
To avoid null values, the method call has been modified as follows:
string? res = null;
int count = 0;
while(string.IsNullOrEmpty(res))
{
++count;
try
{
var chain =
Set("What is my ping to google's main dns server?")
|Crew(new []{manager,pinger},manager);
res = await chain.Run("text");
res = res?.Trim();
}
catch (Exception e)
{
Console.WriteLine(e);
if (count > 10) break;
continue;
}
}
Console.WriteLine(res);
Human: You are manager.
Your personal goal is: assign task to one of your co-workers and return the result. you should add [END] tag at the end of your final answer.
You have access to the following tools:
-
question, Useful to ask a question, opinion or take from on
of the following co-workers: ['pinger'].
The input to this tool should be a pipe (|) separated text of length
three, representing the co-worker you want to ask a question to,
the question and all actual context you have for the question.
For example,coworker|question|context
. -
delegate, Useful to delegate a specific task to one of the
following co-workers: ['pinger'].
The input to this tool should be a pipe (|) separated text of length
three, representing the co-worker you want to assign to,
the task and all actual context you have for the task.
For example,coworker|task|context
To use a tool, please use the exact following format:
Thought: Do I need to use a tool? Yes
Action: the action to take, should be one of [question, delegate], just the name.
Action Input: the input to the action
Observation: the result of the action
When you have a response for your task, or if you do not need to use a tool, you MUST use the format:
Thought: Do I need to use a tool? No
Final Answer: [your response here. should be a simple sentence without explanations.]
Begin! This is VERY important to you, your job depends on it!
Current Task: What is my ping to google's main dns server?
Thought: Do I need to use a tool? Yes
Action: question
Action Input: pinger|google|main dns server
Human: You are pinger.
you using ping command and analyzing it's result. After this you print a single number as your final answer(the connection speed in milliseconds).
Your personal goal is: checks the speed of internet connection
You have access to the following tools:
- ping, executes ping on specified ip address
To use a tool, please use the exact following format:
Thought: Do I need to use a tool? Yes
Action: the action to take, should be one of [ping], just the name.
Action Input: the input to the action
Observation: the result of the action
When you have a response for your task, or if you do not need to use a tool, you MUST use the format:
Thought: Do I need to use a tool? No
Final Answer: [your response here. should be a simple sentence without explanations.]
Begin! This is VERY important to you, your job depends on it!
Current Task: google
This is the context you are working with:
main dns server
Thought: Do I need to use a tool? Yes
Action: ping
Action Input: 8.8.8.8
Human: You are pinger.
you using ping command and analyzing it's result. After this you print a single number as your final answer(the connection speed in milliseconds).
Your personal goal is: checks the speed of internet connection
You have access to the following tools:
- ping, executes ping on specified ip address
To use a tool, please use the exact following format:
Thought: Do I need to use a tool? Yes
Action: the action to take, should be one of [ping], just the name.
Action Input: the input to the action
Observation: the result of the action
When you have a response for your task, or if you do not need to use a tool, you MUST use the format:
Thought: Do I need to use a tool? No
Final Answer: [your response here. should be a simple sentence without explanations.]
Begin! This is VERY important to you, your job depends on it!
Current Task: google
This is the context you are working with:
main dns server
Thought: Do I need to use a tool? Yes
Action: ping
Action Input: 8.8.8.8
Observation:
Ping 8.8.8.8 (使用 32 位元組的資料):
回覆自 8.8.8.8: 位元組=32 時間=4ms TTL=116
回覆自 8.8.8.8: 位元組=32 時間=4ms TTL=116
回覆自 8.8.8.8: 位元組=32 時間=3ms TTL=116
回覆自 8.8.8.8: 位元組=32 時間=3ms TTL=116
回覆自 8.8.8.8: 位元組=32 時間=4ms TTL=116
8.8.8.8 的 Ping 統計資料:
封包: 已傳送 = 5,已收到 = 5, 已遺失 = 0 (0% 遺失),
大約的來回時間 (毫秒):
最小值 = 3ms,最大值 = 4ms,平均 = 3ms
Thought: Do I need to use a tool? No
Final Answer: The connection speed is 3 milliseconds.Human: You are manager.
Your personal goal is: assign task to one of your co-workers and return the result. you should add [END] tag at the end of your final answer.
You have access to the following tools:
-
question, Useful to ask a question, opinion or take from on
of the following co-workers: ['pinger'].
The input to this tool should be a pipe (|) separated text of length
three, representing the co-worker you want to ask a question to,
the question and all actual context you have for the question.
For example,coworker|question|context
. -
delegate, Useful to delegate a specific task to one of the
following co-workers: ['pinger'].
The input to this tool should be a pipe (|) separated text of length
three, representing the co-worker you want to assign to,
the task and all actual context you have for the task.
For example,coworker|task|context
To use a tool, please use the exact following format:
Thought: Do I need to use a tool? Yes
Action: the action to take, should be one of [question, delegate], just the name.
Action Input: the input to the action
Observation: the result of the action
When you have a response for your task, or if you do not need to use a tool, you MUST use the format:
Thought: Do I need to use a tool? No
Final Answer: [your response here. should be a simple sentence without explanations.]
Begin! This is VERY important to you, your job depends on it!
Current Task: What is my ping to google's main dns server?
Thought: Do I need to use a tool? Yes
Action: question
Action Input: pinger|google|main dns server
Observation: The connection speed is 3 milliseconds.
Thought: Do I need to use a tool? No
Final Answer: My ping to google's main DNS server is 3 milliseconds.
My ping to google's main DNS server is 3 milliseconds.