nv-morpheus/Morpheus

[BUG]: LLM Agents Kafka pipeline is broken

Closed this issue · 1 comments

dagardner-nv commented

Version

24.06

Which installation method(s) does this occur on?

Docker, Conda, Source

Describe the bug.

The pipeline has a few issues:

  • Kafka brokers is hard-coded to "auto" this should be exposed as a flag
  • Kafka input topic name should similarly be exposed as a flag
  • Pipeline has gone out of sync with the simple_pipeline
  • The _build_engine and _build_agent_executor methods should be shared with the simple pipeline
  • processing a question results in an exception from the agent chain

Minimum reproducible example

python examples/llm/main.py agents kafka

Relevant log output

Click here to see error details


Entering new AgentExecutor chain...

Error running agent: 'NoneType' object has no attribute 'create'

Traceback (most recent call last):

File "/home/dagardner/work/tmp/mrel2406/morpheus/llm/nodes/langchain_agent_node.py", line 101, in _run_single

return await self._agent_executor.arun(metadata=metadata, **kwargs)

File "/home/dagardner/work/conda/envs/mrel2406/lib/python3.10/site-packages/langchain_core/_api/deprecation.py", line 157, in awarning_emitting_wrapper

return await wrapped(*args, **kwargs)

File "/home/dagardner/work/conda/envs/mrel2406/lib/python3.10/site-packages/langchain/chains/base.py", line 651, in arun

await self.acall(

File "/home/dagardner/work/conda/envs/mrel2406/lib/python3.10/site-packages/langchain_core/_api/deprecation.py", line 157, in awarning_emitting_wrapper

return await wrapped(*args, **kwargs)

File "/home/dagardner/work/conda/envs/mrel2406/lib/python3.10/site-packages/langchain/chains/base.py", line 428, in acall

return await self.ainvoke(

File "/home/dagardner/work/conda/envs/mrel2406/lib/python3.10/site-packages/langchain/chains/base.py", line 212, in ainvoke

raise e

File "/home/dagardner/work/conda/envs/mrel2406/lib/python3.10/site-packages/langchain/chains/base.py", line 203, in ainvoke

await self._acall(inputs, run_manager=run_manager)

File "/home/dagardner/work/conda/envs/mrel2406/lib/python3.10/site-packages/langchain/agents/agent.py", line 1481, in _acall

next_step_output = await self._atake_next_step(

File "/home/dagardner/work/conda/envs/mrel2406/lib/python3.10/site-packages/langchain/agents/agent.py", line 1275, in _atake_next_step

[

File "/home/dagardner/work/conda/envs/mrel2406/lib/python3.10/site-packages/langchain/agents/agent.py", line 1275, in 

[

File "/home/dagardner/work/conda/envs/mrel2406/lib/python3.10/site-packages/langchain/agents/agent.py", line 1303, in _aiter_next_step

output = await self.agent.aplan(

File "/home/dagardner/work/conda/envs/mrel2406/lib/python3.10/site-packages/langchain/agents/agent.py", line 751, in aplan

full_output = await self.llm_chain.apredict(callbacks=callbacks, **full_inputs)

File "/home/dagardner/work/conda/envs/mrel2406/lib/python3.10/site-packages/langchain/chains/llm.py", line 310, in apredict

return (await self.acall(kwargs, callbacks=callbacks))[self.output_key]

File "/home/dagardner/work/conda/envs/mrel2406/lib/python3.10/site-packages/langchain_core/_api/deprecation.py", line 157, in awarning_emitting_wrapper

return await wrapped(*args, **kwargs)

File "/home/dagardner/work/conda/envs/mrel2406/lib/python3.10/site-packages/langchain/chains/base.py", line 428, in acall

return await self.ainvoke(

File "/home/dagardner/work/conda/envs/mrel2406/lib/python3.10/site-packages/langchain/chains/base.py", line 212, in ainvoke

raise e

File "/home/dagardner/work/conda/envs/mrel2406/lib/python3.10/site-packages/langchain/chains/base.py", line 203, in ainvoke

await self._acall(inputs, run_manager=run_manager)

File "/home/dagardner/work/conda/envs/mrel2406/lib/python3.10/site-packages/langchain/chains/llm.py", line 275, in _acall

response = await self.agenerate([inputs], run_manager=run_manager)

File "/home/dagardner/work/conda/envs/mrel2406/lib/python3.10/site-packages/langchain/chains/llm.py", line 142, in agenerate

return await self.llm.agenerate_prompt(

File "/home/dagardner/work/conda/envs/mrel2406/lib/python3.10/site-packages/langchain_core/language_models/llms.py", line 643, in agenerate_prompt

return await self.agenerate(

File "/home/dagardner/work/conda/envs/mrel2406/lib/python3.10/site-packages/langchain_core/language_models/llms.py", line 1018, in agenerate

output = await self._agenerate_helper(

File "/home/dagardner/work/conda/envs/mrel2406/lib/python3.10/site-packages/langchain_core/language_models/llms.py", line 882, in _agenerate_helper

raise e

File "/home/dagardner/work/conda/envs/mrel2406/lib/python3.10/site-packages/langchain_core/language_models/llms.py", line 866, in _agenerate_helper

await self._agenerate(

File "/home/dagardner/work/conda/envs/mrel2406/lib/python3.10/site-packages/langchain_community/llms/openai.py", line 1194, in _agenerate

full_response = await acompletion_with_retry(

File "/home/dagardner/work/conda/envs/mrel2406/lib/python3.10/site-packages/langchain_community/llms/openai.py", line 133, in acompletion_with_retry

return await llm.async_client.create(**kwargs)

AttributeError: 'NoneType' object has no attribute 'create'

Fatal Python error: PyThreadState_Get: the function must be called with the GIL held, but the GIL is released (the current Python thread state is NULL)

Python runtime state: initialized




Thread 0x00007fb410ffe6c0 (most recent call first):

File "/home/dagardner/work/conda/envs/mrel2406/lib/python3.10/concurrent/futures/thread.py", line 81 in _worker

File "/home/dagardner/work/conda/envs/mrel2406/lib/python3.10/threading.py", line 953 in run

File "/home/dagardner/work/conda/envs/mrel2406/lib/python3.10/threading.py", line 1016 in _bootstrap_inner

File "/home/dagardner/work/conda/envs/mrel2406/lib/python3.10/threading.py", line 973 in _bootstrap

Full env printout

Click here to see environment details
[Paste the results of print_env.sh here, it will be hidden by default]

Other/Misc.

No response

Code of Conduct

  • I agree to follow Morpheus' Code of Conduct
  • I have searched the open bugs and have found no duplicates for this bug report
dagardner-nv commented

Once fixed, remove this pipeline from docs/source/extra_info/known_issues.md doc and remove the warning in examples/llm/agents/README.md