Even a simplest request with schema doesn't work

Question

Even a simplest request with schema doesn't work

Closed this issue a month ago · 10 comments

import json

from pydantic import BaseModel
from scrapegraphai.graphs import SmartScraperGraph

# Define the configuration for the scraping pipeline
graph_config = {
    "llm": {
        "api_key": "sk-key",
        "model": "openai/gpt-4o-mini",
    },
    "verbose": True,
    "headless": True,
}

class CompanyDetails(BaseModel):
    name: str
    description: str
    email: str


# Create the SmartScraperGraph instance
smart_scraper_graph = SmartScraperGraph(
    prompt="Find some information about what does the company do, the name and a contact email.",
    source="https://theraiderteam.com/",
    config=graph_config,
    schema=CompanyDetails,
)

# Run the pipeline
result = smart_scraper_graph.run()
print(json.dumps(result, indent=4))

Fails miserably with

Traceback (most recent call last):
  File "/..../test.py", line 31, in <module>
    result = smart_scraper_graph.run()
             ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/..../.venv/lib/python3.11/site-packages/scrapegraphai/graphs/smart_scraper_graph.py", line 212, in run
    self.final_state, self.execution_info = self.graph.execute(inputs)
                                            ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/..../.venv/lib/python3.11/site-packages/scrapegraphai/graphs/base_graph.py", line 327, in execute
    return self._execute_standard(initial_state)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/..../.venv/lib/python3.11/site-packages/scrapegraphai/graphs/base_graph.py", line 274, in _execute_standard
    raise e
  File "/..../.venv/lib/python3.11/site-packages/scrapegraphai/graphs/base_graph.py", line 247, in _execute_standard
    result, node_exec_time, cb_data = self._execute_node(
                                      ^^^^^^^^^^^^^^^^^^^
  File "/..../.venv/lib/python3.11/site-packages/scrapegraphai/graphs/base_graph.py", line 172, in _execute_node
    result = current_node.execute(state)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/..../.venv/lib/python3.11/site-packages/scrapegraphai/nodes/generate_answer_node.py", line 129, in execute
    answer = output_parser.parse(raw_response.content)
             ^^^^^^^^^^^^^^^^^^^
AttributeError: 'function' object has no attribute 'parse'

Answer 1 · 2024-11-05T03:08:33.000Z

44510709 commented 2 months ago

😕1

Answer 2 · 2024-11-05T03:10:42.000Z

"Has it been resolved?" How to solve it

Answer 3 · 2024-11-06T08:26:11.000Z

does it work the test example?

Answer 4 · 2024-11-08T11:43:09.000Z

@pkit Same issue, it seems the last working version was 1.27.0.
The error is here: https://github.com/ScrapeGraphAI/Scrapegraph-ai/blame/b400cc5a666e1670948cd50f284a2ccba3d136ee/scrapegraphai/nodes/generate_answer_node.py#L84
This line returns a function, whereas other code path return an object with a parse method.
also, it seems that's incorrect for Mistral, even fixing the bug doesn't help:

  File "/venv/lib/python3.12/site-packages/scrapegraphai/nodes/generate_answer_node.py", line 87, in execute
    raise NotImplementedError("Structured output is not implemented for ChatOpenAI")
NotImplementedError: Structured output is not implemented for ChatOpenAI

Answer 5 · 2024-11-08T11:49:48.000Z

Can you make the pull request please?

Answer 6 · 2024-11-09T10:59:38.000Z

@bezineb5 just updated. please upload to the new version

Answer 7 · 2024-11-15T15:51:29.000Z

Is this solved in 1.30.0? because I get the same error 🫤.

Answer 8 · 2024-11-16T10:37:52.000Z

I got the same error with the schema yesterday when using the new version.

Answer 9 · 2024-11-16T13:16:39.000Z

I downgraded to 1.27.0, it works there =)

Answer 10 · 2024-11-16T15:31:49.000Z

please try with the new beta