jordanpotti/CloudScraper

Do you need to install a parser library?

ehsandeep opened this issue · 3 comments

MacBook-Pro:CloudScraper user$ pip install -r requirements.txt 
Requirement already satisfied: requests==2.18.4 in /Library/Python/2.7/site-packages/requests-2.18.4-py2.7.egg (from -r requirements.txt (line 1)) (2.18.4)
Requirement already satisfied: BeautifulSoup4 in /Library/Python/2.7/site-packages/beautifulsoup4-4.6.0-py2.7.egg (from -r requirements.txt (line 2)) (4.6.0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /Library/Python/2.7/site-packages (from requests==2.18.4->-r requirements.txt (line 1)) (3.0.4)
Requirement already satisfied: idna<2.7,>=2.5 in /Library/Python/2.7/site-packages/idna-2.6-py2.7.egg (from requests==2.18.4->-r requirements.txt (line 1)) (2.6)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /Library/Python/2.7/site-packages/urllib3-1.22-py2.7.egg (from requests==2.18.4->-r requirements.txt (line 1)) (1.22)
Requirement already satisfied: certifi>=2017.4.17 in /Library/Python/2.7/site-packages (from requests==2.18.4->-r requirements.txt (line 1)) (2018.1.18)
You are using pip version 10.0.0, however version 10.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
MacBook-Pro:CloudScraper user$ python CloudScraper.py -u https://test.com

Description: 
        CloudScraper is a tool to search through the source code of websites in order to find cloud resources belonging to a target.
        by Jordan Potti
        @ok_bye_now

('Beginning search for cloud resources in ', 'https://test.com', '\n')
Traceback (most recent call last):
  File "CloudScraper.py", line 129, in <module>
    main()
  File "CloudScraper.py", line 127, in main
    start(arguments.URL, arguments.depth)
  File "CloudScraper.py", line 32, in start
    soup = BeautifulSoup(start_page.text, "lxml")
  File "/Library/Python/2.7/site-packages/beautifulsoup4-4.6.0-py2.7.egg/bs4/__init__.py", line 165, in __init__
bs4.FeatureNotFound: Couldn't find a tree builder with the features you requested: lxml. Do you need to install a parser library?
MacBook-Pro:CloudScraper user$ 

Hey there, try and run it with python3 and let me know if it works.

Hi,
same request, solved it by doing : sudo apt install python3-lxml

Thank you @Untuouchum, sudo apt install python3-lxml worked for me as well.