cp2004/Octoprint-Upgrade-To-Py3

Installing octoprint fails due to "ValueError: HTML doctype missing or incorrect. Expected <!DOCTYPE html>."

photon-schiesser opened this issue · 5 comments

Here is the output when running the upgrade script on my pi3.

pi@octopi:~ $ python3 upgrade.py 
OctoPrint Upgrade to Py 3 (v2.2.2)

This script will move your existing OctoPrint configuration from Python 2 to Python 3
This script requires an internet connection and it will disrupt any ongoing print jobs.

It will install the latest version of OctoPrint and all plugins.
No configuration or other files will be overwritten

Press [enter] to continue or ctrl-c to quit

Checking system info...
Detected OctoPi version 0.17.0
Found version: Python 2.7.16
Checking OctoPrint version...
OctoPrint version: 1.7.3
Reading installed plugins...

Plugins installed
- Rename Flowspeed & Feedrate
- Macro
- Bed Leveling Wizard
- TouchUI
If you think there is something missing from here, please check the list of plugins in OctoPrint
Press [enter] to continue or ctrl-c to quit

Updating package list...
Checking package list for python3-dev
python3-dev is already installed
Checking package list for python3-venv
python3-venv is already installed
Creating new Python 3 environment...
Successfully created Python 3 environment at /home/pi/oprint
Installing build dependencies...
Updating pip...
Cache entry deserialization failed, entry ignored
Cache entry deserialization failed, entry ignored

Installing OctoPrint... (This may take a while - Do not cancel!)
ERROR: Exception:
Traceback (most recent call last):
  File "/home/pi/oprint/lib/python3.7/site-packages/pip/_internal/cli/base_command.py", line 165, in exc_logging_wrapper
    status = run_func(*args)
  File "/home/pi/oprint/lib/python3.7/site-packages/pip/_internal/cli/req_command.py", line 205, in wrapper
    return func(self, options, args)
  File "/home/pi/oprint/lib/python3.7/site-packages/pip/_internal/commands/install.py", line 340, in run
    reqs, check_supported_wheels=not options.target_dir
  File "/home/pi/oprint/lib/python3.7/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 95, in resolve
    collected.requirements, max_rounds=try_to_avoid_resolution_too_deep
  File "/home/pi/oprint/lib/python3.7/site-packages/pip/_vendor/resolvelib/resolvers.py", line 481, in resolve
    state = resolution.resolve(requirements, max_rounds=max_rounds)
  File "/home/pi/oprint/lib/python3.7/site-packages/pip/_vendor/resolvelib/resolvers.py", line 348, in resolve
    self._add_to_criteria(self.state.criteria, r, parent=None)
  File "/home/pi/oprint/lib/python3.7/site-packages/pip/_vendor/resolvelib/resolvers.py", line 172, in _add_to_criteria
    if not criterion.candidates:
  File "/home/pi/oprint/lib/python3.7/site-packages/pip/_vendor/resolvelib/structs.py", line 151, in __bool__
    return bool(self._sequence)
  File "/home/pi/oprint/lib/python3.7/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 155, in __bool__
    return any(self)
  File "/home/pi/oprint/lib/python3.7/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 143, in <genexpr>
    return (c for c in iterator if id(c) not in self._incompatible_ids)
  File "/home/pi/oprint/lib/python3.7/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 44, in _iter_built
    for version, func in infos:
  File "/home/pi/oprint/lib/python3.7/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 297, in iter_index_candidate_infos
    hashes=hashes,
  File "/home/pi/oprint/lib/python3.7/site-packages/pip/_internal/index/package_finder.py", line 868, in find_best_candidate
    candidates = self.find_all_candidates(project_name)
  File "/home/pi/oprint/lib/python3.7/site-packages/pip/_internal/index/package_finder.py", line 809, in find_all_candidates
    page_candidates = list(page_candidates_it)
  File "/home/pi/oprint/lib/python3.7/site-packages/pip/_internal/index/sources.py", line 134, in page_candidates
    yield from self._candidates_from_page(self._link)
  File "/home/pi/oprint/lib/python3.7/site-packages/pip/_internal/index/package_finder.py", line 773, in process_project_url
    page_links = list(parse_links(html_page, self._use_deprecated_html5lib))
  File "/home/pi/oprint/lib/python3.7/site-packages/pip/_internal/index/collector.py", line 310, in wrapper_wrapper
    return list(fn(page, use_deprecated_html5lib))
  File "/home/pi/oprint/lib/python3.7/site-packages/pip/_internal/index/collector.py", line 350, in parse_links
    parser.feed(page.content.decode(encoding))
  File "/usr/lib/python3.7/html/parser.py", line 111, in feed
    self.goahead(0)
  File "/usr/lib/python3.7/html/parser.py", line 179, in goahead
    k = self.parse_html_declaration(i)
  File "/usr/lib/python3.7/html/parser.py", line 270, in parse_html_declaration
    self.handle_decl(rawdata[i+2:gtpos])
  File "/home/pi/oprint/lib/python3.7/site-packages/pip/_internal/index/collector.py", line 405, in handle_decl
    self._raise_error()
  File "/home/pi/oprint/lib/python3.7/site-packages/pip/_internal/index/collector.py", line 427, in _raise_error
    "HTML doctype missing or incorrect. Expected <!DOCTYPE html>.\n\n"
ValueError: HTML doctype missing or incorrect. Expected <!DOCTYPE html>.

If you believe this error to be incorrect, try passing the command line option --use-deprecated=html5lib and please leave a comment on the pip issue at https://github.com/pypa/pip/issues/10825.
ERROR: OctoPrint failed to install
To restore your previous install, download the file at: 
https://raw.githubusercontent.com/cp2004/Octoprint-Upgrade-To-Py3/master/go_back.py

Cleaning up...
Error installing OctoPrint, cannot proceed

This looks like a python3.7 issue, but I am not sure. Any advice?

based on the linked issue that was posted 6 days ago, this looks to be related to a change in how pip works pypa/pip#10825

I tried adding --use-deprecated=html5lib to the relevant OctoPrint pip install command:

def install_octoprint(venv_path, backup_path):
    print("\nInstalling OctoPrint... ", end="")
    print_c("(This may take a while - Do not cancel!)", TextColors.YELLOW)
    output, poll = run_sys_command(['{}/bin/python'.format(venv_path), '-m', 'pip', 'install', 'OctoPrint', '--use-deprecated=html5lib'], custom_parser=pip_output_parser)

There error now says the following:

Installing OctoPrint... (This may take a while - Do not cancel!)
ERROR: Could not find a version that satisfies the requirement OctoPrint (from versions: none)
ERROR: No matching distribution found for OctoPrint
ERROR: OctoPrint failed to install
To restore your previous install, download the file at: 
https://raw.githubusercontent.com/cp2004/Octoprint-Upgrade-To-Py3/master/go_back.py

Cleaning up...
Error installing OctoPrint, cannot proceed

So, getting closer, I guess.

Try that again! I've released pip 22.0.1, which fixes a bug that was causing the No matching distribution found issue. :)

Excellent! The script finished successfully and OctoPrint is now installed and using Python3.

Great that everything is solved. Thanks @pradyunsg for the quick response here as well, it has to be tough maintaining pip of all things, with so many users. I feel for you...