Python 3 support woes

Question

Python 3 support woes

Closed this issue 8 years ago · 1 comments

I have been trying to make pyicap work under python3+ (3.4.2 to be precise) but it appears there either are some serious encoding/decoding/string concatenation bugs or i'm using it very wrong (although the same code under python2.7.9 - with required language modifications - works perfectly). The only change i have in the pyicap.py file is i renamed the ICAPServer class to PYICAPServer to avoid confusion. The server code follows, i have removed all the irrelevant parts and changed the routing decision based on random.choice between true and false:

simple_server.py:

import socketserver
from random import choice

from pyicap import PYICAPServer, BaseICAPRequestHandler

class ICAPServer(socketserver.ThreadingMixIn, PYICAPServer):

    def __init__(self, address, RequestHandlerClass):
        self.allow_reuse_address = True
        socketserver.TCPServer.__init__(self, ("127.0.0.1", 8000),
                                        RequestHandlerClass)

class ICAPHandler(BaseICAPRequestHandler):

    def __init__(self, request, client_address, server):
        self.server = server
        self.request = request
        self.client_address = client_address
        super().__init__(request, client_address, server)

    def response_OPTIONS(self):
        self.set_icap_response(200)
        self.set_icap_header(b'Methods', b'RESPMOD')
        self.set_icap_header(b'Service', b'ICAP RESPMOD Server')
        self.set_icap_header(b'ISTag', b'"SOME VERSION"')
        self.set_icap_header(b'Encapsulated', b'null-body=0')
        self.set_icap_header(b'Max-Connections', b'1000')
        self.set_icap_header(b'Options-TTL', b'7200')
        self.set_icap_header(b'Allow', b'204')
        self.set_icap_header(b'Preview', b'0')
        self.send_headers(True)

    def response_RESPMOD(self):
        if choice([True, False]):
            self.no_adaptation_required()
            return
        else:
            self.set_icap_response(200)
            if self.has_body:
                # read all the client has to send us or the nex request will be broken
                while True:
                    if self.read_chunk() == b'':
                        break
            self.send_enc_error(
                403,
                body=b'<html><body><h1>Forbidden</h1><br><br>Not allowed</body></html>'
        )

icap_main.py:

import argparse

from simple_server import ICAPServer, ICAPHandler

parser = argparse.ArgumentParser(description='icap_server')
parser.add_argument("--host", type=str, help="listen host", default="127.0.0.1")
parser.add_argument("--port", type=int, help="listen port", default=8080)
args = parser.parse_args()

if __name__ == "__main__":
    server = ICAPServer(
        (args.host, args.port),
        ICAPHandler)
    print("running")
    try:
        while True:
            server.handle_request()
    except KeyboardInterrupt:
        exit()

when i call the no_adaptation_required function all works perfectly, when i call the send_enc_error i get

Exception happened during processing of request from ('127.0.0.1', 50264)
Traceback (most recent call last):
  File "/usr/lib/python3.4/socketserver.py", line 613, in process_request_thread
    self.finish_request(request, client_address)
  File "/usr/lib/python3.4/socketserver.py", line 344, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "/home/debianadmin/icap/icap/simple_server.py", line 19, in __init__
    super().__init__(request, client_address, server)
  File "/usr/lib/python3.4/socketserver.py", line 669, in __init__
    self.handle()
  File "/home/debianadmin/icap/icap/pyicap.py", line 444, in handle
    self.handle_one_request()
  File "/home/debianadmin/icap/icap/pyicap.py", line 494, in handle_one_request
    method()
  File "/home/debianadmin/icap/icap/simple_server.py", line 46, in response_RESPMOD
    body=b'<html><body><h1>Forbidden</h1><br><br>Not allowed</body></html>'
  File "/home/debianadmin/icap/icap/pyicap.py", line 550, in send_enc_error
    self.send_headers(has_body=True)
  File "/home/debianadmin/icap/icap/pyicap.py", line 294, in send_headers
    enc_req_stat = self.enc_status + b'\r\n'
TypeError: Can't convert 'bytes' object to str implicitly

SQUID 3.5.25 configuration follows (for the icap-relevant parts):

http_access allow ALL
icap_enable on
icap_send_client_ip on
icap_send_client_username on
adaptation_send_client_ip on
adaptation_send_username on
icap_client_username_encode off
icap_client_username_header X-Authenticated-User
icap_preview_enable on
icap_206_enable off
icap_preview_size 2048
icap_service service_resp respmod_precache icap://127.0.0.1:8080/response
adaptation_access service_resp allow all

I have tried to change the calls to contain bytes (200 or b'200') and i have poked a bit in the pyicap code but it appears to me that once i fix one occurence of the bug i cause it in another part, so my understanding of the class itself might not be enough. Let me know how i can help debugging this more, many thanks.

Answer 1 · 2017-05-20T01:24:44.000Z

followup to my previous report, this is definitely a non-bug. i spent some time reading the code in more detail and it turned out that the sending side was sending more data, causing all sorts of bad handling. closing.