sibson/vncdotool

Get PIL image object directly instead of saving to disk

Closed this issue · 3 comments

Hello,
I would like to get a screen frame as a PIL object instead of directly writing it to disk, so I can pass that to tesseract for OCR.
I am trying to figure out what is the best way to do that.

I tried something like:

        vnc_client = api.connect(server="ip:port", timeout=5)
        vnc_client.framebufferUpdateRequest(False)
        vnc_client.screen

However, seeing the backend code it does not seem the best way to do it, and in practice I often get either a black frame or a very old frame, while if I call captureSave it is always the correct frame. I see that is probably caused by the deferred operation, is there a way with the current API to achieve this without having to modify it? otherwise I could just add an extra API call (or an intermediate one that does not save the image but returns it).

vncdotool version

vncdotool 1.2.0

framebufferUpdateRequest() just triggers a fetch by sending the low-level RFB command to the VNC server, but you have to wait until all data is transferred back from the server to your client and the local screen is updated. This might take some time and is signaled to the client when commitUpdate() is called. VNCDoToolClient.commitUpdate() translates this event to triggering self.deferred, which the high-level-APIs captureScreen and expectScreen wait for to do their respective actions.

There already is more high-level refreshScreen(), which does the waiting for you:

>>> from vncdotool import api
>>> vnc_client = api.connect(server="skurup:0", timeout=5)
>>> vnc_client.refreshScreen(False)
<vncdotool.client.VNCDoToolClient object at 0x7f8d14212e50>
>>> vnc_client.screen
<PIL.Image.Image image mode=RGB size=1024x768 at 0x7F8D12D69F50>
>>> i.save("x.png")

vnc_client.refreshScreen(False) It takes too long to use this way

vnc_client.refreshScreen(False) It takes too long to use this way

"too slow" is relative; please provide actual timing data. Also this has nothing to do with the original bug report, so please open a new one as this one is also already closed.

refresh(False) has to be called at least once to fetch a initial screen content; afterwards you can use refresh(True) to reduce the required network bandwidth by just fetching the changed regions.
But be warned: VNCDoTool only implements some "old" (less efficient) encodings, which may still be to slow for your use-case; as this is an open-source project you're free implement better encodings by creating a merge request.