sethmlarson/virtualbox-python

UnicodeDecodeError using take_screen_shot_to_array()

noxdafox opened this issue · 22 comments

noxdafox commented

This happens with both VirtualBox PUEL and VirtualBox OSE (setting sys.abiflags = '').

In [1]: import virtualbox
   ...: vbox = virtualbox.VirtualBox()
   ...: vm = vbox.find_machine(vbox.machines[0].name)
   ...: session = vm.create_session()
   ...: h, w, _, _, _, _ = session.console.display.get_screen_resolution(0)
   ...: png = session.console.display.take_screen_shot_to_array(0, h, w, virtualbox.library.BitmapFormat.png)
   ...: 
---------------------------------------------------------------------------
UnicodeDecodeError                        Traceback (most recent call last)
~/development/virtualenvs/pyvbox/lib/python3.5/site-packages/virtualbox/library_base.py in _call_method(self, method, in_p)
    194         try:
--> 195             ret = method(*in_params)
    196         except Exception as exc:

/usr/lib/virtualbox/sdk/bindings/xpcom/python/xpcom/client/__init__.py in takeScreenShotToArray(self, Param1, Param2, Param3, Param4)

UnicodeDecodeError: 'utf-8' codec can't decode byte 0x89 in position 0: invalid start byte

During handling of the above exception, another exception occurred:

AttributeError                            Traceback (most recent call last)
<ipython-input-1-a41323a6ea59> in <module>()
      4 session = vm.create_session()
      5 h, w, _, _, _, _ = session.console.display.get_screen_resolution(0)
----> 6 png = session.console.display.take_screen_shot_to_array(0, h, w, virtualbox.library.BitmapFormat.png)

~/development/virtualenvs/pyvbox/lib/python3.5/site-packages/virtualbox/library.py in take_screen_shot_to_array(self, screen_id, width, height, bitmap_format)
  23680             raise TypeError("bitmap_format can only be an instance of type BitmapFormat")
  23681         screen_data = self._call("takeScreenShotToArray",
> 23682                      in_p=[screen_id, width, height, bitmap_format])
  23683         return screen_data
  23684 

~/development/virtualenvs/pyvbox/lib/python3.5/site-packages/virtualbox/library_base.py in _call(self, name, in_p)
    184         method = self._search_attr(name)
    185         if inspect.isfunction(method) or inspect.ismethod(method):
--> 186             return self._call_method(method, in_p=in_p)
    187         else:
    188             return method

~/development/virtualenvs/pyvbox/lib/python3.5/site-packages/virtualbox/library_base.py in _call_method(self, method, in_p)
    210 
    211             if errobj.msg is None:
--> 212                 errobj.msg = getattr(exc, 'msg', getattr(exc, 'message'))
    213             raise errobj
    214         return ret

AttributeError: 'UnicodeDecodeError' object has no attribute 'message'

XPCOM client init file.
xpcom_cli_init.txt

sethmlarson commented

Can you do me a favor and remove all the try-except logic within _call_method and post the actual traceback? I'm going to be rewriting this logic in v2.

noxdafox commented 
---------------------------------------------------------------------------
UnicodeDecodeError                        Traceback (most recent call last)
<ipython-input-1-a41323a6ea59> in <module>()
      4 session = vm.create_session()
      5 h, w, _, _, _, _ = session.console.display.get_screen_resolution(0)
----> 6 png = session.console.display.take_screen_shot_to_array(0, h, w, virtualbox.library.BitmapFormat.png)

~/development/virtualenvs/pyvbox/lib/python3.5/site-packages/virtualbox/library.py in take_screen_shot_to_array(self, screen_id, width, height, bitmap_format)
  23680             raise TypeError("bitmap_format can only be an instance of type BitmapFormat")
  23681         screen_data = self._call("takeScreenShotToArray",
> 23682                      in_p=[screen_id, width, height, bitmap_format])
  23683         return screen_data
  23684 

~/development/virtualenvs/pyvbox/lib/python3.5/site-packages/virtualbox/library_base.py in _call(self, name, in_p)
    184         method = self._search_attr(name)
    185         if inspect.isfunction(method) or inspect.ismethod(method):
--> 186             return self._call_method(method, in_p=in_p)
    187         else:
    188             return method

~/development/virtualenvs/pyvbox/lib/python3.5/site-packages/virtualbox/library_base.py in _call_method(self, method, in_p)
    193         in_params = [self._cast_to_valuetype(p) for p in in_p]
    194 
--> 195         ret = method(*in_params)
    196 
    197         return ret

/usr/lib/virtualbox/sdk/bindings/xpcom/python/xpcom/client/__init__.py in takeScreenShotToArray(self, Param1, Param2, Param3, Param4)

UnicodeDecodeError: 'utf-8' codec can't decode byte 0x89 in position 0: invalid start byte
sethmlarson commented

It looks like for whatever reason XPCOM is converting the data into UTF-8 when it's probably just raw bytes or a bytearray. :/ It's a shame that traceback doesn't include any line numbers.

noxdafox commented

There is no line number as it's a single array of bytes. It contains the screenshot PNG image.

By PNG specifications, the first byte of the header must be 137 which in hexadecimal is 0x89.

The logic is trying to decode a byte array. It should actually return them as bytes and not str.

sethmlarson commented

I think this might be XPCOM's fault? Even though our logic incorrectly identifies the type octet as a str in the documentation we don't convert the output at all when we receive it.

noxdafox commented

From the trace is definitely hard to tell as we cannot see where it actually fails. Where can I see takeScreenShotToArray? Is it a Python function?

Kepp in mind that, in Python3, the distinction between bytes and str is critical.

sethmlarson commented

I'm aware of the distinction. :) I'm not sure what XPCOM does to load that function into existence because it's not defined within the Python bindings that come with the VirtualBox SDK.

noxdafox commented

Is the code available somewhere?

sethmlarson commented

It gets distributed with the VirtualBox SDK. XPCOM is the library that communicates with the VirtualBox process.

sethmlarson commented

Thanks for debugging this! This means the fault may be outside our jurisdiction?

noxdafox commented

This is what happens with their reference example code.

In [1]: from vboxapi import VirtualBoxManager
In [2]: mgr = VirtualBoxManager(None, None)
In [3]: vbox = mgr.vbox
In [4]: vbox.findMachine('test')
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-4-e0587a4315ea> in <module>()
----> 1 vbox.findMachine('test')

/usr/lib/virtualbox/sdk/bindings/xpcom/python/xpcom/client/__init__.py in findMachine(self, Param1)

TypeError: internal error in PyXPCOM, parameter must be a bytes object
sethmlarson commented

Oh sure, thanks for keeping this issue up to date. Let me know if you learn more!

chroteus commented

Just wanted to say that this bug is still present.
If anyone needs screenshots, use VNC for now.

  • 👀1
guicho271828 commented

this bug is still present.

guicho271828 commented

Regarding the method object that is throwing the utf-8 error, https://github.com/sethmlarson/virtualbox-python/blob/master/virtualbox/library_base.py#L201

It is a function that is built as a string inside xpcom/client/__init__.py (https://www.virtualbox.org/browser/vbox/trunk/src/libs/xpcom18a4/python/client/__init__.py#L70) and gets compiled by compile in BuildMethod. https://www.virtualbox.org/browser/vbox/trunk/src/libs/xpcom18a4/python/client/__init__.py#L130

I printed the string code (by inserting a print) which revealed that the function is

def takeScreenShotToArray(self, Param1, Param2, Param3, Param4):
    return XPTC_InvokeByIndex(self._comobj_, 28, (((128, 6, 0, 0, 0), (128, 6, 0, 0, 0), (128, 6, 0, 0, 0), (128, 6, 0, 0, 0), (64, 6, 0, 0, 0), (96, 148, 4, 4, 4)), (Param1, Param2, Param3, Param4)))

which matches the information from the stack trace:

> /usr/lib/virtualbox/sdk/bindings/xpcom/python/xpcom/client/__init__.py in takeScreenShotToArray(self, Param1, Param2, Param3, Param4)

XPTC_InvokeByIndex is from xpcom._xpcom package https://www.virtualbox.org/browser/vbox/trunk/src/libs/xpcom18a4/python/client/__init__.py#L47

which is a compiled module in https://www.virtualbox.org/browser/vbox/trunk/src/libs/xpcom18a4/python/src/module/_xpcom.cpp

You can find XPTC_InvokeByIndex here https://www.virtualbox.org/browser/vbox/trunk/src/libs/xpcom18a4/python/src/module/_xpcom.cpp#L257

guicho271828 commented 
			  case nsXPTType::T_CHAR_STR: {
				char **pp = (char **)pthis;
				if (*pp==NULL) {
					Py_INCREF(Py_None);
					val = Py_None;
				} else
#if PY_MAJOR_VERSION <= 2
					val = PyString_FromString(*pp);
#else
					val = PyUnicode_FromString(*pp);     <<<<<<<
#endif
				break;
				}
pspcreateprocess commented

According to https://www.virtualbox.org/ticket/19740 the exact location of this error is here:
https://www.virtualbox.org/browser/vbox/trunk/src/libs/xpcom18a4/python/src/VariantUtils.cpp#L627

if (array_type == nsXPTType::T_U8)
#if PY_MAJOR_VERSION <= 2
                return PyString_FromStringAndSize( (char *)array_ptr, sequence_size );
#else
                return PyUnicode_FromStringAndSize( (char *)array_ptr, sequence_size ); // <----- 
#endif

With array of bytes in python3 PyBytes_FromStringAndSize must be called, instead of PyUnicode_FromStringAndSize.

sethmlarson commented

Yeah the issue is upstream, not with this library. Hopefully a fix will land and be released.

guicho271828 commented

ORACLE...

sethmlarson commented

I no longer have time to maintain this library, so am closing this issue.