Building error
nandub opened this issue · 9 comments
After installing yotta, I'm trying to build microbit-samples to make sure everything is setup right, but after running yt build I get the following error:
info: generate for target: bbc-microbit-classic-gcc 0.2.3 at /Users/nandub/development/code/microbit/microbit-samples/yotta_targets/bbc-microbit-classic-gcc
error: Can't pickle <function Pack.runScript at 0x10d7c3820>: it's not the same object as yotta.lib.pack.Pack.runScript
Python version: 3.8.3 on a virtualenv
how can I reproduce this (step by step) ? This looks like multiprocessing error in yotta.
Steps to reproduce:
I'm using pyenv with pyenv-virtualenv to create a clean python environment. Also I'm using a Mac / OS HIgh Sierra
pyenv install 3.8.3
pyenv global 3.8.3
pyenv virtualenv microbit
pyenv activate microbit
Install yotta
pip install yotta
Compile microbit-samples
git clone https://github.com/lancaster-university/microbit-samples.git
cd microbit-samples
yt clean
yt target bbc-microbit-classic-gcc
yt build
info: generate for target: bbc-microbit-classic-gcc 0.2.3 at /Users/nandub/development/code/microbit/microbit-samples/yotta_targets/bbc-microbit-classic-gcc
error: Can't pickle <function Pack.runScript at 0x10cc4d700>: it's not the same object as yotta.lib.pack.Pack.runScript
I'm also seeing the same issue on macOS with Python 3.8.2 and 3.8.3. Following the steps indicated by @nandub:
info: generate for target: bbc-microbit-classic-gcc 0.2.3 at ...
error: Can't pickle <function Pack.runScript at 0x10f07c160>: it's not the same object as yotta.lib.pack.Pack.runScript
$ pip list
Package Version
----------------------------- -----------
appdirs 1.4.4
argcomplete 1.11.1
certifi 2020.6.20
cffi 1.14.0
chardet 3.0.4
cmsis-pack-manager 0.2.10
colorama 0.3.9
cryptography 2.9.2
Deprecated 1.2.10
future 0.18.2
hgapi 1.7.4
hidapi 0.9.0.post3
idna 2.10
intelhex 2.2.1
intervaltree 3.0.2
Jinja2 2.11.2
jsonpointer 1.14
jsonschema 2.6.0
MarkupSafe 1.1.1
mbed-test-wrapper 1.0.0
milksnake 0.1.5
pathlib 1.0.1
pip 19.2.3
prettytable 0.7.2
project-generator 0.8.17
project-generator-definitions 0.2.40
psutil 5.7.0
pycparser 2.20
pyelftools 0.23
PyGithub 1.51
PyJWT 1.7.1
pylink-square 0.6.1
pyocd 0.26.1
pyOpenSSL 19.1.0
pyusb 1.0.2
PyYAML 5.3.1
requests 2.24.0
semantic-version 2.8.5
setuptools 41.2.0
six 1.15.0
sortedcontainers 2.2.2
urllib3 1.25.9
valinor 1.1.4
wrapt 1.12.1
xmltodict 0.12.0
yotta 0.20.5
Seems to be working on Ubuntu 16.04 and Windows 10 with Python 3.8.
On macOS it works fine with Python 3.6.8.
@carlosperate awesome, thanks for testing and confirming the error. I'm able to install and use python 3.6.8.
There must be something wrong with newer version of python, not very familiar with python's internal to try to figure this error out. If someone else knows how to debug the error, please go ahead and try to figure out.
The default pickle protocol changed between python 3.6 and 3.8, so maybe something to do with that?
If you temporarily remove the @dropRootPrivs decorator from runScript , does the build fail later on a different function, or is this the only one causing a problem?
You could try changing the runScript method to make it use a new module-level function which uses the @dropRootPrivs decorator, and leave the class method undecorated.
From Python 3.9 there is support to specify the user and group to subprocess.Popen directly, so could just change the subprocess call to use that in the future, instead of the whacky dropRootPrivs decorator.
@autopulated thanks for the feedback. If I removed @dropRootPrivs I get a similar error but on a different function.
error: Can't pickle <function DerivedTarget.build at 0x10260b160>: it's not the same object as yotta.lib.target.DerivedTarget.build
If I do the same to build and removed @dropRootPrivs the yt build then succeeds.
@nandub Thanks for testing, would you be able to send a pull request fixing it? I assume the solution that would work not just with python 3.9.
@0xc0170 python is not my forte, I'd rather let someone else with more expertise on this area to come up with a pull request with the fixes.
Same error MacOs 11.1
$ yt build
info: generate for target: bbc-microbit-classic-gcc 0.2.3 at /Users/user/Documents/microbit-samples/yotta_targets/bbc-microbit-classic-gcc
error: Can't pickle <function Pack.runScript at 0x10dbaeb80>: it's not the same object as yotta.lib.pack.Pack.runScript
Install as described in http://docs.yottabuild.org/#installing
brew tap ARMmbed/homebrew-formulae
brew install python cmake ninja arm-none-eabi-gcc
pip install yotta
$ python --version
Python 3.9.1
EDIT: After I am commented all @fsutils.dropRootPrivs in /usr/local/lib/python3.9/site-packages/yotta/lib/[pack|target].py - running yt build was done with success.