usage fails on fedora
drewmullen opened this issue · 16 comments
Edit: Resolved.
when installing using chia's guide, which has you compile python, first install readline-devel then reinstall chia starting from the compiling commands.
OP
$ plotman config generate
ModuleNotFoundError: No module named 'readline'`
$ pip install readline
...
ERROR: Command errored out with exit status 1: /home/farmer3/chia/chia-blockchain/venv/bin/python3.7 -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-tdwfs8by/readline_0193af2f4b784d7b8fe7c3dfbad03f70/setup.py'"'"'; __file__='"'"'/tmp/pip-install-tdwfs8by/readline_0193af2f4b784d7b8fe7c3dfbad03f70/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-kvt82h07/install-record.txt --single-version-externally-managed --compile --install-headers /home/farmer3/chia/chia-blockchain/venv/include/site/python3.7/readline Check the logs for full command output.
looked up readline
WARNING: THIS PACKAGE IS DEPRECATED! It has been renamed to GNUREADLINE to resolve a name clash with the standard library module. The new PyPI page is here.
as suggested here i tried reinstall of python + chia + plotman, same errors.
python --version
Python 3.7.7
@drewmullen, we expect to use the stdlib provided package. Do not try to install the PyPI readline package with pip. Last time this came up (#157) it related to issues when compiling Python. Some Linux distributions split up the stdlib. Others may do other things that only conditionally include readline. If you can't python -c 'import readline; print(readline)' and get a print of a path into your Python's stdlib then you'll need to dig into how you got Python and look for support there. Sorry.
I personally use pyenv to get my Pythons. Anyways, here's example output.
$ python -c 'import readline; print(readline)'
<module 'readline' from '/home/altendky/.pyenv/versions/3.9.1/lib/python3.9/lib-dynload/readline.cpython-39-x86_64-linux-gnu.so'>yeah i just realized that the readline pypi link is deprecated because it conflicted with the stdlib readline!
i installed via chia docs:
sudo wget https://www.python.org/ftp/python/3.7.7/Python-3.7.7.tgz
sudo tar -zxvf Python-3.7.7.tgz ; cd Python-3.7.7
./configure --enable-optimizations; sudo make -j$(nproc) altinstall; cd ..i can try pyenv, thats what im used to using in my development environments. i was just trying to go about it the "right way" according to chia. will report back
Uh... Well, I guess it's good for me to know they are recommending compiling your own Python. That's definitely interesting. :| They should probably link to the pyenv wiki for installing deps prior to compiling your own Python. Or just entirely defer to pyenv.
check this out...
$ pyenv install -f 3.7.7
Downloading Python-3.7.7.tar.xz...
-> https://www.python.org/ftp/python/3.7.7/Python-3.7.7.tar.xz
Installing Python-3.7.7...
WARNING: The Python readline extension was not compiled. Missing the GNU readline lib?
Installed Python-3.7.7 to /home/farmer3/.pyenv/versions/3.7.7
*edit - im still reading up researching this - just an update
$ sudo dnf install readline
Package readline-8.1-2.fc34.x86_64 is already installed
@drewmullen, first off, did you get all the deps mentioned in the pyenv wiki installed first? Second, I saw a few tickets on the topic in pyenv. I didn't dig into them. Various bits of python are only included conditionally so you just have to make sure you have the deps in place ahead of time.
@altendky i did not. it was never a problem in the past (probably because macos comes with those by default?) but i see here: "readline-devel"
and that resolved the issue in a venv from pyenv:
$ plotman config generate
Wrote default plotman.yaml to: /home/farmer3/.config/plotman/plotman.yaml
i wonder if installing readline-devel then compiling would work now (without having to go through using pyenv... im gonna try that. will report back then close this puppy - tyvm for the support
I expect it will. Even for people that are compiling Python themselves without pyenv I still recommend the pyenv wiki for installing build deps. You could submit to the Chia wiki you linked to suggest some more steps. :]
I expected that too... however, installing readline-devel to fedora then recompiling did not fix the error. the only way around right now is via pyenv. in the off chance something got tainted in this system, ill try it again with my 4th plotter (hopefully putting together tonight!). once i try again one more time ill look at possibly updating the chia wiki (good suggestion!)
thanks again for your help. for now, recommend your users install via pyenv instead of direct compilation
Did you totally wipe out the existing build environment? There might be some detected config left around. Thanks for the follow ups here.
this host is bare metal in my house (that is currently plotting) so i wasnt able to do a full destroy / rebuild. all i did was remove pyenv shims from my path then:
- recompile
- reinstall chia
sh install.sh - activate the venv it builds
- install plotman
- error
I was referring to the Python build environment. Not the entire computer. I guess that would be the Python-3.7.7 directory from:
sudo tar -zxvf Python-3.7.7.tgz ; cd Python-3.7.7
The ./configure bit is where the dependency availability would be detected, as far as I know. I'm not sure if rerunning it with the existing config reliably redetects or not. Hence suggesting that you may have to totally wipe the Python interpreter build stuff entirely.
But, dig into this only however far you find interesting or appropriate, of course.
@altendky AH i see what you mean. i moved to another dir and repeated the install and now plotman is working as expected!
ill send a PR to chia docs its actually probably more appropriate to have a note in your docs if users have an issue. since plotman isnt a supported tool by chia, thoughts?
do you have docs for configuring rsync? for example, i have a tmp, nvme dest and then mounted share from my nas. id love the rsync feature to take from my nvme dest and move to my nas
archive:
rsyncd_module: plots # not sure what this is?
rsyncd_path: /mnt/synology/usbshare1/rsynctest # local mount to my nas
rsyncd_bwlimit: 80000 # Bandwidth limit in KB/s
rsyncd_host: 192.168.2.101 # nas ip but not sure why this is required since i have mount locally
rsyncd_user: farmer3 # is this the user on my plotting host?
We do need to improve our docs and related explanations. I do consider this to be bad instruction on their part. They are telling you how to compile a maybe-missing-stuff Python without any commentary. So, it really maybe belongs both places. But mostly, if you need to compile Python use pyenv and follow their instructions. :]
@drewmullen does your NAS run rsyncd? I'd personally tend towards that over a mounted share. I don't have good feelings about network mounts being particularly robust or failing nicely. Though maybe that attitude is outdated. Anyways, either rsyncd or wait for #88 to get finished up and then you can configure local rsync such as with a mounted network filesystem or just local drives.
When you configure rsyncd you define 'modules'. In my setup below the module is named sites. I think the rsyncd_user maybe doesn't matter since we aren't managing any authentication with rsyncd anyways. In addition to rsyncd, plotman first connects with ssh to run df -aBK so it can check usage of the relevant drives and see which have space available. You need to setup ssh keys for this.
https://github.com/altendky/farm/blob/1ec82b80ae86949fbe26b21115732dda3c2e04d1/config.yaml
https://github.com/altendky/farm/blob/1ec82b80ae86949fbe26b21115732dda3c2e04d1/rsyncd.conf
ty for this link: #88
ill send a doc PR tomorrow to include a note. tbh i think its OK that chia doesnt mention readline because their package doesnt require it so... from their standpoint it seems to be OK. i only encountered issues when branching out to external tooling
In my case, on CentOS 8.3, with chia also installed per instructions, I just had to install readline-devel (dnf install readline-devel) then recompile Python 3.7.7 (no manual changes) and plotman generates a config successfully afterward.
Confirmed solution above on CentOS7. Install readline-devel and then rebuild Python 3.7.