semuconsulting/PyGPSClient

Raspberry Pi 32-bit

acbtones opened this issue · 10 comments

acbtones commented

Discussed in #83

Originally posted by acbtones August 4, 2023
Hi,

Firstly, thanks for the great software!
However, I can't get it to install on Raspberry Pi 32-bit OS.
Using a RPi 3, I installed 32bit OS ("Raspberry Pi OS Full (32-bit)") and tried the installation - log attached: 32bit_pygpsclient.log
Cryptography fails to install so the PyGPSClient install fails.
Is that related to this: pyca/cryptography#5766 (comment)
I think that's why previous discussions in this forum have suggested Rust is required.
Using the same RPi 3, I installed 64-bit OS ("Raspberry Pi OS (64-bit)") and tried the installation - log attached: 64bit_pygpsclient.log
Successful installation.

My wish is to add PyGPSClient to an existing 32-bit RPi installation running other software that will only run on 32-bit.

Also, when starting PyGPSClient, is it possible to start the stream immediately i.e. without clicking the USB button?

Thanks,

Anthony

32bit_pygpsclient.log
64bit_pygpsclient.log

semuadmin commented

Hi @acbtones

Sorry you're having difficulties.

PyGPSClient uses the pyspartn library, which in turn uses the cryptography library, for decrypting SPARTN messages.

It appears that from cryptography >=38.0 onwards, if you install cryptography via pip from a source file (.tar.gz) rather than a binary wheel (.whl), the installation requires a rust compiler. Most modern 64-bit Linux distros can provide a wheel distribution package, but it appears that some older 32-bit versions of Raspi OS don't.

NB: this is only an issue if pip needs to install cryptography from source - if pip can find a binary wheel installation package for your platform, the problem should not occur.

Sometimes simply updating pip (and purging the cache) can resolve the problem (by making available a wheel distribution package)...

python3 -m pip install --upgrade pip
python3 -m pip cache purge

... though I tried this on my own RPI 3 running Raspi Lite without success.

The following worked for me on an RPI 3 running 32-bit Raspi OS Lite (RPI3 Model B V.2):

  1. Firstly, install the rust compiler dependencies:

    sudo apt-get install build-essential libssl-dev libffi-dev python3-dev cargo pkg-config

  2. Then install cryptography (NB: the compilation will take several minutes on an old 32-bit RPi):

    python -m pip install cryptography

pi@raspberrypi:~ $ python -m pip install cryptography
Defaulting to user installation because normal site-packages is not writeable
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Collecting cryptography
  Using cached cryptography-41.0.3.tar.gz (630 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting cffi>=1.12 (from cryptography)
  Using cached https://www.piwheels.org/simple/cffi/cffi-1.15.1-cp39-cp39-linux_armv7l.whl (364 kB)
Collecting pycparser (from cffi>=1.12->cryptography)
  Using cached https://www.piwheels.org/simple/pycparser/pycparser-2.21-py2.py3-none-any.whl (119 kB)
Building wheels for collected packages: cryptography
  Building wheel for cryptography (pyproject.toml) ... | 
done
  Created wheel for cryptography: filename=cryptography-41.0.3-cp39-cp39-linux_armv7l.whl size=2084218 sha256=6ddc66713c9cd8a74db131edeb1519a34cbb5e34d1d85f0db7c4662efb9a9aa1
  Stored in directory: /home/pi/.cache/pip/wheels/83/ef/54/3d54fa97a47ec9b505300e38f091780aadce82b0d74f5bcd1c
Successfully built cryptography
Installing collected packages: pycparser, cffi, cryptography
Successfully installed cffi-1.15.1 cryptography-41.0.3 pycparser-2.21
  1. You should then be able to install pyspartn (and by implication any packages dependent on it):

python3 -m pip install pyspartn

pi@raspberrypi:~ $ python3 -m pip install pyspartn
Defaulting to user installation because normal site-packages is not writeable
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Collecting pyspartn
  Downloading https://www.piwheels.org/simple/pyspartn/pyspartn-0.1.10-py3-none-any.whl (26 kB)
Requirement already satisfied: cryptography>=39.0.2 in ./.local/lib/python3.9/site-packages (from pyspartn) (41.0.3)
Requirement already satisfied: cffi>=1.12 in ./.local/lib/python3.9/site-packages (from cryptography>=39.0.2->pyspartn) (1.15.1)
Requirement already satisfied: pycparser in ./.local/lib/python3.9/site-packages (from cffi>=1.12->cryptography>=39.0.2->pyspartn) (2.21)
Installing collected packages: pyspartn
Successfully installed pyspartn-0.1.10

Try this and let me know how you get on.

If it works, I'll add these instructions to the README.md for PYGPSClient and pyspartn.

semuadmin commented

@acbtones

In answer to your second point about automatically starting a given serial connection when PyGPSClient starts, this is not supported, but what you can do is start PyGPSClient with a command line option -U or --userport, e.g.:

pygpsclient --userport /dev/ttyACM1

The designated serial port will then be automatically pre-selected in the serial port selection listbox, but you will still need to press the serial 'connect' button.

Out of interest, do you definitely need to use a GUI application for your purposes? There is a separate pygnssutils library which provides much of the underlying functionality of PyGPSClient as a series of CLI utilties, which may be useful for unattended or 'headless' operation.

semuadmin commented

@acbtones

As an aside, I suspect relatively few PyGPSClient users are actually interested in decrypting SPARTN messages (and the pyspartn library is still only in Alpha), so I may look to add an environment variable e.g. PYSPARTN_NODECRYPT which in effect suppresses the dependency on the cryptography library for those users who are happy to live without this functionality.

But, hopefully, the wheel distribution package for cryptography will eventually find its way into older 32-bit distros.

acbtones commented
semuadmin commented

Hi @acbtones

OK so it appears to be objecting to the specific version of the rust compiler installed. Let me install everything from scratch so we're comparing like with like...

  1. I installed a fresh Raspberry Pi OS 32-bit Full on my RPI3 Model B.2 using the official Raspberry Pi Imager, so it should exactly match your machine. I did an initial update:
sudo apt update
sudo apt upgrade -y
sudo apt autoremove -y
sudo reboot

The platfom details after rebooting were as follows:

cat /etc/os-release
PRETTY_NAME="Raspbian GNU/Linux 11 (bullseye)"
NAME="Raspbian GNU/Linux"
VERSION_ID="11"
VERSION="11 (bullseye)"
VERSION_CODENAME=bullseye
ID=raspbian
ID_LIKE=debian
HOME_URL="http://www.raspbian.org/"
SUPPORT_URL="http://www.raspbian.org/RaspbianForums"
BUG_REPORT_URL="http://www.raspbian.org/RaspbianBugs"

uname -a
Linux raspberrypi 6.1.21-v7+ #1642 SMP Mon Apr  3 17:20:52 BST 2023 armv7l GNU/Linux

python3 -V
Python 3.9.2
  1. Install Rust using the official instructions here. This takes about 3 minutes to run:
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
  1. Check the version of the rust compiler installed (it needs to be >= 1.56 apparently):
rustc -V
rustc 1.71.1 (eb26296b5 2023-08-03)
  1. Install the cryptography library build dependencies (omitting cargo, which has already been installed in the previous step):
sudo apt-get install build-essential libssl-dev libffi-dev python3-dev pkg-config
  1. Update pip:
python3 -m pip install --upgrade pip
  1. Install pyspartn using pip (this will take about 2 minutes as it has to compile the cryptography library from a source .tar.gz file):
python3 -m pip install pyspartn
Defaulting to user installation because normal site-packages is not writeable
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Collecting pyspartn
  Using cached https://www.piwheels.org/simple/pyspartn/pyspartn-0.1.10-py3-none-any.whl (26 kB)
Collecting cryptography>=39.0.2 (from pyspartn)
  Using cached cryptography-41.0.3.tar.gz (630 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting cffi>=1.12 (from cryptography>=39.0.2->pyspartn)
  Using cached https://www.piwheels.org/simple/cffi/cffi-1.15.1-cp39-cp39-linux_armv7l.whl (364 kB)
Collecting pycparser (from cffi>=1.12->cryptography>=39.0.2->pyspartn)
  Using cached https://www.piwheels.org/simple/pycparser/pycparser-2.21-py2.py3-none-any.whl (119 kB)
Building wheels for collected packages: cryptography
  Building wheel for cryptography (pyproject.toml) ... done
  Created wheel for cryptography: filename=cryptography-41.0.3-cp39-cp39-linux_armv7l.whl size=2084768 sha256=9aa029edb05e0337f363133d1b58b7be518a4d79e905a4361994323f766ded0b
  Stored in directory: /home/steve/.cache/pip/wheels/83/ef/54/3d54fa97a47ec9b505300e38f091780aadce82b0d74f5bcd1c
Successfully built cryptography
Installing collected packages: pycparser, cffi, cryptography, pyspartn
Successfully installed cffi-1.15.1 cryptography-41.0.3 pycparser-2.21 pyspartn-0.1.10
  1. If this is successful, you should then be able to install PyGPSClient as per the standard installation instructions, remembering to install the python3-pil.imagetk dependency first (it's not installed by default on Raspberry Pi OS):
sudo apt install python3-pil.imagetk
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Suggested packages:
  python-pil-doc python3-pil.imagetk-dbg
The following NEW packages will be installed:
  python3-pil.imagetk
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 74.4 kB of archives.
After this operation, 103 kB of additional disk space will be used.
Get:1 http://raspbian.raspberrypi.org/raspbian bullseye/main armhf python3-pil.imagetk armhf 8.1.2+dfsg-0.3+deb11u1 [74.4 kB]
Fetched 74.4 kB in 1s (129 kB/s)             
Selecting previously unselected package python3-pil.imagetk:armhf.
(Reading database ... 106702 files and directories currently installed.)
Preparing to unpack .../python3-pil.imagetk_8.1.2+dfsg-0.3+deb11u1_armhf.deb ...
Unpacking python3-pil.imagetk:armhf (8.1.2+dfsg-0.3+deb11u1) ...
Setting up python3-pil.imagetk:armhf (8.1.2+dfsg-0.3+deb11u1) ...

python3 -m pip install pygpsclient
...
Successfully installed Pillow-10.0.0 charset-normalizer-3.2.0 paho-mqtt-1.6.1 pygnssutils-1.0.12 pygpsclient-1.4.0 pynmeagps-1.0.28 pyrtcm-1.0.11 pyserial-3.5 pyubx2-1.2.29 requests-2.31.0

And here is PyGPSClient running quite happily on my RPI3 32-bit Full machine:

pygpsclient

Hope this helps.

semuadmin commented

@acbtones

If all else fails, here's a zipped copy of the actual *.whl file generated when I installed cryptography on my RPI3 above.

NB:

  1. This will only work on RPI3 Model B platforms.
  2. Downloading installation files from untrusted sources is not normally to be recommended, so install at your own risk. Check the hash on the unzipped file to ensure it hasn't been tampered with before installing:
shasum -a 256 cryptography-41.0.3-cp39-cp39-linux_armv7l.whl

9aa029edb05e0337f363133d1b58b7be518a4d79e905a4361994323f766ded0b  cryptography-41.0.3-cp39-cp39-linux_armv7l.whl

Once unzipped, you should be able to install this directly using:

python3 -m pip install cryptography-41.0.3-cp39-cp39-linux_armv7l.whl

cryptography-41.0.3-cp39-cp39-linux_armv7l.whl.zip

acbtones commented
semuadmin commented

Hi, Thanks again for your comprehensive follow up! I can confirm that your updated instructions work on a RPi3, 32-bit. I also tried the installation on a RPi4, 32-bit and there is one catch that users should be aware of: when installing Rust on the RPi4, 32-bit...

Thanks for the tip - just confirmed this on my own RPI 4 Model B 4MB. I've come across this issue before - the platform architecture of the RPI4 is indeed aarch64, but when running 32-bit OS on this platform, pip and other package managers do not recognise this as a compatible 'tag' for installation files, so you have to substitute armv7l:

uname -a

Linux raspberrypi 6.1.21-v8+ #1642 SMP PREEMPT Mon Apr  3 17:24:16 BST 2023 aarch64 GNU/Linux
python3 -m pip debug --verbose
...
Compatible tags: 369
...
  cp39-cp39-linux_armv7l
  cp39-abi3-manylinux_2_31_armv7l
  cp39-abi3-manylinux_2_30_armv7l
...

Again, for those who would prefer to avoid the palaver of installing Rust and all its dependencies, here's a zipped .whl file for cryptography 41.0.3 for the RPI4 platform, to use at your own risk (once again, check the hash matches once you've unzipped and copied this to your RPI):

cryptography-41.0.3-cp39-cp39-linux_armv7l.whl.zip

shasum -a 256 cryptography-41.0.3-cp39-cp39-linux_armv7l.whl
98fa0f713078d2036a7bb83c9f37bb7c3446ab7f2367d4b2881b259b6baca9fd  cryptography-41.0.3-cp39-cp39-linux_armv7l.whl

python3 -m pip install cryptography-41.0.3-cp39-cp39-linux_armv7l.whl
Defaulting to user installation because normal site-packages is not writeable
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Processing ./cryptography-41.0.3-cp39-cp39-linux_armv7l.whl
Requirement already satisfied: cffi>=1.12 in /home/steve/.local/lib/python3.9/site-packages (from cryptography==41.0.3) (1.15.1)
Requirement already satisfied: pycparser in /home/steve/.local/lib/python3.9/site-packages (from cffi>=1.12->cryptography==41.0.3) (2.21)
Installing collected packages: cryptography
Successfully installed cryptography-41.0.3

Oh, and thanks for the donation - very much appreciated 👍

semuadmin commented

@acbtones

I've updated the installation documentation in PyGPSClient and pyspartn.

Are you happy to close this issue as resolved now?

acbtones commented