semuconsulting/PyGPSClient

Configuration Save to File

Closed this issue · 26 comments

AgileEngineeringLLC commented

Is your feature request related to a problem? Please describe.
no
A clear and concise description of what the problem is e.g. I'd like to be able to do this [...]
I'd like to be able to save the configuration to a file, and then be able to load that configuration back later (or to a second device)

Describe the solution you'd like
I'd use the current menus to configure the device (I use F9Ps), and then I'd like to store that configuration to a file. This would allow me to quickly configure that same device or another device to my saved configuration. Thx!

A clear and concise description of what you want to happen.

Describe alternatives you've considered

A clear and concise description of any alternative solutions or features you've considered.

Would you be willing to contribute a test device?

If the request relates to a specific u-blox device that is not currently supported, would you be
willing to contribute a device to the project for testing purposes?

Additional context

Add any other context about the feature request here.

semuadmin commented

Hi @AgileEngineeringLLC,

This is not the first time I’ve had a request for such a feature, but unfortunately there is no trivial solution here. To the best of my knowledge, even u-blox’s own u-center tool does not support this facility.

PyGPSClient does already partially support this requirement via its User-Defined Preset facility, which allows users to store their own sequences of UBX queries or commands in a ubxpresets file and send these to any number of compatible devices via a single mouse-click. However, this does require a certain amount of ‘hand-cranking’ to set up the presets in the first place, which I appreciate is not ideal.

Short of interacting with the GNSS device at the hardware level (i.e., interrogating & programming the MCU and/or EEPROM, which is beyond the scope of a UBX protocol parser), the only way of reproducing the current configuration of a u-blox GNSS device via the UBX protocol is to poll every possible configuration parameter using a series of CFG-* GET queries, and then “replay” these as a series of CFG-* SET commands.

For Generation 9+ devices such as the NEO-M9N, ZED-F9P, etc., legacy CFG-* messages are deprecated in favour of the newer CFG-VALGET, CFG-VALSET and CFG-VALDEL message types. You can only poll up to 64 parameters in a single CFG-VALGET message. Given there are over 1,200 possible configuration parameters in the latest configuration database (not all are applicable to all devices), a complete configuration dump would involve around 20 individual queries.

Since the UBX protocol does not support explicit synchronous message acknowledgement, there is no absolute guarantee that each CFG poll would receive a timely asynchronous response, especially if the device is already processing high navigation message rates. So you may have to 'retry' some of the polls to ensure you have a complete set of parameters to replay.

But this is not beyond the realms of possibility and I'm happy to put it on the feature backlog for further investigation, but it'll probably be a while before I get to it.

In the short term, I recommend you take a look at PyGPSClient's User-Defined Presets facility and the example ubxpresets file to see if it can meet at least some of your needs. If you need any assistance in configuring the ubxpresets file, get back to me.

Purely out of interest, are you able to give me a flavour of the sort of configuration(s) you would be looking to apply to multiple u-blox devices? In my experience, most working configurations typically involve changes to no more than 20 default settings, so performing a complete dump of all 1,200 parameters may be overkill.

AgileEngineeringLLC commented
semuadmin commented

Hi Don,

Ok so this is a fairly straightforward configuration on the F9P, either using legacy CFG-* commands or the equivalent CFG-VALSET configuration database keys. I’m not sure how familiar you are with the UBX protocol so apologies if I’m stating the obvious here:

Legacy configuration commands:

  1. Set the measRate to 250ms using the CFG-RATE command (the default is 1000ms).
  2. Set the message rate for the requisite NMEA messages/ports to 1 (that is, one every navigation solution, or every 250ms) using the CFG-MSG command. Set all other NMEA message rates to 0 if you want them suppressed.
  3. Set the message rate for the requisite RXM messages/ports to 4 (that is, one every 4th navigation solution, or every 1000ms (250ms * 4)) using the CFG-MSG command.
  4. Filter the requisite GNSS constellation and signal types using the CFG-GNSS command. So if, for example, you wanted only the GPS L1C signal, you would set gnssId = 0 (GPS), enable = 1, sigCfgMask = 0x01. Refer to the UBX F9P Interface Specification for details of other sigCfgMask settings.

Equivalent CFG-VALSET configuration database keys (assuming protocol HPG 1.30 or later and device is connected via USB port):

  1. Measurement rate: CFG-RATE-MEAS
  2. Message rate NMEA: CFG-MSGOUT-NMEA_ID_GGA_USB, CFG-MSGOUT-NMEA_ID_GSA_USB and so on...
  3. Message rate RXM: CFG-MSGOUT-UBX_RXM_MEASX_USB, CFG-MSGOUT-UBX_RXM_RAWX_USB
  4. GNSS signal filter: CFG-SIGNAL-GPS_ENA, CFG-SIGNAL-GPS_L1CA_ENA

FYI the F9P Interface Specification has a cross-reference of legacy vs. modern configuration commands in section 6.10.

Hope this helps. If you need pointers setting up these commands as PyGPSClient user-defined preset strings, give me a shout.

AgileEngineeringLLC commented
semuadmin commented

Hi @AgileEngineeringLLC ,

For what it's worth, I've added a couple of command line examples which illustrate how one might save and reload a complete set of configuration parameters from one Generation 9+ GNSS device to anothe (e.g. NEO-M9N or ZED-F9P).

The save and load processes typically take between 15 and 30 seconds to complete via USB - maybe quicker if you're using a high baud rate UART port.

https://github.com/semuconsulting/PyGPSClient/blob/rc-1.3.14/examples/ubxconfigsave.py
https://github.com/semuconsulting/PyGPSClient/blob/rc-1.3.14/examples/ubxconfigload.py

If these are of any use, I can consider adding this functionality to PyGPSClient in due course.

AgileEngineeringLLC commented
AgileEngineeringLLC commented
semuadmin commented
semuadmin commented
AgileEngineeringLLC commented
semuadmin commented

you probably need to update your version of pyubx2 to the latest 1.2.21 (I only recently added RXM-SPARTN-KEY support):

python3 -m pip install --upgrade pyubx2

Try this and let me know how you get on

semuadmin commented

Like anything accessing a serial port, the script will require SOLE ACCESS to the serial port so, yes, you'll need to disconnect anything else that's currently accessing the same port, including PyGPSClient.

Also, are you sure that's the correct serial port designation? Typically you'd want something like "/dev/tty.usbmodem2101" (tty rather than cu - there is a subtle difference). But try both.

Ths simple script script runs 'blind', as it were, and you won't immediately see any acknowledgement.

If you want to see whether the upload was successful, you can poll the RXM-SPARTN-KEY message in PyGPSClient or u-center. Alternatively, the RXM-COR message will report whether you are receiving and decoding corrections from the D9S correctly.

semuadmin commented

By the way, I'd be careful about broadcasting your SPARTN keys in plain text - Thingstream may not appreciate that!

Please redact them first if you need to send me any further diagnostics.

Also, the keys should be 16 bytes in length - you've entered the hexadecimal version instead. If you bear with me a second, I can update the script to accept the hexadecimal and convert it to bytes.

semuadmin commented

@AgileEngineeringLLC OK I've just this minute uploaded an updated version of the spartnkey.py script which accepts the key as hexadecimal characters. It also simplifies the entering of the From date - simply enter the date in YYYYMMDD format.

https://github.com/semuconsulting/pyubx2/blob/master/examples/spartnkey.py

Download this updated version and see if you get any joy.

semuadmin commented

Don - you do realise this conversation is in the public domain, right? :-)

AgileEngineeringLLC commented

Thanks, I had not realized they were getting posted, so I've deleted certain ones. Thanks.

semuadmin commented
AgileEngineeringLLC commented

In pyGPSClient, at the very top right of the GUI should I be seeing"fix:3D" go to "fix:FLOAT" or "fix:FIXED" when we're in RTK lock? I'm wondering what's the easiest means to tell if the solution is actually using SPARTN corrections.

AgileEngineeringLLC commented

Looks like I can check the GGA messages for the quality flag...

semuadmin commented
AgileEngineeringLLC commented

Roger, thanks much. I'll keep trying some things. It does seem to be staying in 3D/DGPS mode only, never going to Float or Fixed (per the GGA NMEA message). I have it working well via U-Center, so still likely something I've not enabled correctly.

semuadmin commented

FYI I've just uploaded the same key values myself into my ZED-F9P using a PyGPSClient user present (which the script generates) and sent an RXM-SPARTN-KEY poll to confirm that they have been uploaded successfully:

rxmpoll

So the keys are definitely being uploaded OK. I don't have a D9S to hand at the moment so can't test a SPARTN feed.

Just a thought, but the Valid From dates you've entered into the script appear to correspond to dates 11th Feb 2023 (2248 518418) and 11th Mar 2023 (2252 518418), in which case neither of the two keys you're using are actually valid yet. Are those the dates that Thingstream quoted?

AgileEngineeringLLC commented

Yes, those dates work in U-Center and with an Arduino script I can run (provided by Sparkfun).

semuadmin commented

ok bizarre - maybe I have an incorrect understanding of 'Valid From' date!

AgileEngineeringLLC commented

It's an expires date

semuadmin commented

The ZED-F9P actually needs to know when the keys are valid FROM, rather than when they expire. Each key is valid for four weeks, so we need to work backwards 4 weeks from the expiry date quoted. It's possible that u-center is doing this automatically, so if you enter a date of 11th Feb 2023 it automatically winds this back to 11th Jan 2023. I can easily do the same in the script if this is the case.

The way to check would be to upload the keys in u-center and then poll for a RXM-SPARTN-KEY message yourself. If the wno and tow are 4 weeks earlier than the date you entered (e.g. 2244 259218 rather than 2248 518418), then we know that's what's happening.

If you want to try this in PyGPSClient, add the following line to the ubxpresets file in your home directory, then restart PyGPSClient:

Poll RXM-SPARTN-KEY, RXM, RXM-SPARTN-KEY, , 2

You can then select this command from the UBX Configuration panel Presets dropdown list. You should see a response similar to the screen shot I posted earlier.