Ntrip client + pyrtcm
foxittt opened this issue · 42 comments
RTCM data can also be obtained from RTCM sources using RTCM clients such as https://github.com/nunojpg/ntripclient or https://github.com/liukai-tech/NtripClient-Tools.
Is it possible to submit a stream from an RTCM client to pyrtcm?
Hi @foxittt Sorry, no. pyrtcm can parse RTCM3 data from any Serial (UART) data stream or binary datafile, but not (for example) an HTTP/TCP socket, so it would be necessary to convert any NTRIP data stream into one or other of these accepted formats before pyrtcm could parse it.
There are no current plans to add such functionality, though I'll try to look into what might be involved when I get some time and if there appears to be sufficient interest.
Re-opened as I think this merits further investigation, but no guarantee on timescale at this point.
Just found this library with a similar interest in mind. I'm writing a little robot navigation program, that's connecting to an NTRIP client, and then I want to feed the output of the NTRIP session into pyrtcm, and then yield complete RTCM messages that can be forwarded to a ublox GPS module.
It would be nice to be able to feed data into the RTCMReader, and then async await any complete messages. Will be taking a closer look shortly.
FYI I'm working on a simple command line NTRIP client and hope to upload a preliminary version into the \examples folder in the next week or so. It will connect to a designated NTRIP server (caster), parse the header response and RTCM3 output from that server, and handle the uploading of NMEA GGA position records (using the companion pynmeagps library).
Watch this space.
Here's a sneak preview of the output:
Connected to: 3.23.52.207:2101
GET /AEAUK HTTP/1.1
User-Agent: pyrtcm NTRIP client/0.1
Authorization: Basic xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Ntrip-Version: Ntrip/2.0
*** Start Of Header ***
HTTP/1.1 200 OK
Date: Mon, 28 March 2022 22:22:13 UTC
Server: SubCarrier Systems Corp SNIP simpleNTRIP_Caster_[3.04.34]RwPRO
Ntrip-Version: Ntrip/2.0
Cache-Control: no-store, no-cache, max-age=0
Pragma: no-cache
Connection: close
Content-Type: gnss/data
*** End Of Header ***
Data bytes received: 304
<RTCM(1074, DF002=1074, DF003=0, GNSSEpoch=166958000, DF393=1, DF409=4, DF001_7=0, DF411=1, DF412=0, DF417=0, DF418=0, DF394=13011088448063275008, NSat=11, DF395=1346437120, NSig=4, DF396=17592186040047, DF400_01=0.00054365, DF400_02=-0.00070077, DF400_03=-0.00082338, DF400_04=0.00080037, DF400_05=0.00089228, DF400_06=-0.00095767, DF400_07=0.00072014, DF400_08=-0.00062102, DF400_09=0.00079465, DF400_10=-0.00061792, DF400_11=0.00063735, DF400_12=0.0006839, DF400_13=-0.00078541, DF400_14=-0.00020713, DF400_15=-0.00020695, DF400_16=3.862e-05, DF400_17=-0.00093812, DF400_18=0.00096422, DF400_19=-0.00074476, DF400_20=0.00047678, DF400_21=0.00023258, DF400_22=0.00022769, DF400_23=-0.00074893, DF400_24=0.00022799, DF400_25=-1.621e-05, DF400_26=0.00024801, DF400_27=-0.00024045, DF400_28=5.19e-06, DF400_29=0.00049347, DF400_30=0.00068736, DF400_31=0.00019866, DF400_32=-0.00077677, DF400_33=-0.00028884, DF400_34=-0.00026464, DF400_35=-0.0007531, DF400_36=0.00022393, DF400_37=0.00095648, DF400_38=0.00072396, DF400_39=0.00023574, DF400_40=-0.00025272, DF400_41=-0.00025266, DF400_42=0.00067472, DF400_43=0.00091863, DF400_44=0.00091839, DF401_01=-0.00188884, DF401_02=3.839e-05, DF401_03=-0.00300141, DF401_04=-0.00141553, DF401_05=0.00263151, DF401_06=0.00089338, DF401_07=0.00229209, DF401_08=0.0006876, DF401_09=0.0006876, DF401_10=0.00074765, DF401_11=0.00073787, DF401_12=-0.00010199, DF401_13=-0.00010198, DF401_14=-4.435e-05, DF401_15=-2.805e-05, DF401_16=-0.0002424, DF401_17=-0.0002424, DF401_18=-0.000231, DF401_19=-0.00022614, DF401_20=8.828e-05, DF401_21=8.831e-05, DF401_22=0.00012761, DF401_23=0.00015692, DF401_24=-0.00070773, DF401_25=-0.00070772, DF401_26=-0.00069049, DF401_27=-0.00070195, DF401_28=-0.00032431, DF401_29=-0.00032431, DF401_30=-0.00032484, DF401_31=-0.00031344, DF401_32=-6.229e-05, DF401_33=-6.229e-05, DF401_34=-1.779e-05, DF401_35=-2.755e-05, DF401_36=-0.00084885, DF401_37=-0.00084886, DF401_38=-0.00086509, DF401_39=0.00042122, DF401_40=0.00042121, DF401_41=0.00036823, DF401_42=-0.00050283, DF401_43=-0.00050284, DF401_44=-0.00046498, DF402_01=14, DF402_02=4, DF402_03=5, DF402_04=12, DF402_05=2, DF402_06=15, DF402_07=9, DF402_08=1, DF402_09=7, DF402_10=1, DF402_11=5, DF402_12=14, DF402_13=4, DF402_14=9, DF402_15=8, DF402_16=4, DF402_17=15, DF402_18=9, DF402_19=4, DF402_20=1, DF402_21=7, DF402_22=3, DF402_23=15, DF402_24=15, DF402_25=15, DF402_26=15, DF402_27=15, DF402_28=15, DF402_29=15, DF402_30=15, DF402_31=15, DF402_32=15, DF402_33=15, DF402_34=15, DF402_35=15, DF402_36=15, DF402_37=15, DF402_38=15, DF402_39=15, DF402_40=15, DF402_41=15, DF402_42=15, DF402_43=15, DF402_44=15, DF420_01=1, DF420_02=1, DF420_03=1, DF420_04=1, DF420_05=1, DF420_06=1, DF420_07=1, DF420_08=1, DF420_09=1, DF420_10=1, DF420_11=1, DF420_12=1, DF420_13=1, DF420_14=1, DF420_15=1, DF420_16=1, DF420_17=1, DF420_18=1, DF420_19=1, DF420_20=1, DF420_21=1, DF420_22=1, DF420_23=1, DF420_24=1, DF420_25=1, DF420_26=1, DF420_27=1, DF420_28=1, DF420_29=1, DF420_30=1, DF420_31=1, DF420_32=1, DF420_33=1, DF420_34=1, DF420_35=1, DF420_36=1, DF420_37=1, DF420_38=1, DF420_39=1, DF420_40=1, DF420_41=1, DF420_42=1, DF420_43=1, DF420_44=1, DF403_01=63, DF403_02=63, DF403_03=63, DF403_04=63, DF403_05=63, DF403_06=32, DF403_07=0, DF403_08=0, DF403_09=0, DF403_10=0, DF403_11=0, DF403_12=0, DF403_13=45, DF403_14=47, DF403_15=46, DF403_16=46, DF403_17=50, DF403_18=51, DF403_19=51, DF403_20=47, DF403_21=46, DF403_22=34, DF403_23=35, DF403_24=45, DF403_25=41, DF403_26=27, DF403_27=27, DF403_28=39, DF403_29=41, DF403_30=36, DF403_31=36, DF403_32=37, DF403_33=36, DF403_34=17, DF403_35=17, DF403_36=33, DF403_37=47, DF403_38=39, DF403_39=39, DF403_40=44, DF403_41=45, DF403_42=36, DF403_43=36, DF403_44=41)>
Data bytes received: 460
<RTCM(1084, DF002=1084, DF003=0, GNSSEpoch=273375456, DF393=1, DF409=4, DF001_7=0, DF411=1, DF412=0, DF417=0, DF418=0, DF394=16195156465768464384, NSat=8, DF395=1090519040, NSig=2, DF396=65471, DF400_01=0.00054353, DF400_02=-0.00069696, DF400_03=-0.00033909, DF400_04=-0.0001781, DF400_05=-0.00083333, DF400_06=0.00084567, DF400_07=-0.00051057, DF400_08=0.00050557, DF400_09=-0.00069439, DF400_10=0.00074321, DF400_11=0.00064403, DF400_12=-0.00041932, DF400_13=-0.00073594, DF400_14=-2.259e-05, DF400_15=0.00085479, DF400_16=-0.00037801, DF401_01=-0.00210683, DF401_02=-0.00359596, DF401_03=-0.00378544, DF401_04=0.00306693, DF401_05=0.00286818, DF401_06=0.00384931, DF401_07=-0.00241077, DF401_08=-0.00225978, DF401_09=0.00161068, DF401_10=0.00178191, DF401_11=0.00154348, DF401_12=0.00159468, DF401_13=-0.00128505, DF401_14=-0.00125057, DF401_15=0.00247344, DF401_16=-0.00237724, DF402_01=11, DF402_02=5, DF402_03=14, DF402_04=8, DF402_05=8, DF402_06=1, DF402_07=4, DF402_08=8, DF402_09=3, DF402_10=7, DF402_11=0, DF402_12=5, DF402_13=5, DF402_14=1, DF402_15=7, DF402_16=11, DF420_01=1, DF420_02=1, DF420_03=1, DF420_04=1, DF420_05=0, DF420_06=0, DF420_07=1, DF420_08=0, DF420_09=1, DF420_10=1, DF420_11=1, DF420_12=0, DF420_13=0, DF420_14=1, DF420_15=1, DF420_16=0, DF403_01=23, DF403_02=60, DF403_03=57, DF403_04=45, DF403_05=63, DF403_06=63, DF403_07=63, DF403_08=63, DF403_09=63, DF403_10=63, DF403_11=47, DF403_12=63, DF403_13=63, DF403_14=63, DF403_15=56, DF403_16=0)>
Data bytes received: 442
<RTCM(1074, DF002=1074, DF003=0, GNSSEpoch=166960000, DF393=1, DF409=4, DF001_7=0, DF411=1, DF412=0, DF417=0, DF418=0, DF394=13011088448063275008, NSat=11, DF395=1346437120, NSig=4, DF396=17592186040047, DF400_01=0.00054365, DF400_02=-0.00070077, DF400_03=-0.00082338, DF400_04=0.00080037, DF400_05=0.00089228, DF400_06=-0.00095767, DF400_07=0.00075066, DF400_08=-0.00037783, DF400_09=0.00076413, DF400_10=0.00011426, DF400_11=0.00063735, DF400_12=0.00019658, DF400_13=-0.00076252, DF400_14=-0.00027585, DF400_15=0.00045657, DF400_16=0.0007022, DF400_17=-0.00076276, DF400_18=0.00019014, DF400_19=-5.4e-05, DF400_20=-0.00078559, DF400_21=-0.00078559, DF400_22=-0.0001964, DF400_23=0.00029194, DF400_24=4.828e-05, DF400_25=0.00053644, DF400_26=-0.00051343, DF400_27=0.00021923, DF400_28=-0.00075638, DF400_29=0.00046462, DF400_30=-0.00077266, DF400_31=-0.00052881, DF400_32=0.00020462, DF400_33=-0.00077206, DF400_34=0.00094515, DF400_35=0.00094497, DF400_36=0.00045735, DF400_37=-0.00027502, DF400_38=-0.00069577, DF400_39=0.00076902, DF400_40=0.00052482, DF400_41=-0.00045168, DF400_42=-0.00097322, DF400_43=-0.00072944, DF400_44=0.00049108, DF401_01=-5.081e-05, DF401_02=0.00103738, DF401_03=-0.00036669, DF401_04=-0.00012541, DF401_05=-0.00151841, DF401_06=0.00093153, DF401_07=0.00082747, DF401_08=-0.0004138, DF401_09=-0.0004138, DF401_10=-0.00035376, DF401_11=-0.00036354, DF401_12=-0.0007684, DF401_13=-0.0007684, DF401_14=-0.00071076, DF401_15=-0.00069447, DF401_16=0.00078646, DF401_17=0.00078648, DF401_18=0.00079787, DF401_19=0.00080274, DF401_20=-0.00037438, DF401_21=-0.00037436, DF401_22=-0.00033505, DF401_23=-0.00030573, DF401_24=-0.00062741, DF401_25=-0.00062742, DF401_26=-0.00061017, DF401_27=-0.00062165, DF401_28=-0.00050234, DF401_29=-0.00050236, DF401_30=-0.00050285, DF401_31=-0.00049147, DF401_32=0.00065958, DF401_33=0.00065958, DF401_34=0.00070409, DF401_35=0.00069432, DF401_36=0.0001262, DF401_37=0.0001262, DF401_38=0.00010996, DF401_39=-3.938e-05, DF401_40=-3.934e-05, DF401_41=-9.232e-05, DF401_42=0.0010432, DF401_43=0.0010432, DF401_44=0.00108102, DF402_01=14, DF402_02=4, DF402_03=9, DF402_04=8, DF402_05=3, DF402_06=15, DF402_07=9, DF402_08=2, DF402_09=6, DF402_10=1, DF402_11=9, DF402_12=14, DF402_13=4, DF402_14=13, DF402_15=4, DF402_16=6, DF402_17=7, DF402_18=9, DF402_19=5, DF402_20=0, DF402_21=6, DF402_22=15, DF402_23=15, DF402_24=15, DF402_25=15, DF402_26=15, DF402_27=15, DF402_28=15, DF402_29=15, DF402_30=15, DF402_31=15, DF402_32=15, DF402_33=15, DF402_34=15, DF402_35=15, DF402_36=15, DF402_37=15, DF402_38=15, DF402_39=15, DF402_40=15, DF402_41=15, DF402_42=15, DF402_43=15, DF402_44=15, DF420_01=1, DF420_02=1, DF420_03=1, DF420_04=1, DF420_05=1, DF420_06=1, DF420_07=1, DF420_08=1, DF420_09=1, DF420_10=1, DF420_11=1, DF420_12=1, DF420_13=1, DF420_14=1, DF420_15=1, DF420_16=1, DF420_17=1, DF420_18=1, DF420_19=1, DF420_20=1, DF420_21=1, DF420_22=1, DF420_23=1, DF420_24=1, DF420_25=1, DF420_26=1, DF420_27=1, DF420_28=1, DF420_29=1, DF420_30=1, DF420_31=1, DF420_32=1, DF420_33=1, DF420_34=1, DF420_35=1, DF420_36=1, DF420_37=1, DF420_38=1, DF420_39=1, DF420_40=1, DF420_41=1, DF420_42=1, DF420_43=1, DF420_44=1, DF403_01=63, DF403_02=63, DF403_03=63, DF403_04=63, DF403_05=63, DF403_06=32, DF403_07=0, DF403_08=0, DF403_09=0, DF403_10=0, DF403_11=0, DF403_12=0, DF403_13=46, DF403_14=47, DF403_15=47, DF403_16=46, DF403_17=50, DF403_18=51, DF403_19=51, DF403_20=47, DF403_21=45, DF403_22=35, DF403_23=35, DF403_24=45, DF403_25=41, DF403_26=27, DF403_27=27, DF403_28=39, DF403_29=41, DF403_30=36, DF403_31=36, DF403_32=37, DF403_33=36, DF403_34=17, DF403_35=17, DF403_36=33, DF403_37=47, DF403_38=38, DF403_39=39, DF403_40=45, DF403_41=46, DF403_42=36, DF403_43=36, DF403_44=41)>
Data bytes received: 471
<RTCM(1094, DF002=1094, DF003=0, GNSSEpoch=166960000, DF393=1, DF409=4, DF001_7=0, DF411=1, DF412=0, DF417=0, DF418=0, DF394=148654798856519680, NSat=6, DF395=134287616, NSig=4, DF396=16777215, DF400_01=0.00063562, DF400_02=0.00034088, DF400_03=-0.0008139, DF400_04=-0.00027132, DF400_05=-0.00079119, DF400_06=-0.00085056, DF400_07=-0.00070149, DF400_08=0.00054318, DF400_09=-0.00067657, DF400_10=0.00054401, DF400_11=-0.00067681, DF400_12=-0.0007062, DF400_13=-0.00095075, DF400_14=-0.00095087, DF400_15=0.00026971, DF400_16=-0.00071502, DF400_17=-0.00071478, DF400_18=-0.00095904, DF400_19=-0.000471, DF400_20=-0.00043929, DF400_21=-0.00043774, DF400_22=5.037e-05, DF400_23=0.00053853, DF400_24=0.00021714, DF401_01=-0.00303585, DF401_02=0.00199467, DF401_03=0.00248398, DF401_04=-0.00232298, DF401_05=-0.00364087, DF401_06=0.00192578, DF401_07=0.00203523, DF401_08=0.00204951, DF401_09=0.00204801, DF401_10=0.00093637, DF401_11=0.00100198, DF401_12=0.00100524, DF401_13=0.00100604, DF401_14=0.00057825, DF401_15=0.0006052, DF401_16=0.00060336, DF401_17=0.00059891, DF401_18=0.00155793, DF401_19=0.00161519, DF401_20=0.00160277, DF401_21=0.00159427, DF401_22=-0.00068408, DF401_23=-0.00052607, DF401_24=-0.0005276, DF402_01=14, DF402_02=14, DF402_03=11, DF402_04=0, DF402_05=2, DF402_06=0, DF402_07=7, DF402_08=2, DF402_09=12, DF402_10=9, DF402_11=0, DF402_12=2, DF402_13=0, DF402_14=8, DF402_15=9, DF402_16=6, DF402_17=0, DF402_18=8, DF402_19=4, DF402_20=1, DF402_21=14, DF402_22=4, DF402_23=2, DF402_24=1, DF420_01=0, DF420_02=0, DF420_03=0, DF420_04=0, DF420_05=0, DF420_06=0, DF420_07=0, DF420_08=1, DF420_09=1, DF420_10=0, DF420_11=0, DF420_12=1, DF420_13=1, DF420_14=1, DF420_15=1, DF420_16=1, DF420_17=1, DF420_18=1, DF420_19=1, DF420_20=1, DF420_21=1, DF420_22=1, DF420_23=1, DF420_24=1, DF403_01=63, DF403_02=63, DF403_03=63, DF403_04=63, DF403_05=63, DF403_06=63, DF403_07=63, DF403_08=63, DF403_09=63, DF403_10=63, DF403_11=63, DF403_12=63, DF403_13=63, DF403_14=63, DF403_15=0, DF403_16=0, DF403_17=0, DF403_18=0, DF403_19=44, DF403_20=50, DF403_21=49, DF403_22=49, DF403_23=41, DF403_24=45)>
^CConnection Terminated by User
Nice! I am also experimenting a bit on my side, just got myself a ublox F9P breakout board, and am connecting to a UNAVCO NTRIP server. You can see some of the code I wrote here: https://github.com/jakepoz/deweeder/blob/main/src/ntrip.py
My goal is to write everything using python async generators. So, with my code you just await the ntrip_client and it will continuously yield messages that you can pass along to your serial port to the GPS. (In my case, I am also mixing a regular flow of message traffic to the GPS on the same serial port)
In my case, it turned out I didn't need to fully parse RTCM messages, just extract complete packets one at a time from the byte-stream that the NTRIP server is sending, so that you send complete messages one-at-a-time to the serial port.
Hi @jakepoz Sounds interesting.
I've just posted a somewhat rudimentary NTRIP client to the \examples folder. It's clearly not a production-strength implementation but it hopefully demonstrates the basic concepts and provides a basis for further development.
I've just had a quick look at your code and the async HTTP methodology is clearly more advanced than mine; if you fancy collaborating on a more sophisticated implementation, I'd certainly welcome your contributions. Otherwise, I'd be more than happy to add a reference link to your project.
As you say, if the intention is to receive RTCM3 messages from a designated NTRIP server and simply pass these on to a local GNSS device via UART (or even a remote device via Zigbee radio or UDP socket), you don't actually need to decode the payloads - you can simply forward the individual binary messages.
Out of interest, is it possible to implement an aiohttp bitstream as a stream class and use this directly in RTCMReader?
FYI I've only recently started to explore RTK / NTRIP as part of my SEMU GNSS library suite, as I wasn't sure if there was any significant appetite for Python implementations given the ubiquity of utilities like RTKLIB, gpsd etc. But it appears there is, so I'm hoping to commit more resources to this in future releases (including acquiring the definitive NTRIP protocol specifications rather than relying on public domain resources!). As I say, contributions very welcome.
@jakepoz FYI I've just added a new branch add_parse_from_buffer which implements a new static method RTCMReader.parse_buffer() along the lines of the parse_rtcm() routine you use in your ntrip.py code. This supplements the existing RTCMReader.parse() method and allows users to parse data from both a serial stream or a buffer.
I've amended my simple ntripclient.py example to use this new method - hopefully others can do the same in their NTRIP client implementations.
I'll close this issue once this branch is successfully merged, but if you have any further comments or suggestions, by all means raise them.
Thank you! I'm still learning about the async stuff, this is my practical project in order to get the hang of it, so I'm not sure about your question. But I will be taking a look as I can here!
Hello.
Thanks for your work.
Sometimes the script ends with this error.
I think I fixed that in the latest version of my script: https://github.com/jakepoz/deweeder/blob/main/src/ntrip.py
Basically if the buffer doesn't have enough bytes to parse it needs to wait to fill up.
Hi @foxittt (and @jakepoz!) This should now be fixed in pyrtcm version 0.2.1-beta. Please install this latest version of pyrtcm and ensure you are using the very latest version of ntripclient.pyin the \examples folder.
If you don't mind me asking, which NTRIP server, port and mountpoint were you using i.e. what does your command line look like? (no need to disclose user or password)
>>> python3 ntripclient.py server=??.??.??.?? port=???? mountpoint=???????? ...
If you have any further problems, feel free to raise another issue.
Please, check again.
If you don't mind me asking, which NTRIP server, port and mountpoint were you using i.e. what does your command line look like? (no need to disclose user or password)
>>> python3 ntripclient.py server=??.??.??.?? port=???? mountpoint=???????? ...
python ntripclient.py server=rt.igs.org port=2101 mountpoint=SSRC01IGS1
Please, check again.
Hi @foxittt you need to be running the very latest version of pyrtcm (0.2.1) - this error message suggests you're still running an older version (<=0.1.8) which doesn't have the parse_buffer() function. Try upgrading using pip:
> python3 -m pip install --upgrade pyrtcm
> python3 -m pip show pyrtcm
Name: pyrtcm
Version: 0.2.1
Summary: RTCM3 Protocol Parser
Home-page: https://github.com/semuconsulting/pyrtcm
Author: semuadmin
Author-email: semuadmin@semuconsulting.com
License: BSD 3-Clause 'Modified' License
Location: /Users/semuadmin/Library/Python/3.10/lib/python/site-packages
Requires: pyserial
Required-by: PyGPSClient
If you still get the same issue after this, it's possible you may have multiple versions of pyrtcm in different site_packages directories hanging over from previous installations. Run the following command repeatedly until all previous versions are removed and you get the 'not installed' warning, then run the installation command above again.
> python3 -m pip uninstall pyrtcm
WARNING: Skipping pyrtcm as it is not installed.
Please appreciate that my ntripclient.py is just an example implementation of a simple ntrip client, to illustrate the basic concepts involved and provide a basis for further development. It's not intended to be a 'production strength' application.
The script works.
But the previous error still appears.
It always appears in the order shown in the screenshot below.
Ok sorry you're still having problems, and thanks for your patience. I'm reopening this issue and will keep it open until you're happy with the resolution.
I believe message type 1042 is a relatively new (>=RTCM v3.3) message type containing Beidou Satellite Ephemeris Data which is not defined in the 3.2 standard (which was the original basis for my code). I'll get round to updating to RTCM 3.3 in due course, but this alone should not be causing the error you're seeing.
I've spotted a schoolboy error in my original fix - sorry about that - and hope to have this resolved asap.
Hi @foxittt OK this should hopefully be properly fixed in v0.2.3, which you'll need to upgrade to before retrying.
This version also includes the missing emphemerides message types (1041, 1042, 1044, 1045, 1046) so you should see these parsed correctly in your output.
I'll keep the issue open until you're able to confirm. Thanks again for your patience and support.
Hi
I've been testing the script for the last few days.
It works without errors.
Hi @foxittt Thanks for the update. I'll close this now.
FYI I'm working on adding basic NTRIP Client functionality to the PyGPSClient GUI application:
https://github.com/semuconsulting/PyGPSClient/tree/add-NTRIP-client
Hi @foxittt Thanks for the update. I'll close this now.
FYI I'm working on adding basic NTRIP Client functionality to the
PyGPSClientGUI application: https://github.com/semuconsulting/PyGPSClient/tree/add-NTRIP-client
Is it possible to test new functionality?
Hi @foxittt Thanks for the update. I'll close this now.
FYI I'm working on adding basic NTRIP Client functionality to thePyGPSClientGUI application: https://github.com/semuconsulting/PyGPSClient/tree/add-NTRIP-clientIs it possible to test new functionality?
The PyGPSClient NTRIP Client enhancements are still a work in progress, but you're welcome to install the add-NTRIP-client branch manually and have a play yourself.
Always happy to receive feature requests for new functionality in any of the SEMU Python GNSS libraries.
In relation to NTRIP services, just be mindful that some commercial NTRIP services take a dim view of using their services to test or evaluate new software. There are alternatives, such as setting up a local SNIP LITE server on your own machine, but this requires some prior knowledge of the tech.
Please, check again.
Hi @foxittt
FYI I'm in the processing of moving some of the reader functionality which is common to pyubx2, pynmeagps and pyrtcm into a separate utility library pygnssutils. This contains more fully-featured RTK and NTRIP classes and examples.
You might like to take a look at:
https://github.com/semuconsulting/pygnssutils#gnssntripclient
https://github.com/semuconsulting/pygnssutils/blob/main/examples/rtk_example.py
Some of the examples in this library will in due course be moving to pygnssutils, which will hopefully minimise the chances of this kind of issue regression happening.
Hope this helps
Should be sorted now From: foxittt @.> Date: Wednesday, 29 June 2022 at 07:30 To: semuconsulting/pyrtcm @.> Cc: semuadmin @.>, State change @.> Subject: Re: [semuconsulting/pyrtcm] Ntrip client + pyrtcm (Issue #1) Please, check again. [image]https://user-images.githubusercontent.com/2026553/176366952-377a9010-9cfa-4bab-93df-5f7938b510c7.png [image]https://user-images.githubusercontent.com/2026553/176367042-7d887a56-c324-4099-a48a-508d7a67ab72.png — Reply to this email directly, view it on GitHub<#1 (comment)>, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AGZ7C3YYAALYNFD3BURIJRTVRPUQLANCNFSM5RKU4QOA. You are receiving this because you modified the open/close state.Message ID: @.***>
Hi @semuadmin
Still an error
Please, check again.
Hi @foxittt
FYI I'm in the processing of moving some of the reader functionality which is common to
pyubx2,pynmeagpsandpyrtcminto a separate utility librarypygnssutils. This contains more fully-featured RTK and NTRIP classes and examples.You might like to take a look at:
https://github.com/semuconsulting/pygnssutils#gnssntripclient https://github.com/semuconsulting/pygnssutils/blob/main/examples/rtk_example.py
Some of the examples in this library will in due course be moving to
pygnssutils, which will hopefully minimise the chances of this kind of issue regression happening.Hope this helps
Hi @semuadmin
It is very interesting and useful library.
Is it possible to add such functionality:
to format rtcm output as table like this
Hi @foxittt
What error are you still getting with the ntripclient.py example? Are you definitely running the latest updated example code with the latest version of pyrtcm (v0.2.6)?
It seems to be working fine for me...
> python3 ntripclient.py server=rtk2go.com port=2101 mountpoint=WEBBPARTNERS user=anon password=anon
Connected to: rtk2go.com:2101
GET /WEBBPARTNERS HTTP/1.1
User-Agent: pyrtcm NTRIP client/0.1
Authorization: Basic YW5vbjphbm9u
Ntrip-Version: Ntrip/2.0
*** Start Of Header ***
HTTP/1.1 200 OK
Date: Thu, 30 June 2022 07:28:45 UTC
Server: SubCarrier Systems Corp SNIP simpleNTRIP_Caster_[3.05.00]RwPRO
Ntrip-Version: Ntrip/2.0
Cache-Control: no-store, no-cache, max-age=0
Pragma: no-cache
Connection: close
Content-Type: gnss/data
*** End Of Header ***
<RTCM(1077, DF002=1077, DF003=0, GNSSEpoch=372545000, DF393=1, DF409=0, DF001_7=0, DF411=0, DF412=0, DF417=0, DF418=0, DF394=1765781494858514432, NSat=11, DF395=1073807360, NSig=2, DF396=4176895, DF405_01=0.00061272, DF405_02=0.00027956, DF405_03=0.00061271, DF405_04=0.00052751, DF405_05=0.00054932, DF405_06=0.0, DF405_07=3.7e-07, DF405_08=0.00040184, DF405_09=0.00084047, DF405_10=0.00097229, DF405_11=0.0008106, DF405_12=3.615e-05, DF405_13=0.0004822, DF405_14=0.00050736, DF405_15=-0.00036199, DF405_16=0.00020313, DF405_17=-0.00018435, DF405_18=3.147e-05, DF405_19=-0.00044269, DF405_20=-1.428e-05, DF405_21=-0.00017828, DF405_22=-0.0003604, DF406_01=-0.00019168, DF406_02=0.00243546, DF406_03=-0.00186527, DF406_04=-0.00341065, DF406_05=0.00045907, DF406_06=-0.00150083, DF406_07=0.00168963, DF406_08=0.00158228, DF406_09=-0.00048309, DF406_10=4.093e-05, DF406_11=-0.00120554, DF406_12=-0.0021954, DF406_13=0.00161642, DF406_14=0.00152614, DF406_15=0.00089166, DF406_16=-0.00012048, DF406_17=-0.00023344, DF406_18=0.00204755, DF406_19=0.00200356, DF406_20=-0.00368327, DF406_21=-0.0037881, DF406_22=0.00066355, DF407_01=298, DF407_02=139, DF407_03=17, DF407_04=745, DF407_05=1, DF407_06=773, DF407_07=163, DF407_08=867, DF407_09=58, DF407_10=503, DF407_11=908, DF407_12=32, DF407_13=66, DF407_14=692, DF407_15=4, DF407_16=182, DF407_17=835, DF407_18=354, DF407_19=616, DF407_20=237, DF407_21=303, DF407_22=255, DF420_01=0, DF420_02=0, DF420_03=0, DF420_04=0, DF420_05=0, DF420_06=1, DF420_07=1, DF420_08=0, DF420_09=0, DF420_10=1, DF420_11=0, DF420_12=1, DF420_13=0, DF420_14=1, DF420_15=1, DF420_16=0, DF420_17=1, DF420_18=1, DF420_19=1, DF420_20=1, DF420_21=1, DF420_22=1, DF408_01=10432, DF408_02=8000, DF408_03=2112, DF408_04=10128, DF408_05=8304, DF408_06=10224, DF408_07=6800, DF408_08=2960, DF408_09=2944, DF408_10=8576, DF408_11=8576, DF408_12=3456, DF408_13=3456, DF408_14=13968, DF408_15=2192, DF408_16=2192, DF408_17=416, DF408_18=1440, DF408_19=1680, DF408_20=6800, DF408_21=6736, DF408_22=9824, DF404_01=0.5435, DF404_02=1.0089, DF404_03=-0.7368, DF404_04=-1.6384, DF404_05=0.0328, DF404_06=0.9738, DF404_07=-0.7888, DF404_08=0.9481, DF404_09=0.036, DF404_10=1.1787, DF404_11=-1.6072, DF404_12=1.025, DF404_13=0.8576, DF404_14=1.1782, DF404_15=-1.616, DF404_16=1.1786, DF404_17=-0.7824, DF404_18=1.0764, DF404_19=-0.9348, DF404_20=-0.9166, DF404_21=0.557, DF404_22=0.5844)>
<RTCM(1087, DF002=1087, DF003=0, GNSSEpoch=574597912, DF393=1, DF409=0, DF001_7=0, DF411=0, DF412=0, DF417=0, DF418=0, DF394=9277679115173888000, NSat=7, DF395=1090519040, NSig=2, DF396=16127, DF405_01=0.00058938, DF405_02=-0.0008167, DF405_03=0.00049059, DF405_04=0.00061573, DF405_05=-0.00076875, DF405_06=-0.00033882, DF405_07=-0.00073091, DF405_08=-1.645e-05, DF405_09=-0.00094597, DF405_10=2.089e-05, DF405_11=0.00041193, DF405_12=-0.00036437, DF405_13=0.00039156, DF405_14=3.44e-05, DF406_01=-0.00198338, DF406_02=0.00070582, DF406_03=-0.00191314, DF406_04=0.00101879, DF406_05=0.00066027, DF406_06=-0.00355762, DF406_07=-0.00115241, DF406_08=-0.00086473, DF406_09=-7.867e-05, DF406_10=-0.00142049, DF406_11=-0.00162673, DF406_12=0.00015748, DF406_13=0.00155485, DF406_14=-0.00066995, DF407_01=894, DF407_02=688, DF407_03=477, DF407_04=408, DF407_05=26, DF407_06=386, DF407_07=700, DF407_08=685, DF407_09=755, DF407_10=807, DF407_11=90, DF407_12=527, DF407_13=995, DF407_14=433, DF420_01=0, DF420_02=0, DF420_03=0, DF420_04=0, DF420_05=0, DF420_06=0, DF420_07=0, DF420_08=0, DF420_09=1, DF420_10=0, DF420_11=1, DF420_12=1, DF420_13=0, DF420_14=0, DF408_01=5872, DF408_02=16064, DF408_03=2832, DF408_04=16336, DF408_05=2640, DF408_06=5664, DF408_07=15904, DF408_08=5728, DF408_09=608, DF408_10=544, DF408_11=4672, DF408_12=7744, DF408_13=7776, DF408_14=7776, DF404_01=1.5517, DF404_02=0.5215, DF404_03=0.3042, DF404_04=0.0005, DF404_05=0.4224, DF404_06=0.6277, DF404_07=-1.2108, DF404_08=0.5636, DF404_09=-0.8, DF404_10=0.5636, DF404_11=-1.622, DF404_12=0.4356, DF404_13=-0.7967, DF404_14=0.0737)>
Connection Terminated by User
If you're still getting errors with this after checking you're using the latest code, could I ask you to raise a new issue? This one is getting a little long now. Thanks.
Hi @foxittt In answer to your question about pygnssutils, it should be perfectly possible to use the pygnssutils library as the basis for an application that formats the RTCM data in the tabular way you've illustrated, but you'd need to write the app yourself. I'm not in a position to write your application code for you.
Hi @foxittt
What error are you still getting with the
ntripclient.pyexample? Are you definitely running the latest updated example code with the latest version of pyrtcm (v0.2.6)?It seems to be working fine for me...
> python3 ntripclient.py server=rtk2go.com port=2101 mountpoint=WEBBPARTNERS user=anon password=anon Connected to: rtk2go.com:2101 GET /WEBBPARTNERS HTTP/1.1 User-Agent: pyrtcm NTRIP client/0.1 Authorization: Basic YW5vbjphbm9u Ntrip-Version: Ntrip/2.0 *** Start Of Header *** HTTP/1.1 200 OK Date: Thu, 30 June 2022 07:28:45 UTC Server: SubCarrier Systems Corp SNIP simpleNTRIP_Caster_[3.05.00]RwPRO Ntrip-Version: Ntrip/2.0 Cache-Control: no-store, no-cache, max-age=0 Pragma: no-cache Connection: close Content-Type: gnss/data *** End Of Header *** <RTCM(1077, DF002=1077, DF003=0, GNSSEpoch=372545000, DF393=1, DF409=0, DF001_7=0, DF411=0, DF412=0, DF417=0, DF418=0, DF394=1765781494858514432, NSat=11, DF395=1073807360, NSig=2, DF396=4176895, DF405_01=0.00061272, DF405_02=0.00027956, DF405_03=0.00061271, DF405_04=0.00052751, DF405_05=0.00054932, DF405_06=0.0, DF405_07=3.7e-07, DF405_08=0.00040184, DF405_09=0.00084047, DF405_10=0.00097229, DF405_11=0.0008106, DF405_12=3.615e-05, DF405_13=0.0004822, DF405_14=0.00050736, DF405_15=-0.00036199, DF405_16=0.00020313, DF405_17=-0.00018435, DF405_18=3.147e-05, DF405_19=-0.00044269, DF405_20=-1.428e-05, DF405_21=-0.00017828, DF405_22=-0.0003604, DF406_01=-0.00019168, DF406_02=0.00243546, DF406_03=-0.00186527, DF406_04=-0.00341065, DF406_05=0.00045907, DF406_06=-0.00150083, DF406_07=0.00168963, DF406_08=0.00158228, DF406_09=-0.00048309, DF406_10=4.093e-05, DF406_11=-0.00120554, DF406_12=-0.0021954, DF406_13=0.00161642, DF406_14=0.00152614, DF406_15=0.00089166, DF406_16=-0.00012048, DF406_17=-0.00023344, DF406_18=0.00204755, DF406_19=0.00200356, DF406_20=-0.00368327, DF406_21=-0.0037881, DF406_22=0.00066355, DF407_01=298, DF407_02=139, DF407_03=17, DF407_04=745, DF407_05=1, DF407_06=773, DF407_07=163, DF407_08=867, DF407_09=58, DF407_10=503, DF407_11=908, DF407_12=32, DF407_13=66, DF407_14=692, DF407_15=4, DF407_16=182, DF407_17=835, DF407_18=354, DF407_19=616, DF407_20=237, DF407_21=303, DF407_22=255, DF420_01=0, DF420_02=0, DF420_03=0, DF420_04=0, DF420_05=0, DF420_06=1, DF420_07=1, DF420_08=0, DF420_09=0, DF420_10=1, DF420_11=0, DF420_12=1, DF420_13=0, DF420_14=1, DF420_15=1, DF420_16=0, DF420_17=1, DF420_18=1, DF420_19=1, DF420_20=1, DF420_21=1, DF420_22=1, DF408_01=10432, DF408_02=8000, DF408_03=2112, DF408_04=10128, DF408_05=8304, DF408_06=10224, DF408_07=6800, DF408_08=2960, DF408_09=2944, DF408_10=8576, DF408_11=8576, DF408_12=3456, DF408_13=3456, DF408_14=13968, DF408_15=2192, DF408_16=2192, DF408_17=416, DF408_18=1440, DF408_19=1680, DF408_20=6800, DF408_21=6736, DF408_22=9824, DF404_01=0.5435, DF404_02=1.0089, DF404_03=-0.7368, DF404_04=-1.6384, DF404_05=0.0328, DF404_06=0.9738, DF404_07=-0.7888, DF404_08=0.9481, DF404_09=0.036, DF404_10=1.1787, DF404_11=-1.6072, DF404_12=1.025, DF404_13=0.8576, DF404_14=1.1782, DF404_15=-1.616, DF404_16=1.1786, DF404_17=-0.7824, DF404_18=1.0764, DF404_19=-0.9348, DF404_20=-0.9166, DF404_21=0.557, DF404_22=0.5844)> <RTCM(1087, DF002=1087, DF003=0, GNSSEpoch=574597912, DF393=1, DF409=0, DF001_7=0, DF411=0, DF412=0, DF417=0, DF418=0, DF394=9277679115173888000, NSat=7, DF395=1090519040, NSig=2, DF396=16127, DF405_01=0.00058938, DF405_02=-0.0008167, DF405_03=0.00049059, DF405_04=0.00061573, DF405_05=-0.00076875, DF405_06=-0.00033882, DF405_07=-0.00073091, DF405_08=-1.645e-05, DF405_09=-0.00094597, DF405_10=2.089e-05, DF405_11=0.00041193, DF405_12=-0.00036437, DF405_13=0.00039156, DF405_14=3.44e-05, DF406_01=-0.00198338, DF406_02=0.00070582, DF406_03=-0.00191314, DF406_04=0.00101879, DF406_05=0.00066027, DF406_06=-0.00355762, DF406_07=-0.00115241, DF406_08=-0.00086473, DF406_09=-7.867e-05, DF406_10=-0.00142049, DF406_11=-0.00162673, DF406_12=0.00015748, DF406_13=0.00155485, DF406_14=-0.00066995, DF407_01=894, DF407_02=688, DF407_03=477, DF407_04=408, DF407_05=26, DF407_06=386, DF407_07=700, DF407_08=685, DF407_09=755, DF407_10=807, DF407_11=90, DF407_12=527, DF407_13=995, DF407_14=433, DF420_01=0, DF420_02=0, DF420_03=0, DF420_04=0, DF420_05=0, DF420_06=0, DF420_07=0, DF420_08=0, DF420_09=1, DF420_10=0, DF420_11=1, DF420_12=1, DF420_13=0, DF420_14=0, DF408_01=5872, DF408_02=16064, DF408_03=2832, DF408_04=16336, DF408_05=2640, DF408_06=5664, DF408_07=15904, DF408_08=5728, DF408_09=608, DF408_10=544, DF408_11=4672, DF408_12=7744, DF408_13=7776, DF408_14=7776, DF404_01=1.5517, DF404_02=0.5215, DF404_03=0.3042, DF404_04=0.0005, DF404_05=0.4224, DF404_06=0.6277, DF404_07=-1.2108, DF404_08=0.5636, DF404_09=-0.8, DF404_10=0.5636, DF404_11=-1.622, DF404_12=0.4356, DF404_13=-0.7967, DF404_14=0.0737)> Connection Terminated by UserIf you're still getting errors with this after checking you're using the latest code, could I ask you to raise a new issue? This one is getting a little long now. Thanks.
Hi @foxittt In answer to your question about
pygnssutils, it should be perfectly possible to use the pygnssutils library as the basis for an application that formats the RTCM data in the tabular way you've illustrated, but you'd need to write the app yourself. I'm not in a position to write your application code for you.
I`ll try
If you're still getting the RTCMReader has no attribute 'parse_buffer' error message, you're still using the old version of ntripclient.py. Download the new version, updated yesterday, and use this instead.
Thanks.
Now everything is working.
A word of warning - pyrtcm can parse RTCM3 data, but if you're looking to write an application to analyse the RTCM3 data, you may struggle without access to the full RTCM3 specification. Unfortunately the RTCM3 protocol is proprietary and the specification costs $340 to download from here:
For copyright reasons I'm not permitted to reproduce the full specification here.
Thanks for the info
Hi @foxittt
I've deleted your recent message as it contains a link to content which is almost certainly in breach of RTCM's copyright. It also referred to the older 3.2 standard - the latest is 3.3.
You can find all sorts of things on the internet, but they're not necessarily legal ;-)
Hi @semuadmin
Sorry,
I won't post anymore such links.
Hi @foxittt
In pyrtcm, the number of satellites contained as a bitmask in DF394 is already rendered as the attribute NSat for convenience; similarly DS395 is rendered as NSig. The attribute NCells = NSat * NSig and this represents the number of repeating groups in the MSM messages. e.g.
<RTCM(1077, DF002=1077, DF003=0, GNSSEpoch=304671000, DF393=1, DF409=0, DF001_7=0, DF411=0, DF412=0, DF417=0, DF418=0, DF394=11620426274396176384, NSat=9, DF395=1073807360, NSig=2, DF396=262063, DF405_01=0.00052882, ...)>
Your recent messages have been deleted AGAIN as a screenshot also potentially constitutes infringement of the copyright. If there is any further infringement I will have no choice but to block you from the repo.
In answer to your question, this is application code which you need to work out for yourself.
PLEASE - NO FURTHER QUERIES AGAINST THIS CLOSED ISSUE.
@foxittt - a friendly gesture, hope this helps but no follow-ups please :-) ...
If I understand what you're after correctly, try something like this:
"""
prnsigmap.py
Illustration of how to map satellite PRN to signal ID from
values of DF394, DF395 and DF396 in RTCM3 MSM7 messages (e.g. 1077).
I can't swear that this is 100% correct, but this is my interpretation
of the RTCM3 specification for MSM messages.
"""
# from RTCM3 specification; similar maps are given for GAL, GLO, etc.
GPS_PRN_MAP = {
1: "1",
2: "2",
3: "3",
4: "4",
5: "5",
6: "6",
7: "7",
8: "8",
9: "9",
10: "10",
11: "11",
12: "12",
13: "13",
14: "14",
15: "15",
16: "16",
17: "17",
18: "18",
19: "19",
20: "20",
21: "21",
22: "22",
23: "23",
24: "24",
25: "25",
26: "26",
27: "27",
28: "28",
29: "29",
30: "30",
31: "31",
32: "32",
33: "33",
34: "34",
35: "35",
36: "36",
37: "37",
38: "38",
39: "39",
40: "40",
41: "41",
42: "42",
43: "43",
44: "44",
45: "45",
46: "46",
47: "47",
48: "48",
49: "49",
50: "50",
51: "51",
52: "52",
53: "53",
54: "54",
55: "55",
56: "56",
57: "57",
58: "58",
59: "59",
60: "60",
61: "61",
62: "62",
63: "63",
64: "Reserved",
}
# from RTCM3 specification; similar maps are given for GAL, GLO, etc.
GPS_SIG_MAP = {
1: "Reserved",
2: "1C",
3: "1P",
4: "1W",
5: "Reserved",
6: "Reserved",
7: "Reserved",
8: "2C",
9: "2P",
10: "2W",
11: "Reserved",
12: "Reserved",
13: "Reserved",
14: "Reserved",
15: "2S",
16: "2L",
17: "2X",
18: "Reserved",
19: "Reserved",
20: "Reserved",
21: "Reserved",
22: "5I",
23: "5Q",
24: "5X",
25: "Reserved",
26: "Reserved",
27: "Reserved",
28: "Reserved",
29: "Reserved",
30: "1S",
31: "1L",
32: "1X",
}
def cell_2_prn(prnmap: dict, sigmap: dict, df394: int, df395: int, df396: int):
"""
Map Satellite PRN and Signal ID
"""
print("\nSatellite PRN mapping from DF394...")
sats = []
nsat = 0
for idx in range(64, 0, -1):
if df394 & pow(2, idx):
prn = prnmap[64 - idx]
nsat += 1
sats.append(prn)
print(f"NSat: {nsat}; Satellite PRNs: {sats}\n")
print("Signal ID mapping from DF395...")
sigs = []
nsig = 0
for idx in range(32, 0, -1):
if df395 & pow(2, idx):
sig = sigmap[32 - idx]
nsig += 1
sigs.append(sig)
print(f"NSig: {nsig}; Signal IDs: {sigs}\n")
print("Mapping of Cell to PRN to Signal ID from DF396...")
cln = nsat * nsig
ncell = 0
for prn in range(nsat):
for sig in range(nsig):
cln -= 1
if df396 & pow(2, cln):
print(f"cell: {ncell+1:02d} prn: {sats[prn]} sig: {sigs[sig]}")
else:
print(f"cell: {ncell+1:02d} prn: {sats[prn]} sig: n/a")
ncell += 1
DF394 = 11620426274396176384
print(f"DF394 in binary: {DF394:0>64b}")
DF395 = 1073807360
print(f"DF395 in binary: {DF395:0>32b}")
DF396 = 262063
print(f"DF396 in binary: {DF396:0>18b}")
cell_2_prn(GPS_PRN_MAP, GPS_SIG_MAP, DF394, DF395, DF396)> python3 prnsigmap.py
DF394 in binary: 1010000101000100000011000010000100000000000000000000000000000000
DF395 in binary: 01000000000000010000000000000000
DF396 in binary: 111111111110101111
Satellite PRN mapping from DF394...
NSat: 9; Satellite PRNs: ['1', '3', '8', '10', '14', '21', '22', '27', '32']
Signal ID mapping from DF395...
NSig: 2; Signal IDs: ['1C', '2L']
Mapping of Cell to PRN to Signal ID from DF396...
cell: 01 prn: 1 sig: 1C
cell: 02 prn: 1 sig: 2L
cell: 03 prn: 3 sig: 1C
cell: 04 prn: 3 sig: 2L
cell: 05 prn: 8 sig: 1C
cell: 06 prn: 8 sig: 2L
cell: 07 prn: 10 sig: 1C
cell: 08 prn: 10 sig: 2L
cell: 09 prn: 14 sig: 1C
cell: 10 prn: 14 sig: 2L
cell: 11 prn: 21 sig: 1C
cell: 12 prn: 21 sig: n/a
cell: 13 prn: 22 sig: 1C
cell: 14 prn: 22 sig: n/a
cell: 15 prn: 27 sig: 1C
cell: 16 prn: 27 sig: 2L
cell: 17 prn: 32 sig: 1C
cell: 18 prn: 32 sig: 2LThanks!