semuconsulting/pyspartn

SPARTN-1X-OCB-GLO bad decryption

jonathanmuller opened this issue · 15 comments

jonathanmuller commented

Decryption of SPARTN-1X-OCB-GLO doesn't seem to be correct. The output result looks random (not a single field looks correct)

It looks to be related to the ambiguity resolution as SPARTN-1X-OCB-GLO are the only messages with 16 bits GNSS time tag defined in GLONASS epoch (rather than being 32 bits with no ambiguity or defined in GPS epoch)

Ideally a parameter of one of the function in the chain should specify if GPS time needs to be used or GLONASS
I'm not sure how to solve it cleanly for now but I'll think about it and try to come with a proposal (or maybe @semuadmin has an idea on how to do that properly)

image

def convert_timetag(timetag16: int, basedate: datetime = datetime.now()) -> int:
    ...
    ...
    base16 = datetime(basedate.year, basedate.month, basedate.day, 0, 0, 0)
    secs = timetag16
    if basedate.hour >= 12:
        secs += 43200
    time16 = base16 + timedelta(seconds=secs)
    return date2timetag(time16)

Code to reproduce :

import datetime

from pyspartn import SPARTNReader, SPARTNMessage

msgs = []

key = "930d847b779b126863c8b3b2766ae7cc"

# msgs.append(b's\x00\x14h\x04\x120[\x11\x08(\x18\x14&\x98\x9e|\xdaz\x14\x01\xc0\xa5HS\xbc\xdb\xba\xc7\xcd\x83VuQ\xa5O\xc6\xc8]G\xe8\xc6t\xbcX\xd79\xe1*6\xa9\xfe\xc2')
# msgs.append(b's\x00\x0e\xec\x10\x1d [\x1e\xc8\xd6_\xf6q\xb8\x94\xdfja\x7fLh\x94\x89#G9\xbd%\xdfi\x15B\xc6\xe6y\xb3V\t\x04D)')
msgs.append(
    b's\x02\xb8j\x18\xd7)+p[\x10\x88\x04\xcf\xe7\xb3\xec\xf2l\xc1\xd9J\xd2qj\xc4\x8fW[\\\x08\x87\xe4\xa0\xb1\x1c\xb3\x96\xfc\xdci\x07\xf2\xf5\'5\t\x7f\xd0.\xe42\x0c\x0b\xb8\xd9r|\x8b\xcf\xd8c\x02\xe8t~\xfayqJHVU\xb9D\x1b\x02\xe1\xedK\xb6%p\xf1Z\xc2y+$M\xf9\xad\xa1?\xcb\xde6\xc0\x99\x02)\x7f\xde\xb1\xc85\xc7\xab\xc0x\x9f(\xc3K\xe66\xa0c\\4d\xf6\xb0\xaf-\x10x\x06\xa7\xadGl4\xec\xe6_\xa5;\xde\x12\xb4_\xdd\x10\x83J\xfb\xf6!\xf5]\xeaR\x08\x06)\x9fg\x03${\x10\xefVR+\xa7d=\x8d\x97U\xe7\x88\xc7\x01&\xccD#j\x19d\x87\x92">\xcb\xdaOET\xbfL\xfc\x8bQ{\xca\xb9\x89\xae\xe0\xd6\x7f\xcf\x154\n\xe7\x87Bau\xb5\xf3&\xcb\x8dp\xbbi\xcck\x99\xf4\xdfq\xd5\xa4\x85U\xb8\xb0NE\x8eO\x1a\xc9\xf6\xf3\x95\xcc\xc7\xe0\xa5~\x97\r\xca\x954.\x0e\x0c5\xfc\x011\xdbc\x8d\x1e\x8d\x84>\x07\x0cL\x87\xc9\xec\xa5\xecBo\xce\xc13:\xcd\x83\x824~\xd05\xa3\xcaL\xb1M\xc3d\xba\xe0ob\xceC\xf3\xa8Vw\xaakJ\xd8dT\x83D\x03g\xad\x04\xb0s\x90!BJ\xa8zh!\xa5\xe1\xffiYb\xf3\x16\x18\xbd\xca,\xe8v\xff\xf6gLq\xeb\xbf\xe0\x02c\xf0\xa8\xf9\xd3\x11\x07\x9f\x1a\xc3\xd4\x867\x89\xf1\x0e\xe6\x99\xf3\x06\x87\x1e-m\xd4\x9a\xf1\xec\r\xf2\xb4P8')
msgs.append(
    b's\x02\x9d\xeb\x18\xd7)+p[\x11\x08P\xe3T\xd7\xbe5\xd9Q\x86\x95\xc2\xe3x\xbdK\x8aF)\xdf\xe7\xe3\xf7H\x076G\xab\xa9:\xa1\xc0\xd5;\xd2}t\xbc\x80\xe5\x88\xc0\xa7\xba7\xb93\xc0\xc7\xab\xfd,@\xe5\x9f\xff6\xae\xac\xe8-,U(\xdf[\xe4\x0c\xa3d%l\x8a\xdb\xfd\xa2e\xc1\xbeR\x1c\xa2\x10\x01\rNJ\xd4-Z\x17}\xf2O\xf6\xb4\x1fH\xf3\xd2T\x12\xe2W\x13\x12@\xdf\x8a\x16\x85S\x10{{\x97\xbej$\xb5@\x98\xd3\xaf\x90\xee\xe2.\xef\x82\x1c\xca\x9f\xf80\x9a(\xbe\xe2\x97\xcf\x85\xa8\x98\xccy{R.\xe1\x0fa\xdcX\x02\x80\x18\xe0\xc7\xd6\xd9s\xf6\t\x1b.T\xb0z\xaa\x82\x1d\xb2\x0bu\x1b\xd5\x04E\xb6\xaa\xe3\x1b\xbc\xc91\x9c\xa9\xd8\xefA\xa2Z\xf8\xe9T\xc1{\xa6\xf2Rz|1/;\xbd\xf2+\xf6Y\x10\xa6O\xcd\x8b\xe7\x13\xabO\xf4d\x05\x8e&\xe8\xdaQ\xd9\xban:p\xb8\xe9\x12\xe5":\x82\xf6v4\x8e\xf6?\x05\xd6\\Hf\x9d\x85\x04\x93L\x07\xde)\xa0\xaf\x06\'\xae*1\xc7\xbb\xdcoWL\x02\xed&\xf7\x94v\xcdZ\xa4>\xf5\x90%3\xdf\x13\xe8y\x14y\x9d\x7f\xe3G\x95\x84R\x0fa\xb6\xbeP\xe4U,\xd8A+\xef-\xa0A>\xea\xb5\xdaM')
msgs.append(
    b's\x02\x97\xee\x18\xd7)+p[\x11H\xef\x15K\xc1\\\x10\rQ\xaa@\x12\xea\xb59=\t\x03\xca\xf5\xd7\t\xae\n\xae\x9a.|<\x8b\xf3r"N\x18I\xc98\x01\x15\xfa\x0f\xb5\x91\xe7%Ph\t\x93\xb41\xe3\xcc\x89\xf6\xcb\x9a\xe2\xb9|\x82?%$\x06\xe6\xfc\xfd\n\xd7\xdd\\\xdapV\xf5\xc8\x8a\x8a-0\xb4\xf5\x8d;\xbd\xcf7\xa8.3\x1aTe\xdc\xf9\xa9\xee!3Z\xe8X\x106\xebw\\J;\xf1\xdd;\x9c\x14"\xf1mj\xf9\xc4Nkpb&\x1f\x15\xb9\x1a|\xd3u\xf9\x1e\xa0\x8cq\x0c/\xe4\xa8\xfd\x10\xa8DQ\xfcTOU8j\xeeq\x15\xf6\x84w\x1c\x9f\xa2H\x84:\x88c#~p\xf03&\x0e\xdd5$\x98\xce&(\xd4\xd1@M\x83i\xe9\xd9\x9a\xaf\xca;K\xbe\xc5ZDAO\xe3\x02Q)\xd2/\xff\x06\xe2v!\x17\x9a\x00\xffW\x03\xcaV\xfe\x03kCA`\x05\xf1"\xb3X/%\xc6\x8cI\xbd\x91:\xd3ht1\x00^\x1d6}\xd1\x8e\xcbE\xb4\xce\xf1\xdeq,\x97<fT\xe25n\xce@\t\xd1\xd5m>I\x82xU\x82\x1b\xa6\xc3\x07\x82jm\x80p_\x10\x06\xc4EupT\x86cIk\x0b\x16\xf3h\xc6\xc0\xb6\xe0D')
msgs.append(
    b's\x02\xabj8\xd7\'\xda\x10[\x1f\xc8\xd1\x07\xfb\xde\x1a\x08\x90\\{\x7f;\xa1\xe5+2@y{\x89\xca\xad\x8a\x9f\x8fu\x8ahrCm\xae\xdb\xbdh\xac\x9a/\xfcb\x8eH\xcdE\xf1\x17#\xd1E&\x17\x87\xb5\x06\xcb\xddW\xea(\xbe\x19c\x0f`\xf7%\x82\xdf\x84\x8f\xbb<\xdb\xf1\xbeV\x86)\r\x95\x08\xe6\x12|\x83_\xea\xe7\x04$\x18\x86\x0f\x83\xef\x1f\xcc`V\x8a-\xe8\xdd\xf9\xceW\x1f\xc2\xc4"\x8c;\xf6;\xb5\xbf\xe3G(\xd4\xc5kT\x8f\xdb\xb1\xfap\xa0\xd3\xe3\xb2[ \x8a\x7f*\r\xc3\x05\xa4\x80)A\x05\xd1^\xd6\x87\x9e\x86\xfcH\xce\xa5\xd5\xbe{4\xa9\xaa?G\xdc\xc8\x89c2s\xccR\x90\xc2<\xf3\xf2\x13\xb0\xdc%>q\xc38\xf3 \xd4}\xfcST\xb8\xa9\xbb\xd2\x0fw\x156-5\x11#\xc3\xac\xf9\xe4\xf4\xe4\xdbU\x13\xb7\xc5\xdc\x9b\xca\xf3\xee\x0b!\x9aI\x1bP}\xc2\xcd+\x9b\x96{\x91\xcbL\x84T\xdb\t,\xcf\xa7\x9f\x93\x9b\x8d\x93\xb99\xbb\xd2\\;\xb5x\xc7\x9f%F\x95wzX\x01\x1d\x80\x15\xe0|Ix%\xe8\x90\xe3ps\x1f\x8e\x90+\xa5\xbfXB(\x85\xde\xb1\xffY\xe0N\xc2~\xb1\x01\xed\xd2\xe6\x00\x9c]\x9f%\x84\xcbB\xbf\x89\xb4\xda\x8b\xf6/\xaa\t\xc5\xb5\xe2&\x1e\xcdk\xe2r\x11\xf0\x0cz\x9b\xa9v\x99\xe0~\x8d\xf5W\xde\xcf\n\xac{\x88\x0e')
msgs.append(
    b's\x02\xc8\xee8\xd7\'\xda\x10[\x10\x08<uA)]O\xe24\x95\xbd)\x05\x9e)\x02M\x16\xb2\xdd\xbe\x1c\xce\x15\x0b\x03\xc2\xb1\x8e"\xcd\xd1\xce0\x8di}\xc5$\x8ezd\xc6\x83]QO\xbf\xf44\x95\nv\xb0\x16\xec\xc1\xc3\xc0\x9d\x95\xdc\x9cR\x07l\xda\xc8\x05{\x18|\x11\x81\xdc\x1c\x84\x14\x91\x87\xba\xbc\x05L\xa4\xaa\xed\x8d\rsfqOh\xd7\x17b\x92w\x13\x959\x9cT\x8d\xd7\x0f\xd4\xb0/\x8a\xcfH\xe9\xce\xd0\xd3\x0e\x85R\xb9N7\x18Q\xddK\xb2z\xe5\xc5\xef\x82\x06\x9b\x94\x0c\xa8Sl\xe6\xb0QN\xe7\xd0Y\xab\x8f\x03\xa9n \r\x0f`\xe3\xfd*\xcbN\xe3\x08\x08\x0c\xb5\xb7h:\x06\xe1\x98\xc6At\xdah\xcd\xd7JC\xe9\x0f\xca\xd8\x96\xcc\xd08\xdb\xabw\xa4\xa7+c\xff\x8b\x056\x98\xf1\xca\x19l\x04V!p\xe2\xf3K\xaf\xe2b\xae\x9f\xd3@\xcc\x8fM"n\xc6_9\xdf\x01\ta\xba3\xf1G\x17?\xbd\x8d\xc0\xf6\xb4#\xccRe\x8e<\x1c\xa8\xcc\x02\x11\x89\x18Z\xdd\xca\xdcms\x82\xbak\xf3P\x08y\x12a\xd2$\x9cn\xc2*6/\xc6(Ksc\xaf1\xb8k\xb9\'\xd0\x84\xee\x1b\x82^\xcdG\x10\xf1\xe8\xe5\xb8\x88\x81\xe7y\xbfZ\xa9\xb9"\xf1LpL\xb2\x06\xf9\x95O\xef\xf63\xe19\xb9\xe9\xa2\xe5\xba\xa7\xd7j\xbd\xdf1y_\xbe\xcc+\x936\x94;U"\x81\xac\x83_\xd7\x8b\xb9\xdb\x1e\x06\xbd\xcf\xd5pz\x8cS\xcb\xcb\xf1\x96\x04\xf5jUJ\xc9G\x83\xf6\xces\xc7\xad\xf1=\x00\xbe\x91\x134\xa9\xf3\x07M\x1d@aUPJ5z\x06\x9e\xcdv%')
msgs.append(
    b"s\x02\xce\xed8\xd7'\xda\x10[\x10\x88'\xe1\t\x8b\xdbq\x0c/\xc7\x88`\xd5\xe3\xb5\x90(C\xc3\xd9\xc2/:\xaa\x81\xd187i\xa5}\x03\xfe\x1e\xb6'\x15{c\xbdLr\x8b)\x1d[\xf4\x1fu\xce\xf3\xdc\xe3%\x00v\xc7\xc5\x986\xc9l\xd6\xe7\xec\x8e\xa5\x8f=\x89\xf1[C\xa9,B\x12\x97\xe7\xc4\xady\x13M\xc5XI\x8fV\xf3\x7f\xee:u\x1d}ORN\x97\x80\xb7\xcap\tgr\x06\xa7\xfc/NE6\x816\xd5\xb74.\r\x92\xb1\x8e\xf1n\xca\xa3$\xab\xee\x11\xec\n\xcc\x1d\x8f\xe6\x1d\x1b\xbe\xd97CQ\xca\xf7\xf78\xa0\xb5\x91\xfe\xa0\xdbv(@\x15A\x88\xe6\x92\xc3\xd4\x80\xbc\xd0sv\xf2P\x8a*:'\xc1\xa0'\xde\xed\xdb\nl\xcdm\x0f\x97\x15Z\xd3X[\xdd\x05\xe8Q\x96\x01q\xdb\xd1\x8cgI\xe1~5Y\xd8\x82\x9b#\xaa\xe6\xe9\x01\xf9\x0f\x15f)\xee\x03_s\xdb\xdeWh\xa9;\xbeC\xf4\xa2\xb5U\xe9\xf5\r\x1f,q\x8co\x9a\xef\x7f\xc9\x8e\xf7\xbe\xb6Rg\xe3\x99\xa3J\xf8.\x94]U\x9d\x1f\xa9F:\x93\xca\xce)\x92hF\xa9[!\x08\xf6\xce\xdfO`\xfe'\xad5\xa33\xda\x8aq\x80\x92\xdb\x06r\xc4S:\r\x13d$\xa8\x1e\xc8b\xbb\xed\xd9\xfaC\xb3\xee\xc0~<\xaa\x044Q\xca!>\xfe>\x84E\x8a\xf4*\x19\xb9X\xe9\x9c\x1b{\x95\x13\x1b$v\x045hy21\xa8D\x06\x9f7\x7f\xbf\x1b\x95\xd8\xea]-\xc7mM[\x85*\x10\xfc\xff\x1d\x87s\x82k\x92\xf02\xb6\xd0\xb7\xe5\xe4*\x92\xb1R\x90\xb4\xc3\x07\r\xda-$\xe2\xd4:\xe7\x95\x03\x02\x16/\x82o\x8ag\xc6\xdbz\xe8")
msgs.append(b"s\x02'`8\xd7'\xda\x10[\x10\xc8\xb8\xf0\x1b\xe6\xba\xc9:\x1a\xb6{\xae$cf\\k3\xed\xec\xac\xc4T\xad{\xbe:/7\x0b\x89zHa\x14\x91\xdfJ\xc3(\\\xad\x14\xc5I\x9d|\x8c\xc6\xd1\x97\x0fW\xd34\xa7\xf8\xcb\x1a\x1c\x0c6wT7:g\xfdineJ\x18\xb0\x8c\xb3\x96\x92u$\xf3\xd9")
msgs.append(
    b's\x02\xeb\xe5(\xd7\'\xda\x80[\x10\x08\x84\xc4\xf4\xb5r\x17\xb9e\x82\x9c#J\xe7|;\xce;NZ\x93\xf3\x80\x9c\xc7s\x1dG\xdc\xde\xfc\x92j\xe4-R\xb7\xcb\xac\x14\xff\x04`_xI\xdd\xb72\x06SC{c\x9e_R\x8e\x17\xfa\xf3r\x0b.\x9aC\xffr\x8dJ)\xeb\xb0Ik\x8d\xe5:\x8dU\xb7\x8f\xe4\x99\xeeD*\xc4\xf5\xbd\xb7-\xa4B\x11TV\x93%\xf3\xd9\xecLw\xca\xb7f\x8e\xe5\x1dq\xd8\xaeIY\xde\xeeRO\xd4\xb5\x86\xa7\x18\xe6 \xee\x9c\xb5&\xeee\xbb\xeb1\xa5\xadJ\x93\xbah \xbe\x92\xfc\xfa\xeb\xa0X\xa7\x89\x81\xe9\xbeJ\x91\xfa8j\xa4\xb6\xc2\xb0\xed\xban)L\xd3\x96\xe9 2?&\xe9w\x11\x8e;\xeb]<FX*\xc5D{\xb4\xc1p\xaf\tE\xbd\x19\xa4) \x9d[\t6\xc1`\xfa=T\xeeo\xd3Q\xdet\xecs\x9c|e\xa5\xdc\xe7Qx\xb4S\xb9\x0fU\xfaS\xb3\x0c\xdf\xe0\x04\xf7\x89<"\xd2\xb8\xcc\x0b\xa5\xbb\xa0\xa21T\x88\x1d\x07h\x01\x15\xbb\xe7<\xee{F\xb3t\xed\x06\xd1\xe5\x9c\xb8\xba_\xb0uZ\xca\xd3\x91"\xc0mB-\xff\x18\xde\xda$\xe7\xc1LX*\xe0.\xf6\x91@\xfd\x8cCx\xed\x87\x0e\xa7{\xf5[g\x15<\xd8\xd9j\xfd\x967\xf0\x83\xe7Y#C2\xde\xb4\xe1\x06\x16\xf3\x94!\xf9:\x12S\xb0\xae\xdf-\x01U\x0e\xf8\xc8\xae\xae\xec\xeaT\xc4\xf3[\x11\x00l\x1e\x95\xaf\x00\x81\x7f\xda\xee\x08\xcdLw[\xe6TLe\xa2\xff\x08\r\x91(\xfcf\x9d\xc6<\xc29\xc6\x83\xb3tB\x95R\xf3\xe5\x89\xc4\x9d\x8e\x99\x1cj^\xa8\xc1\xe0x1=\x01\x14\xf1\x88\xcaOr\x05e\xb1\xc2\xd3\x0em\x7f\xc2\x15\xeb\x05\r\xc5\xa8\xff\xa9W~\xf1q\x1f\xe7\n\x1aB\xc1\xeb\xfd\x98xF=\xe9\xde\xdd\xd9\xc0\xdf\'\xd3\x03\x81\xbc\x97\xf4` \x03KQ\xaa\xe2\xc2\xb5^\xa3')
msgs.append(
    b's\x02\xee\xee(\xd7\'\xda\x80[\x10H{\x1e$z\x86\xee\xae\xfd\x97f\xef\xefa\xbeX\xb0\x0f\xde\xfa\x97\x1d\xe8$\x8c\x08\xe6\xf1m\x14\xd6#S\x19\xea\x1b\xd9\xee/\x15\xa1^\xc4t\x8c\x1d\xcca\x18Zq,\xb8\x0e\x90\xe1[\x96\x1b\xd1\x16\x87\xb7\x9f(\xf28\x83\x17x<\x16\xf8\x13T\xda\xa1\xec\xde"\xfe~\'\x1ap\xbb\x1c{\xf0\x88\xe8\xe1V\xf5\xfd\xc1\x8a|\xc4\x0e\xf3s\x17z\xa7\xbd\xd8w\x9f\xca\xce\xbc\xb3\x97w\xa0\xd0+\x0e9J&/~9\x9f\xdeT\x9d\xcf\x17\x99]\x177L\xf2\xadO\xd6\xf1M\xf7Wv\xcc\xdfK\tX\xaf\xe0X\xff\xa1\xb5y\xd9/4\xde8\xe4\x90}\x08Z\xa4\x0cAr\xa8\xd5\x82G#7\x85\x83V\xbd\x8f\xef]\\\xb6&6\xeer\xc8{h\xea<\x9e\xaaFc\xa6\xcdx\x84\xbfU\xa1l\x7f-n\xcf\x10P2H^\xba?\x97S\xd3LT\xfc\xb0\x05\xae[\xf9\x9d\x17\x0b\xa5\xe6E?2_fr\x14\xc22\xe2?I\x18\xbc\x92\x9c\xc1\xe8\x93i\xef\xb4\x95%U\x1d:9\xb5\xd5\xb4U\xd2\x1b\xb5\x89\x9e\xd2\xe8\x1d\x05\xaaa\x84\xc5\x12\xd5\x04\xf9\xb4\xbe\xe1g\xe1a\xc8\x96}\x89x\x00\xc2\x90u\x0b\xe4\x02\xc9\x0110\xf6\x08\xf0\x8bw3\xb1\x8ckU\x82&)"p")l\tZLci\xae\x08\xd0\x12\xe0\xe1#W^P8\xa0\xf4\x84\x06lbH\x07bX1\xf0\xcda1/\xc0?\xa2\xb0-\xe3\xfd\xed\x01\xb15Z\\\x97\xea\x1f\x8a\x9d\x08\xa8wO"\x95\xfdJ\xbb}EC\x0c\x99\xf1\xf3\xfa\xbe,1\xf2\xd8\x01Rf/\xa2\xf5x\x8e\x90\xc8B?\x15\xe7\x95\xe6\x1b?\x10\xc4\xdb\x01f\xf6t\x03\xc2\x1a\xb5\xf6CX\xfa\xc0A\xfe\xc47-\xe86\xeb\x9b\xe1\xb9\xa7\xf4\xf0\xcd\x91\x86\xf3\x1c\xb9\x9a\xde\x0b<\xce\x02\x12\x13\x8e\xddV\xa1_\xe4\xcc\xb9\t\xb6\x7f\x01\x83|\xf4\xc6\xb8uso\xa8\x05K\x08')
msgs.append(b"s\x02+\xef(\xd7'\xda\x80[\x11\x08N(pwa\x85\xab\x9a90y\xf6+3\x88C\xb8\x97^|\xba\xdb5\xa1\xba\x81\xb6$Z\x94\xd6\xdfE\r>8\xc7\xd2U:\x1b}z\xb9|$\xd7\x8bs>I\x01i\xe4\x8eL}\x06\xe8\x99\xff\xc0h\xd5\xee\xc6\xb7\xeaAb{\x13\x8dc\xa0N\x1c\x05\x8f\x08a?\xa9g9i(\t\xc5&")
msgs.append(
    b"s\x00}\xe9\x08\xd7'\xda\x80[\x11\x88\x97\xfa%m\xeb\r\x9evnH<\xe0\xca\xa9a@\xef\xb8\xd9E\xd4\xban\xd5\xd5\x9b\xce\xe8\x11\xf0&2\x83A\x0e\x8aR\x14)\x14\xe8R\xa8+\xa4\xe3\x05B+\xc5\xe3\xd6/\x13\x84\xb7!\x98\xaecu\xb2j!kq\x82\xb6\xf2I\xec\xfc\xca\x9b\xb1\x8e\xf8\xc6\xbeC`\xe0\xa8p`Qg\xa2\x01Q\xfb\xbb\x8d\xdf\xa6\x9e\xa2i1m\x15XD\xbd\xb7\x87\x0en\xa8\xaa|\t\ro\xe6\x18  \x8a\xafQ\xd1\xeb\x9erJ\x98\x07#\x11\xa1\xec\xdd\x10\x00#^\xb3WRz\x86\x9c`IL\xaa(\x1b\x9c;Z\xa4\xb4P\x9f\x0c\xe5\xb2\xa4\xb4\x9c\xce\xb2@\x9aO\xf7OM\xc4&>&-5\xf3:\xcf\xbbuA\x044\x98\xeb\x85\xd7\x95\xcb\x1di\x9ci]\xb3K\xbd\xc5\xfb\x9fLQ\x1a~\xe9^\xf5{\xf3\xb2\xa39'\x96!\xachE\xdd\xfa!\xea\xea)z\xd4p_\xed\xa8k\xf3L\x1b\xc8\x8fZ,\x91\x86\xaa\x18f\xd4U\x8cLD\x8a\xa7-\xaa\xbc")
msgs.append(b's\x00B\xec\x18\xd7)+p[\x1fH0\xb6\xb6`j\xef\xa5\x03\xb3\x1a\x1cd\xbc\x17\xa7\xf5\xb3\x95K\xfa)\xb2F\xc4\x9a:y\x867\xf5\x7fW\x96o\xbb|\x97\x81#\x8c\xef\x8ey\xd2\xfb\x89\xffF)1\xd3RA]\xe2\xe1\x17>\xf9\xfe\xf9\xa7XJ\xab[\xa6,\xb59\xd0W\xa1\x93\xa7H\x83Zb\xc3\x0b\xb5Y\xebtF5E:\xc6P\xd8\xccP\xfd\xfe:om\x9c\xf6p\x8a\xeb\x10]*/\xe7KU\xb76b\x97$\xf4\x8f^\xfb\xe8\xe4\xb2\xbdF\xf1\xbeH~\xa8\x9e\x1c\x81\xb1\x1d\xd7')
msgs.append(b's\x00Fg8\xd7\'\xda\x10[\x17HX\xfe\x1b|\xae\xf2\x18f\xd9\xaa\xfb\xaf0\xec\xf9\xabq\xcf*F\xed\x1e!\xd0^"(\xd3i<\xbbt\x1aU\xe0\xfez\xd4d\xfes\x00\x0eg#\x07\xed\x1f\xe2%\x9bb,B7&}\x98=F\xb4S\xb2\xccd\\\xe0Q\xd8Uc\xa29\x83K{\xaa%\xf4"\x00\x13U\xf6\xe3h\x10\xc2\xb7\xa0\t\x87\x9e\xce\x12\xf7\x1d}0d+4\x88\xaf;\x17\x95;\xd9\x8e\x94\xca\xf99\x898M,\xa2\xc9h\xd1\xa5\xf5spN\x1cc\x8aZx}\xa9X\x8c\xe4S\xe0\xe2f7\x97')
msgs.append(
    b"s\x00_\xeb(\xd7'\xda\x80[\x1fH\xa9\x81\xe5>2`~E\xa3O\xf7\x9b\x08R)Z\x96\xccI\xb0\xd0\xb4\xd4C\x8d\xcb\xcb\xadDx\x18\xcc\x94\xdbG\xce\x95\x8e\xef\xd9\xff\x1c\xef\xdd\xcf\xe8\xe7\x8fT\x80\xb5\x99\xc3\x10\x07=\x93\x82\xc9\xa4\x82\xbe)\xbc\xe6,\x16\x9e\xa5\xb4`N\x92\r2x1\x85\x82\x99M\xdd\x83\xec\xf3\xe8ov\x0c\xe5!\xcb\x85d\x8di\xe4Q\x1d\xcaE))\x871\x81\xc5\xbd\xbfc<\x0f\x82\xe0\x8b\x81\xd2\xc8T0\xfaz'\xdb\xf0\xc0\xa8\x8c\x13x\x93\x88$\x9c\xe7;\xfd\xa8\xe5t\xeb~Mc\x1aY\x80d\xcd_\xf1y\x94\xbd\xe5\xc6\xb6\xfd\x14i\xb6\x1e\xc0\x14\x96$]\xe1(\xe1\xb7v\x02$?\x9bd\xef\x8d\xc9\x12\xff\x99j3\x95U\xf5\x8e1p$\xa9\xd9")
msgs.append(b's\x00\x14h\x04\x12\xa8[\x11\xc8V\xf4^R\xac\xaf\xc7.Q\t\x82\x9c\xa3\xc2\x02H\xca\xb1J\xc4\x18\xc6A:\x90qZ-\x08\xee\xf8\xa8\x93\xc3\xfa\xcb\xdf\x9a\xa9\x0e\xaa\x9e\xf1')
msgs.append(b's\x00\x0e\xec\x10\x1d\x98[\x1f\x88r^\\\xff\xf6\x1a\xf8\x97\xf0)u"\xb1\x04\xe0\xb8\x1b\xbeu\x8d\xcao{\x01sUn\xc1\xc1b\x05I')
msgs.append(b"s\x00\x14h\x04\x12\xd0[\x12\x08&\x9b\nOi\x90\xe3\xccKp\x99\xc7w9\xd4\x8cY\xed'\xd2\xcb\x97`\xa4mT\xb5S\xbe&\xac-\x00u\x19\x1d\xcch\x96\x87\xcat\xcc")

msgs.append(b"s\x00\x0e\xec\x10\x1d\xc0[\x1f\xc8'e\x0f\xfc\xc4\x14z\x96\xc4h\x14\xe1\xec=\x05\x03\xc9\xa1h\xab\xc9\x99w\xbfN\x1fh\xee\xad\xb4\xf7\x1d")
msgs.append(b's\x00\x0c\xed4\x12`[\x17\xc8\x1c\\\x8d}\xf2\xafp=\xc9\x02\xf2J\xab\xa2\xd2Pc\x11\x00\r\xac\x16\xec\xd6\xd4j\x95?')
msgs.append(b"s\x00\x11c$\x12\xd0[\x1f\xc8\xd8O\x9e\x0f\x04X\xe7N7!\x9c\xa6g\x8c7\xbf[\t\x1c'\x99k\x954\xb8\xe8jR\x9dmBG\xd8\x07\x05\xd5K")

for msg in msgs:
    try:
        parsed = SPARTNReader.parse(msg, decode=True, key=key, basedate=datetime.datetime.utcfromtimestamp(1713519000))
        # print(parsed._decode)
        parsed_typed: SPARTNMessage = parsed
        print(f"{parsed_typed.identity}")
    except Exception as e:
        print(e)
semuadmin commented

Hi again @jonathanmuller

So - yes - if the decryption is incorrect the entire payload will be junk - there's no middle ground there!

Are you able to ascertain if this issue only affects SPARTN-1X-OCB-GLO messages with 16-bit timetags (timeTagtype = 0), or does it also affect other GNSS types (e.g. OCB-GPS or OCB-GAL) with 16-bit timetags?

Are you able to confirm whether SPARTN-1X-OCB-GLO messages with timeTagtype = 1 (32-bit timetag) are being decrypted correctly?

This is puzzling because, based on the SPARTN sources I've been using (Thingstream PointPerfect MQTT and NTRIP services), all SPARTN-1X-GAD (global area definition) messages have 16-bit timetags (timeTagtype = 0) but these are definitely being decrypted correctly provided I use the correct key and basedate. I can verify this against exemplary output - see for example the gad_plot_map.png illustration. If I deliberately provide an invalid basedate, the GAD output is - as expected - gibberish.

The fact that GAD messags with timeTagtype = 0 are decrypted correctly suggests that the basic IV (Initialisation Vector) and decryption algorithms in pyspartn are working as expected as long as the key and basedate values are correct.

Unfortunately I don't have access to exemplary output for OCB or HPAC messages so it's much more difficult to ascertain whether they are in fact being decoded correctly. There's no explicit encryption validation CRC or hash, for example (understandably, as this is good cryptographic practice). One avenue I can use is to compare them with contemporary output from the Thingstream PointPerfect NTRIP service, which outputs unencrypted SPARTN messages (eaf=0).

Few suggestions just to discount any obvious hiccups (I'm sure you've checked these but just to confirm):

  1. I'm assuming the key you're using is valid, but it might be worth double checking it hasn't expired.
  2. What are using as the basedate? If a datetime value, what value are you using and how have you obtained that?
  3. Have you tried using a 32-bit timetag from the same datastream in place of a datetime value as the basedate (pyspartn will accept either) - i.e. the gnssTimeTag value from a contemporaneous message with timeTagtype = 1 (maybe an HPAC message)?
  4. Have you tried using the spartn_mqtt_client.py or spartn_ntrip_client.py examples to obtain a 'real time' SPARTN data stream, by way of comparison?

In essence, can we discount the possibility that this is simply an issue of incorrect decryption parameters (key and basedate)?

jonathanmuller commented

I can confirm it's only SPARTN-1X-OCB-GLO messages.
In the same stream (60s of data), all HPAC, GAD and OCB messages are correctly decrypted, except the SPARTN-1X-OCB-GLO

  1. Key looks valid as all other messages are decrypted correctly
  2. I'm using basedate=datetime.datetime.utcfromtimestamp(1713519000))with this epoch being the time I started the mqtt client
  3. I'll try that, but as all the other messages of the stream are correctly parsed I'm relatively confident the base time is correct
  4. Yes, I'm using convert_timetag and copy/paste the messages for convenience

I'm relatively sure that's it's only a decryption problem for GLONASS ambiguity on 16 bits timestamp (for SPARTN-1X-OCB-GLO). I'll double check with HPAC-GLO decryption just to make sure.

Keeping you updated

  • 👍1
semuadmin commented

I'll review the latest SPARTN protocol specification again to see if there's anything I may have missed with regard to special treatment for GLO 16-bit decryption, but as I recall it's fairly vague around the whole timetag ambiguity area. The IV and timetag conversion algorithms used by pyspartn were arrived at after lengthy discussions on the u-blox community forum.

jonathanmuller commented

I just tried with a fresh stream : datetime.datetime(2024, 4, 22, 7, 35, 0)
SPARTN-1X-OCB-GLO decryption worked without trouble ( I only launched the test on few minutes of data)
That leaves 2 main solution :

  • It always worked and my date time was not accurate enough and was right at the limit of the 16 bits overflow
  • As GLONASS time is relying on Moscov timezone (UTC+3) and not UTC+0, there is a 3h period around midday (and midnight) where the parsing is incorrect, and I was (un)lucky enough to record data in that time

I'll try again with a stream recorded few minutes before and after this critical time switch that is midday UTC+0

  • 👍1
  • 👀1
jonathanmuller commented

That seems to be it : to get a correct parsing (or at least what seems to be valid) of SPARTN-1X-OCB-GLO I need to set a datetime 3h in the future to account for conversion between GPS UTC+0 and Moscow UTC+3.
Obviously it works most of the day without trouble unless I record a stream between 12h and 15h. Just for completeness I'll try after 15h to see if parsing works again because the shift is "resolved".

There also seem to be a problem with datetime counting leap seconds while GPS doesn't, but as it's only 18s shift it's really hard to reproduce.

Let me know how I can help you and if you would like me to try open a PR for those 16 bits resolution ambiguity time shift.

EDIT : confirmed, parsing after 15h (UTC+0) works

semuadmin commented

That seems to be it : to get a correct parsing (or at least what seems to be valid) of SPARTN-1X-OCB-GLO I need to set a datetime 3h in the future to account for conversion between GPS UTC+0 and Moscow UTC+3. Obviously it works most of the day without trouble unless I record a stream between 12h and 15h. Just for completeness I'll try after 15h to see if parsing works again because the shift is "resolved".

There also seem to be a problem with datetime counting leap seconds while GPS doesn't, but as it's only 18s shift it's really hard to reproduce.

Let me know how I can help you and if you would like me to try open a PR for those 16 bits resolution ambiguity time shift.

EDIT : confirmed, parsing after 15h (UTC+0) works

I can't fault your logic, but this feels incredibly counter-intuitive to me. Should a comparable UTC+n adjustment apply to, say, BEI or QZS constellations based on their respective base timezones?

Can I clarify a couple of points:

  1. What specific criteria are you using to confirm that the HPAC or OCB payloads are in fact being decrypted correctly? Do you have exemplary data to validate the output against?
  2. Have you tried using, as a basedate, a 32-bit timetag from an OCB-GLO message in the same datastream - i.e. use the 32-bit timetag from an OCB-GLO message with timeTagtype = 1 to decrypt one with a 16-bit timetag (timeTagtype = 0), without making any further adjustment based on UTC? If so, does it work?

I'm kind of in two minds about whether to implement a code fix for this UTC basedate adjustment, or simply add an advisory in the README. If we're to add an automated UTC adjustment, I'd like to see it tested against all possible GNSS constellation types (i.e. OCB-GPS, OCB-GAL, OCB-GLO, OCB-BEI and OCB-QZS messages with timeTagtype = 0). If we can establish it definitely works in all cases, very happy to consider a PR.

semuadmin commented

FYI As an aside, I've added a further fix in RC 0.3.1 to use the correct Area Count iteration value. I realised the attribute SF030 Area Count represents (area count - 1), not (area count) as previously handled. This will affect HPAC and GAD payloads but not OCB, and it has no direct bearing on the conversation above. I presume it was done this way so as to accommodate up to 32 iterations in a 5-bit field, but AFAICS it's not explicitly stated in the SPARTN specification.

  • 👍1
semuadmin commented

FYI As an aside, I've added a further fix in RC 0.3.1 to use the correct Area Count iteration value. I realised the attribute SF030 Area Count represents (area count - 1), not (area count) as previously handled.

As part of this change, I've added a protected attribute _padding to all SPARTNMessage objects. This contains the number of redundant bits added to the payload in order to byte-align the payload with the exact number of bytes specified in the transport layer payload length attribute nData i.e. (actual content length in bits + msg._padding = msg.nData * 8). If the payload has been successfully decrypted and decoded, this value should always be >=0, <=8, so checking the value provides a rough-and-ready (but by no means definitive) indication of successful decryption.

jonathanmuller commented 
1. What specific criteria are you using to confirm that the HPAC or OCB payloads are in fact being decrypted correctly? Do you have exemplary data to validate the output against?

2. Have you tried using, as a `basedate`, a 32-bit timetag from an OCB-GLO message in the same datastream - i.e. use the 32-bit timetag from an OCB-GLO message with `timeTagtype` = 1 to decrypt one with a 16-bit timetag (`timeTagtype` = 0), without making any further adjustment based on UTC? If so, does it work?
  1. I'm verifying the message is fully parsed (between 0 and 7 bits left and not trying to read above the message limit), as well as that all the Do Not Use (DNU) flags are set to 0, and that the Solution issue of update (SIOU) follow the other messages.
  2. That is a really clever idea and would have saved time if I though about it earlier, nice one ! Here are the timestamp in a continuous stream (few seconds of data) :

As you can see, GLO = GPS time + 3600*3 - 18
However, it also seems there is a 14 second difference between BeiDou and GPS ... : BeiDou = GPS + 14

I'll edit my message/re-comment when I have more details

Message Name 32 Bit Timestamp 16 Bit Timestamp
SPARTN-1X-GAD 5430
SPARTN-1X-GAD 5430
SPARTN-1X-HPAC-GPS 451488630
SPARTN-1X-HPAC-GPS 451488630
SPARTN-1X-HPAC-GPS 451488630
SPARTN-1X-HPAC-GPS 451488630
SPARTN-1X-HPAC-GPS 451488630
SPARTN-1X-HPAC-GLO 451499412
SPARTN-1X-HPAC-GLO 451499412
SPARTN-1X-HPAC-GLO 451499412
SPARTN-1X-HPAC-GLO 451499412
SPARTN-1X-HPAC-GLO 451499412
SPARTN-1X-HPAC-BEI 451488616
SPARTN-1X-HPAC-BEI 451488616
SPARTN-1X-HPAC-BEI 451488616
SPARTN-1X-HPAC-BEI 451488616
SPARTN-1X-HPAC-BEI 451488616
SPARTN-1X-HPAC-GAL 451488630
SPARTN-1X-HPAC-GAL 451488630
SPARTN-1X-HPAC-GAL 451488630
SPARTN-1X-HPAC-GAL 451488630
SPARTN-1X-HPAC-GAL 451488630
SPARTN-1X-OCB-GPS 451488630
SPARTN-1X-OCB-GLO 451499412
SPARTN-1X-OCB-BEI 451488616
SPARTN-1X-OCB-GAL 451488630
SPARTN-1X-OCB-GPS 5435
SPARTN-1X-OCB-GLO 16217
SPARTN-1X-OCB-BEI 5421
SPARTN-1X-OCB-GAL 5435
SPARTN-1X-OCB-GPS 5440
SPARTN-1X-OCB-GLO 16222
SPARTN-1X-OCB-BEI 5426
SPARTN-1X-OCB-GAL 5440
semuadmin commented

👍

1. I'm verifying the message is fully parsed (between 0 and 7 bits left and not trying to read above the message limit), as well as that all the Do Not Use (DNU) flags are set to 0, and that the Solution issue of update (SIOU) follow the other messages.

This is essentially what I've been doing as well in debug mode - hopefully the new _padding attribute makes this a bit easier.

semuadmin commented

As you can see, GLO = GPS time + 3600*3 - 18 However, it also seems there is a 14 second difference between BeiDou and GPS ... : BeiDou = GPS + 14

Yeah different GNSS constellations follow different leap second adjustment rules (I have an old private GitHub branch which was intended to address this for historical logs - I may dust it off sometime). It's not normally a major issue in practice but it's clearly significant in the context of using (reasonably) accurate timestamps to derive cryptographic initialisation vectors (IV). I understand that it's good practice to use 'unpredictable' values in IV derivations, but I really wish u-blox hadn't chosen basedate given all the potential confusion around UTC and leap second adjustments. I suspect their intention was for end users to always use a 32-bit timetag from the same datastream, but this is geared towards 'real time' embedded applications.

semuadmin commented

FYI see #19 - not sure this is a complete fix, but it should work OK provided there are sufficient unambiguous 32-bit timetags in the incoming datastream.

There is still value in applying an automatic UTC (and ideally leap second) shift to any 'basedate' input argument to cater for situations where there is no available 32-bit timetag in the incoming datastream. The only other option I can see is to have separate 'basedate' arguments for each msgSubtype (i.e. GNSS constellation), which feels awfully clunky.

If you have the time, give this branch a go and let me know how you get on. Feedback and corrections welcome.

jonathanmuller commented

I was thinking about doing it simpler (but your PR is obviously a great addition !)
Given a close enough date-time and an ambiguous 16 bits time :

  • Calculate the unambiguous 32 bits time -> this is T
  • Add 12 hours to T -> this is T + half a day
  • Subtract 12h to T -> this is T - half a day
  • Compare the given date-time to T, T+half, T-half, and look which one is the closest
  • The closest it the one to be returned

That way, as long as the provided date-time is +/- 6h accurate, the resolution should work, regardless of GNSS time difference (even the +3h of Glonass)

I'll try to open a PR as soon as I can find some time

semuadmin commented

That would also work.

I think collating 32-bit timetags from the data stream for use as basedates for 16-bit timetags is a solid policy and - I suspect - u-blox's intention for implementation. But the method for providing a 'backup' basedate for when there is no 32-bit timetag available (yet) is up for grabs. Happy to consider alternative approaches.

The basedate provided to SPARTNMessage for an individual message must obviously correspond to that specific message.

I think I'll merge the PR for RC 0.3.2 sooner rather than later as it includes a bunch of other enhancements I've had on the pile for a while now (iincluded automatic conversion of float values and a bunch of enumerations), but we can look to enhance further in RC 0.3.3.

  • 👍1
semuadmin commented

PR has automatically closed this but feel free to re-open, or raise another issue/PR, if you have a better approach.

I've opened a Discussion for v0.3.2-beta so feel free to contribute to that - https://github.com/semuconsulting/pyspartn/discussions/20