Connecting with FreeTAKServer over TLS
PeterQFR opened this issue · 11 comments
I have exported the .p12 files generated from FTS to .pem format and applying these to the config. I want to check that this is only for the client.p12 and not the server.p12 in the zip file?
When I do so, and disable certificate verification I still get an error on the socket.
2022-09-24 23:56:30,373 pytak INFO pytak.classes.run:281 - Complete: <Task finished name='Task-2' coro=<RclpyTakReceiver.run() done, defined at /home/peter-laptop/spurt/install/lib/python3.8/site-packages/spurt_tak_bridge/tak_bridge.py:59> exception=IncompleteReadError('0 bytes read on a total of undefined expected bytes')>
ERROR:asyncio:Task exception was never retrieved
future: <Task finished name='Task-2' coro=<RclpyTakReceiver.run() done, defined at /home/peter-laptop/spurt/install/lib/python3.8/site-packages/spurt_tak_bridge/tak_bridge.py:59> exception=IncompleteReadError('0 bytes read on a total of undefined expected bytes')>
Traceback (most recent call last):
File "/home/peter-laptop/spurt/install/lib/python3.8/site-packages/spurt_tak_bridge/tak_bridge.py", line 64, in run
cot = await self.readcot() #reader.readuntil(separator=b'</event>')
File "/usr/local/lib/python3.8/dist-packages/pytak/classes.py", line 163, in readcot
return await self.reader.readuntil("</event>".encode("UTF-8"))
File "/usr/lib/python3.8/asyncio/streams.py", line 629, in readuntil
raise exceptions.IncompleteReadError(chunk, None)
asyncio.exceptions.IncompleteReadError: 0 bytes read on a total of undefined expected bytes
2022-09-24 23:56:30,384 pytak DEBUG pytak.classes.handle_data:130 - TX: b'<event version="2.0" type="t-x-d-d" uid="takPong" how="m-g" time="2022-09-24T13:56:30.373072Z" start="2022-09-24T13:56:30.373095Z" stale="2022-09-24T14:56:30.373101Z" />'
ERROR:asyncio:Fatal error on SSL transport
protocol: <asyncio.sslproto.SSLProtocol object at 0x7fb13451a520>
transport: <_SelectorSocketTransport closing fd=14>
Traceback (most recent call last):
File "/usr/lib/python3.8/asyncio/selector_events.py", line 910, in write
n = self._sock.send(data)
OSError: [Errno 9] Bad file descriptor
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.8/asyncio/sslproto.py", line 685, in _process_write_backlog
self._transport.write(chunk)
File "/usr/lib/python3.8/asyncio/selector_events.py", line 916, in write
self._fatal_error(exc, 'Fatal write error on socket transport')
File "/usr/lib/python3.8/asyncio/selector_events.py", line 711, in _fatal_error
self._force_close(exc)
File "/usr/lib/python3.8/asyncio/selector_events.py", line 723, in _force_close
self._loop.call_soon(self._call_connection_lost, exc)
File "/usr/lib/python3.8/asyncio/base_events.py", line 719, in call_soon
self._check_closed()
File "/usr/lib/python3.8/asyncio/base_events.py", line 508, in _check_closed
raise RuntimeError('Event loop is closed``')
RuntimeError: Event loop is closed
This code works with TAKy on a TCP connection. I assume that both TAKy and FTS do not support a mixture of TLS and TCP connections. There is a passphrase associated with the user not sure if this is an issue. I didn't have a problem when connecting iTAK to it.
self.config = ConfigParser()['DEFAULT']
self.config["COT_URL"] = 'tls://127.0.0.1:8089'
self.config["PYTAK_TLS_CLIENT_CERT"] = '/home/peter-laptop/Downloads/spurt02keys/newfile.crt.pem'
self.config["PYTAK_TLS_CLIENT_KEY"] = '/home/peter-laptop/Downloads/spurt02keys/newfile.key.pem'
self.config["PYTAK_TLS_DONT_VERIFY"] = 'yes'
I assume you mean by 'file' the .pem files. I've attached them here.
Also will add the output from FTS, the zip mission package which includes both the user key and what seems to be a server key:
Here is the FTS server side. I think there are some messages left over in stdout from an itak client. But this should show the connection and an error
86
handling conn data
client connected over ssl ('127.0.0.1', 35116) 1664055765.3420637
data received b'<event version="2.0" uid="67FC57AE-9D87-476E-865A-11F685848934" type="a-f-G-E-V-C" how="m-g" time="2022-09-24T21:42:45Z" start="2022-09-24T21:42:45Z" stale="2022-09-24T21:44:45Z"><point lat="-27.522451836921938" lon="152.82239594658412" hae="64.77165603637695" ce="9999999.0" le="9999999.0" /><detail><contact callsign="Jada45" phone="" endpoint="*:-1:stcp" /><__group name="Green" role="Team Lead" /><precisionlocation geopointsrc="GPS" altsrc="???" /><status battery="61" /><takv device="iPhone" platform="iTAK" os="11" version="2.3.1.590" /><track speed="-1.0" course="322.61480712890625" /><uid Droid="Jada45" /></detail></event>'
serialized CoT b'<event version="2.0" uid="67FC57AE-9D87-476E-865A-11F685848934" type="a-f-G-E-V-C" how="m-g" time="2022-09-24T21:42:45Z" start="2022-09-24T21:42:45Z" stale="2022-09-24T21:44:45Z"><point lat="-27.522451836921938" lon="152.82239594658412" hae="64.77165603637695" ce="9999999.0" le="9999999.0" /><detail><contact callsign="Jada45" phone="" endpoint="*:-1:stcp" /><__group name="Green" role="Team Lead" /><precisionlocation geopointsrc="GPS" altsrc="???" /><status battery="61" /><takv device="iPhone" platform="iTAK" os="11" version="2.3.1.590" /><track speed="-1.0" course="322.61480712890625" /><uid Droid="Jada45" /></detail></event>' to type <FreeTAKServer.controllers.SpecificCoTControllers.SendUserUpdateController.SendUserUpdateController object at 0x7ff9c15e9f40>
client update has been sent through queue <FreeTAKServer.model.SpecificCoT.SendUserUpdate.SendUserUpdate object at 0x7ff9c046a070>
cot serialized 67FC57AE-9D87-476E-865A-11F685848934
data received b'<event version="2.0" uid="67FC57AE-9D87-476E-865A-11F685848934" type="a-f-G-E-V-C" how="m-g" time="2022-09-24T21:42:47Z" start="2022-09-24T21:42:47Z" stale="2022-09-24T21:44:47Z"><point lat="-27.522451836921938" lon="152.82239594658412" hae="64.77165603637695" ce="9999999.0" le="9999999.0" /><detail><contact callsign="Jada45" phone="" endpoint="*:-1:stcp" /><__group name="Green" role="Team Lead" /><precisionlocation geopointsrc="GPS" altsrc="???" /><status battery="61" /><takv device="iPhone" platform="iTAK" os="11" version="2.3.1.590" /><track speed="-1.0" course="322.61480712890625" /><uid Droid="Jada45" /></detail></event>'
serialized CoT b'<event version="2.0" uid="67FC57AE-9D87-476E-865A-11F685848934" type="a-f-G-E-V-C" how="m-g" time="2022-09-24T21:42:47Z" start="2022-09-24T21:42:47Z" stale="2022-09-24T21:44:47Z"><point lat="-27.522451836921938" lon="152.82239594658412" hae="64.77165603637695" ce="9999999.0" le="9999999.0" /><detail><contact callsign="Jada45" phone="" endpoint="*:-1:stcp" /><__group name="Green" role="Team Lead" /><precisionlocation geopointsrc="GPS" altsrc="???" /><status battery="61" /><takv device="iPhone" platform="iTAK" os="11" version="2.3.1.590" /><track speed="-1.0" course="322.61480712890625" /><uid Droid="Jada45" /></detail></event>' to type <FreeTAKServer.controllers.SpecificCoTControllers.SendUserUpdateController.SendUserUpdateController object at 0x7ff9c15e9f40>
client update has been sent through queue <FreeTAKServer.model.SpecificCoT.SendUserUpdate.SendUserUpdate object at 0x7ff9c046a460>
cot serialized 67FC57AE-9D87-476E-865A-11F685848934
client connected
client accepted
87
handling conn data
client has connected
client conn A
adding client
client conn B
client addition has been sent through queue <FreeTAKServer.model.ClientInformation.ClientInformation object at 0x7ff9c046a310>
client added
<FreeTAKServer.model.FTSModel.Chatgrp.Chatgrp object at 0x7ff9c046a0d0>
serialized CoT <event><point/><detail><remarks>Welcome to FreeTAKServer FreeTAKServer-1.9.9.6 Public. The Parrot is not dead. It’s just resting</remarks><marti><dest/></marti></detail></event> to type <FreeTAKServer.controllers.SpecificCoTControllers.SendGeoChatController.SendGeoChatController object at 0x7ff9c15e9f70>
there has been an error in a clients connectioncan only concatenate str (not "NoneType") to str
there has been an error in the reception of data in the monitoring of pipes B'RawConnectionInformation' object has no attribute 'clientInformation'
<event version="2.0" type="t-x-d-d" uid="pytak@peterlaptop-P65-67HSHP" how="m-g" time="2022-09-24T21:42:45.342603Z" start="2022-09-24T21:42:45.342707Z" stale="2022-09-24T22:42:45.342731Z"/> sent to client 67FC57AE-9D87-476E-865A-11F685848934
<event version="2.0" type="t-x-d-d" uid="pytak@peterlaptop-P65-67HSHP" how="m-g" time="2022-09-24T21:42:45.342603Z" start="2022-09-24T21:42:45.342707Z" stale="2022-09-24T22:42:45.342731Z"/> sent to client pytak@peterlaptop-P65-67HSHP
I'm doing the minimal inheritance on the RXWorker:
class RclpyTakReceiver(pytak.RXWorker):
"""
receive CoT from TAK and publish to ROS
"""
def __init__(self, queue: asyncio.Queue, config: dict, reader: asyncio.Protocol) -> None:
super().__init__(queue, config, reader)
async def run(self):
while True:
cot = await self.readcot() #reader.readuntil(separator=b'</event>')
print('COT RX {}'.format(cot.decode()))
class RclpyTakBridge(Node):
"""
Proxy CoT messages (xml strings) between ROS and TAK agents.
"""
def __init__(self):
super().__init__("rostak_bridge")
self.config = ConfigParser()['DEFAULT']
self.config["COT_URL"] = 'tls://127.0.0.1:8089'
self.config["PYTAK_TLS_CLIENT_CERT"] = '/home/peter-laptop/Downloads/spurt02keys/newfile.crt.pem'
self.config["PYTAK_TLS_CLIENT_KEY"] = '/home/peter-laptop/Downloads/spurt02keys/newfile-nopass.key.pem'
self.config["PYTAK_TLS_DONT_VERIFY"] = 'yes'
async def run(self):
# connect to tak server
try:
rx_proto, tx_proto = await pytak.protocol_factory(self.config)
except:
raise ValueError('TAK url must be valid')
clitool=pytak.CLITool(self.config)
await clitool.setup()
#clitool.add_tasks(set([RclpyCotWorker(clitool.tx_queue, self.config)]))
clitool.add_tasks(set([RclpyTakReceiver(clitool.rx_queue, self.config, rx_proto)]))
await clitool.run()
The only question here is that I am supplying the RXWorker protocol reader rx_proto
from the pytak protocol factory, is that going to be the same as produced from the CLITool?
I added the self.queue.put_nowait(data)
in the loop in my Run method of my RXWorker similar to the RXWorker in the CLI tool, the error message from FTS is a little different.
handling conn data
client has connected
client conn A
adding client
client conn B
client addition has been sent through queue <FreeTAKServer.model.ClientInformation.ClientInformation object at 0x7f4fe4361c10>
client added
<FreeTAKServer.model.FTSModel.Chatgrp.Chatgrp object at 0x7f4fe4361070>
serialized CoT <event><point/><detail><remarks>Welcome to FreeTAKServer FreeTAKServer-1.9.9.6 Public. The Parrot is not dead. It’s just resting</remarks><marti><dest/></marti></detail></event> to type <FreeTAKServer.controllers.SpecificCoTControllers.SendGeoChatController.SendGeoChatController object at 0x7f4fe4353c70>
there has been an error in a clients connectioncan only concatenate str (not "NoneType") to str
there has been an error in the reception of data in the monitoring of pipes B'RawConnectionInformation' object has no attribute 'clientInformation'
<event version="2.0" type="t-x-d-d" uid="pytak@peterlaptop-P65-67HSHP" how="m-g" time="2022-09-25T21:25:39.270355Z" start="2022-09-25T21:25:39.270438Z" stale="2022-09-25T22:25:39.270462Z"/> sent to client 67FC57AE-9D87-476E-865A-11F685848934
<event version="2.0" type="t-x-d-d" uid="pytak@peterlaptop-P65-67HSHP" how="m-g" time="2022-09-25T21:25:39.270355Z" start="2022-09-25T21:25:39.270438Z" stale="2022-09-25T22:25:39.270462Z"/> sent to client pytak@peterlaptop-P65-67HSHP
data received b'<event version="2.0" uid="67FC57AE-9D87-476E-865A-11F685848934" type="a-f-G-E-V-C" how="m-g" time="2022-09-25T21:25:45Z" start="2022-09-25T21:25:45Z" stale="2022-09-25T21:27:45Z"><point lat="-27.52255351749723" lon="152.82244389890954" hae="69.44781755004078" ce="9999999.0" le="9999999.0" /><detail><contact callsign="Jada45" phone="" endpoint="*:-1:stcp" /><__group name="Green" role="Team Lead" /><precisionlocation geopointsrc="GPS" altsrc="???" /><status battery="62" /><takv device="iPhone" platform="iTAK" os="11" version="2.3.1.590" /><track speed="0.0" course="239.72006225585938" /><uid Droid="Jada45" /></detail></event>'
serialized CoT b'<event version="2.0" uid="67FC57AE-9D87-476E-865A-11F685848934" type="a-f-G-E-V-C" how="m-g" time="2022-09-25T21:25:45Z" start="2022-09-25T21:25:45Z" stale="2022-09-25T21:27:45Z"><point lat="-27.52255351749723" lon="152.82244389890954" hae="69.44781755004078" ce="9999999.0" le="9999999.0" /><detail><contact callsign="Jada45" phone="" endpoint="*:-1:stcp" /><__group name="Green" role="Team Lead" /><precisionlocation geopointsrc="GPS" altsrc="???" /><status battery="62" /><takv device="iPhone" platform="iTAK" os="11" version="2.3.1.590" /><track speed="0.0" course="239.72006225585938" /><uid Droid="Jada45" /></detail></event>' to type <FreeTAKServer.controllers.SpecificCoTControllers.SendUserUpdateController.SendUserUpdateController object at 0x7f4fe4361970>
client update has been sent through queue <FreeTAKServer.model.SpecificCoT.SendUserUpdate.SendUserUpdate object at 0x7f4fe4371b20>
cot serialized 67FC57AE-9D87-476E-865A-11F685848934
error in sending of data [Errno 32] Broken pipe
issue sending data to client now disconnecting
disconnecting client
client pytak@peterlaptop-P65-67HSHP disconnected File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/usr/local/lib/python3.8/dist-packages/FreeTAKServer/controllers/services/FTS.py", line 899, in <module>
FTS().startup(args.CoTPort, args.CoTIP, args.DataPackagePort, args.DataPackageIP, args.SSLDataPackagePort,
File "/usr/local/lib/python3.8/dist-packages/FreeTAKServer/controllers/services/FTS.py", line 749, in startup
self.start_all(StartupObject)
File "/usr/local/lib/python3.8/dist-packages/FreeTAKServer/controllers/services/FTS.py", line 461, in start_all
self.start_ssl_cot_service(FTSServiceStartupConfigObject)
File "/usr/local/lib/python3.8/dist-packages/FreeTAKServer/controllers/services/FTS.py", line 282, in start_ssl_cot_service
self.SSLCoTService.start()
File "/usr/lib/python3.8/multiprocessing/process.py", line 121, in start
self._popen = self._Popen(self)
File "/usr/lib/python3.8/multiprocessing/context.py", line 224, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
File "/usr/lib/python3.8/multiprocessing/context.py", line 277, in _Popen
return Popen(process_obj)
File "/usr/lib/python3.8/multiprocessing/popen_fork.py", line 19, in __init__
self._launch(process_obj)
File "/usr/lib/python3.8/multiprocessing/popen_fork.py", line 75, in _launch
code = process_obj._bootstrap(parent_sentinel=child_r)
File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
self.run()
File "/usr/lib/python3.8/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/lib/python3.8/dist-packages/FreeTAKServer/controllers/services/SSLCoTServiceController.py", line 32, in start
self.mainRunFunction(clientData, receiveConnection, sock, pool, Event, clientDataPipe,
File "/usr/local/lib/python3.8/dist-packages/FreeTAKServer/controllers/services/Orchestrator.py", line 636, in mainRunFunction
CoTOutput = self.handel_regular_data(clientDataOutput)
File "/usr/local/lib/python3.8/dist-packages/FreeTAKServer/controllers/services/Orchestrator.py", line 745, in handel_regular_data
self.clientDisconnected(output[1])
File "/usr/local/lib/python3.8/dist-packages/FreeTAKServer/controllers/services/Orchestrator.py", line 454, in clientDisconnected
traceback.format_stack()))
client removal has been sent through queue <FreeTAKServer.model.User.User object at 0x7f4fe43901c0>
stage 1 c
disconnecting client
removing client pytak@peterlaptop-P65-67HSHP
client pytak@peterlaptop-P65-67HSHP deleted from user_dict
user_dict is now {'67FC57AE-9D87-476E-865A-11F685848934': <FreeTAKServer.model.User.User object at 0x7f4fe6c3a5b0>}
error shutting socket down in client disconnection [Errno 107] Transport endpoint is not connected File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/usr/local/lib/python3.8/dist-packages/FreeTAKServer/controllers/services/FTS.py", line 899, in <module>
FTS().startup(args.CoTPort, args.CoTIP, args.DataPackagePort, args.DataPackageIP, args.SSLDataPackagePort,
File "/usr/local/lib/python3.8/dist-packages/FreeTAKServer/controllers/services/FTS.py", line 749, in startup
self.start_all(StartupObject)
File "/usr/local/lib/python3.8/dist-packages/FreeTAKServer/controllers/services/FTS.py", line 461, in start_all
self.start_ssl_cot_service(FTSServiceStartupConfigObject)
File "/usr/local/lib/python3.8/dist-packages/FreeTAKServer/controllers/services/FTS.py", line 282, in start_ssl_cot_service
self.SSLCoTService.start()
File "/usr/lib/python3.8/multiprocessing/process.py", line 121, in start
self._popen = self._Popen(self)
File "/usr/lib/python3.8/multiprocessing/context.py", line 224, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
File "/usr/lib/python3.8/multiprocessing/context.py", line 277, in _Popen
return Popen(process_obj)
File "/usr/lib/python3.8/multiprocessing/popen_fork.py", line 19, in __init__
self._launch(process_obj)
File "/usr/lib/python3.8/multiprocessing/popen_fork.py", line 75, in _launch
code = process_obj._bootstrap(parent_sentinel=child_r)
File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
self.run()
File "/usr/lib/python3.8/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/lib/python3.8/dist-packages/FreeTAKServer/controllers/services/SSLCoTServiceController.py", line 32, in start
self.mainRunFunction(clientData, receiveConnection, sock, pool, Event, clientDataPipe,
File "/usr/local/lib/python3.8/dist-packages/FreeTAKServer/controllers/services/Orchestrator.py", line 636, in mainRunFunction
CoTOutput = self.handel_regular_data(clientDataOutput)
File "/usr/local/lib/python3.8/dist-packages/FreeTAKServer/controllers/services/Orchestrator.py", line 745, in handel_regular_data
self.clientDisconnected(output[1])
File "/usr/local/lib/python3.8/dist-packages/FreeTAKServer/controllers/services/Orchestrator.py", line 473, in clientDisconnected
self.disconnect_socket(sock)
Running two RX Workers doesn't work. But just having the original one in the CLItools does. I reformatted my TakReciver as a queueworker. as below:
class RclpyTakReceiver(pytak.QueueWorker):
"""
receive CoT from TAK and publish to ROS
"""
def __init__(self, queue: asyncio.Queue, config: dict) -> None:
super().__init__(queue, config)
async def run(self):
while True:
cot = await self.queue.get()#readcot() #reader.readuntil(separator=b'</event>')
print('COT RX {}'.format(cot.decode()))
It was simple issue, by having two RXWorkers may have been causing a conflict on the port. I think that QueueWorker deserves an example as a COT reciever for those not familiar with working with asyncio.Queues, as only a transmitter is shown in the README. Also potentially a QueueWorker.get() method might be worthwhile to abstact the functionality of queues.
I think I did get the TLS Connection. I did use the config FTS_COMPAT = true and PYTAK_SLEEP=1 in addition to what I had above. I haven't connected it in a while, so Ill check
@ampledata Hey I'm trying to go back through generating the keys, however I didn't document my workflow,
when I'm converting from the p12 file provided by FreeTAKServer to cert and key pem files, it doesn't accept my password. I get a Mac verify error, any thoughts? I'm passing for example:
openssl pkcs12 -in spurt02b08c594c-c19f-4bb0-bd4c-93954524c7d3.p12 -out newfile.crt.pem -clcerts -nokeys
but I get a Mac verify error: invalid password?
Actually worked out that the password is not the password supplied to the user in FreeTAKServer. in the zip file there is a fts.pref file which lists the password as a default "supersecret" its a bit garbage.
All said I have got a connection to Free TAK Server