"docker live stop" command failing due to unescaped '$' character
Closed this issue · 1 comments
In my local algorithm deployments I frequently experienced an issue where the command "docker live stop" did not stop my live deployments and resulting in errors like this:
Context information:
Python version: 3.10.12 (main, Jun 11 2023, 05:26:28) [GCC 11.4.0]
OS: Linux-6.2.0-33-generic-x86_64-with-glibc2.35
Lean CLI version: 1.0.169
live.send_command(): stop - sending command.
live.get_result(): stop - waiting for results...
Traceback (most recent call last):
File "/home/constantine/.local/lib/python3.10/site-packages/lean/main.py", line 94, in main
lean.main(standalone_mode=False)
File "/home/constantine/.local/lib/python3.10/site-packages/click/core.py", line 1055, in main
rv = self.invoke(ctx)
File "/home/constantine/.local/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/constantine/.local/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/constantine/.local/lib/python3.10/site-packages/lean/click.py", line 159, in invoke
result = super().invoke(ctx)
File "/home/constantine/.local/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/constantine/.local/lib/python3.10/site-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "/home/constantine/.local/lib/python3.10/site-packages/lean/commands/live/stop.py", line 37, in stop
get_result(command_id, docker_container_name)
File "/home/constantine/.local/lib/python3.10/site-packages/lean/commands/live/live.py", line 88, in get_result
raise Exception((f"live.get_result(): {stack()[1].function} - "
Exception: live.get_result(): stop - Failed: to execute the command successfully. Failed to read result from docker file
result-command-6b31a490ac444ef88713ffbb6d082ee0.json within 30 seconds
Error: live.get_result(): stop - Failed: to execute the command successfully. Failed to read result from docker file
result-command-6b31a490ac444ef88713ffbb6d082ee0.json within 30 seconds
I have determined that in the file lean/commands/live/stop.py the issue lies upon the following variable "data":
data = {
"$type": "QuantConnect.Commands.AlgorithmStatusCommand, QuantConnect.Common",
"Id": command_id,
"Status": "Stopped"
}
where the "$" character in the json member "$type" is not being escaped when it is passed to the docker exec command in function write_to_file() in lean-cli/lean/components/docker/docker_manager.py and thus fails the write the stop command to the docker container.
I found a workaround by adding a line after line 450 in the write_to_file() function of docker_manager.py to the following:
data = data.replace('"','\\"')
data = data.replace('$','\\$')
command = f'docker exec {docker_container_name} bash -c "echo \'{data}\' > {docker_file.as_posix()}"'
And after adding that line the error no longer occurs
Hey @Grenite!
We believe this was due to a delay on lean engine shutting down, we've increased the command timeout because it was tight and improve the error log. If it happens again please run docker logs {container name}
so we can look closer at what is lean doing, which is causing the delay. To get the docker container name you can run docker ps
. Release in lean cli version 1.0.171
Thank you, closing the issue for now 👍