3.1.8 zombie crash automation doesn't work and deployed flows don't generate heartbeats
Opened this issue · 1 comments
Bug summary
I followed the instruction to enable automation for crashing zombie flows.
When I tried it on flow that was deployed using .serve()
, it generated heartbeats, and when I tried deploying same flow using prefect --no-prompt deploy --prefect-file prefect.yaml -n Zombie
, it didn't
But even with flow that was generating heartbeats successfully, automation from the same instruction didn't work. My procedure was:
- Serve the flow
- Run it
- Wait for the first heartbeat to happen
- Stop serving process.
Result: flow continues to run as if automation wasn't there.
Setup description:
Flow code:
import time
from prefect import flow, task, get_run_logger
@task
def task_a():
logger = get_run_logger()
logger.info("Hello")
time.sleep(5)
@flow
def flow_a():
while True:
task_a.submit().result()
if __name__ == "__main__":
flow_a.serve()
prefect.yaml
:
name: Zombie
work-queue-default: &default
work_pool:
name: work_pool_1
work_queue_name: default
job_variables:
stream_output: true
deployments:
- name: Zombie
entrypoint: zombie.py:flow_a
<<: *default
prefect config view
:
🚀 you are connected to:
http://127.0.0.1:4200
PREFECT_PROFILE='default'
PREFECT_API_KEY='********' (from profile)
PREFECT_API_URL='http://127.0.0.1:4200/api' (from profile)
PREFECT_LOCAL_STORAGE_PATH='C:\Users\ihor.ramskyi\PycharmProjects\Prefect_Tutorial\local_storage' (from profile)
PREFECT_RUNNER_HEARTBEAT_FREQUENCY='30' (from profile)
Worker was started using command prefect worker start -p work_pool_1 -t process -q default
Flow was deployed using command prefect --no-prompt deploy --prefect-file prefect.yaml -n Zombie
Version info
Version: 3.1.8
API version: 0.8.4
Python version: 3.10.11
Git commit: 53a83ebc
Built: Tue, Dec 17, 2024 10:20 AM
OS/Arch: win32/AMD64
Profile: default
Server type: server
Pydantic version: 2.7.1
Additional context
No response
Thanks for the issue @ihor-ramskyi-globallogic! My hunch is that the heartbeat setting is getting set in your execution environment. I think the easiest way to do that is to add a prefect.toml
file in the root of your project that looks like this:
runner.heartbeat_frequency = 30
Once that file is created, you can redeploy your flow, and you should see heartbeats coming from flow runs for that deployment.
Let me know if you run into any more issues with this! If creating a prefect.toml
works for you, we should update the docs to include that step.