fly deploy nil pointer dereference: machine not found
Opened this issue · 0 comments
technillogue commented
Describe the bug
flyctl deploy segfaults, maybe related to removing one machine and adding a volume
- Operating system: ubuntu
fly version
** Paste your fly.toml
app = 'llama-3-8b-trt'
primary_region = 'ord'
[build]
image = 'r8.im/technillogue/llama-89@sha256:89d39913391a900f1c42cc9db5e9bd65dfb34975dc9e732c9b126a1e60eae44e'
entrypoint = "bash"
[http_service]
internal_port = 5000
force_https = true
auto_stop_machines = false
auto_start_machines = true
min_machines_running = 0
processes = ['app']
[[vm]]
size = 'l40s'
[mounts]
source="models"
destination="/weights"
** Command output: **
fly deploy --local-only
==> Verifying app config
Validating /home/sylv/r8/tmp/fly-mistral/fly.toml
✓ Configuration is valid
--> Verified app config
==> Building image
Searching for image 'r8.im/technillogue/llama-89@sha256:89d39913391a900f1c42cc9db5e9bd65dfb34975dc9e732c9b126a1e60eae44e' locally...
Searching for image 'r8.im/technillogue/llama-89@sha256:89d39913391a900f1c42cc9db5e9bd65dfb34975dc9e732c9b126a1e60eae44e' remotely...
image found: img_8rlxp2nw7o97p3jq
Watch your deployment at https://fly.io/apps/llama-3-8b-trt/monitoring
-------
Updating existing machines in 'llama-3-8b-trt' with rolling strategy
Failed to update machines: failed to update machine 148e54e9a34448: failed to launch VM: insufficient resources to create new machine with existing volume 'vol_4yj1lm1k9n66ow14' Retrying...
WARN error refreshing lease for machine 148e54e9a34448: failed to get lease on VM 148e54e9a34448: machine not found
-------
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x20e9a9e]
✖ failed to launch VM: insufficient resources to create new machine with existing volume 'vol_4yj1lm1k9n66ow14'
goroutine 87 [running]:
github.com/superfly/flyctl/internal/command/deploy.(*machineDeployment).updateProcessGroup.func1()
/home/runner/work/flyctl/flyctl/internal/command/deploy/plan.go:339 +0x1e
golang.org/x/sync/errgroup.(*Group).Go.func1()
/home/runner/go/pkg/mod/golang.org/x/sync@v0.8.0/errgroup/errgroup.go:78 +0x56
created by golang.org/x/sync/errgroup.(*Group).Go in goroutine 86
/home/runner/go/pkg/mod/golang.org/x/sync@v0.8.0/errgroup/errgroup.go:75 +0x96