superfly/flyctl

fly deploy nil pointer dereference: machine not found

Opened this issue · 0 comments

Describe the bug
flyctl deploy segfaults, maybe related to removing one machine and adding a volume

  • Operating system: ubuntu
  • fly version

** Paste your fly.toml

app = 'llama-3-8b-trt'
primary_region = 'ord'
[build]
  image = 'r8.im/technillogue/llama-89@sha256:89d39913391a900f1c42cc9db5e9bd65dfb34975dc9e732c9b126a1e60eae44e'
  entrypoint = "bash"
[http_service]
  internal_port = 5000
  force_https = true
  auto_stop_machines = false
  auto_start_machines = true
  min_machines_running = 0
  processes = ['app']
[[vm]]
  size = 'l40s'
[mounts]
  source="models"
  destination="/weights"

** Command output: **

fly deploy --local-only
==> Verifying app config
Validating /home/sylv/r8/tmp/fly-mistral/fly.toml
✓ Configuration is valid
--> Verified app config
==> Building image
Searching for image 'r8.im/technillogue/llama-89@sha256:89d39913391a900f1c42cc9db5e9bd65dfb34975dc9e732c9b126a1e60eae44e' locally...
Searching for image 'r8.im/technillogue/llama-89@sha256:89d39913391a900f1c42cc9db5e9bd65dfb34975dc9e732c9b126a1e60eae44e' remotely...
image found: img_8rlxp2nw7o97p3jq

Watch your deployment at https://fly.io/apps/llama-3-8b-trt/monitoring

-------
Updating existing machines in 'llama-3-8b-trt' with rolling strategy
Failed to update machines: failed to update machine 148e54e9a34448: failed to launch VM: insufficient resources to create new machine with existing volume 'vol_4yj1lm1k9n66ow14' Retrying...
WARN error refreshing lease for machine 148e54e9a34448: failed to get lease on VM 148e54e9a34448: machine not found
-------
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x20e9a9e]
 ✖ failed to launch VM: insufficient resources to create new machine with existing volume 'vol_4yj1lm1k9n66ow14'
goroutine 87 [running]:
github.com/superfly/flyctl/internal/command/deploy.(*machineDeployment).updateProcessGroup.func1()
        /home/runner/work/flyctl/flyctl/internal/command/deploy/plan.go:339 +0x1e
golang.org/x/sync/errgroup.(*Group).Go.func1()
        /home/runner/go/pkg/mod/golang.org/x/sync@v0.8.0/errgroup/errgroup.go:78 +0x56
created by golang.org/x/sync/errgroup.(*Group).Go in goroutine 86
        /home/runner/go/pkg/mod/golang.org/x/sync@v0.8.0/errgroup/errgroup.go:75 +0x96