Pointers to troubleshoot freeze during rpi3 upload task?

[samuelventura]

Environment

• Elixir version (elixir -v):
• Nerves environment: (mix nerves.env --info)
• Additional information about your host, target hardware or environment that
  may help

samuel@build:~/src/athasha/terminal$ elixir -v
Erlang/OTP 25 [erts-13.0.2] [source] [64-bit] [smp:32:32] [ds:32:32:10] [async-threads:1] [jit:ns]

Elixir 1.13.4 (compiled with Erlang/OTP 25)
samuel@build:~/src/athasha/terminal$ MIX_TARGET=rpi3 mix nerves.env --info
==> nerves
==> athasha_terminal
|nerves_bootstrap| Environment Package List

  Pkg:         nerves_toolchain_ctng
  Vsn:         1.9.0
  Type:        toolchain_platform
  BuildRunner: {nil, []}

  Pkg:         nerves_toolchain_armv7_nerves_linux_gnueabihf
  Vsn:         1.6.0
  Type:        toolchain
  BuildRunner: {Nerves.Artifact.BuildRunners.Local, []}

  Pkg:         nerves_system_br
  Vsn:         1.20.3
  Type:        system_platform
  BuildRunner: {nil, []}

  Pkg:         nerves_system_rpi3
  Vsn:         1.20.0
  Type:        system
  BuildRunner: {Nerves.Artifact.BuildRunners.Local, [make_args: ["source", "all", "legal-info"]]}

|nerves_bootstrap| Loadpaths Start

|nerves_bootstrap| Precompile Start

==> nerves_toolchain_armv7_nerves_linux_gnueabihf
|nerves_bootstrap| Compile.NervesPackage start

|nerves_bootstrap| Compile.NervesPackage end

==> nerves_system_rpi3
|nerves_bootstrap| Compile.NervesPackage start

|nerves_bootstrap| Compile.NervesPackage end

==> athasha_terminal
|nerves_bootstrap| Precompile End


Nerves environment
  MIX_TARGET:   rpi3
  MIX_ENV:      dev

|nerves_bootstrap| Environment Variable List
  target:     rpi3
  toolchain:  /home/samuel/.nerves/artifacts/nerves_toolchain_armv7_nerves_linux_gnueabihf-linux_x86_64-1.6.0
  system:     /home/samuel/.nerves/artifacts/nerves_system_rpi3-portable-1.20.0
  app:        .

|nerves_bootstrap| Loadpaths End

Current behavior

Upload tasks freezes at different percentages most of the times with some success from time to time. Freeze happens with rpi3 target. Same project uploads great to rpi4 target. Already changed SD card.

samuel@build:~/src/athasha/terminal$ MIX_TARGET=rpi3 mix upload athasha-ee0c
==> nerves
==> athasha_terminal

Nerves environment
  MIX_TARGET:   rpi3
  MIX_ENV:      dev

Path: /home/samuel/src/athasha/terminal/_build/rpi3_dev/nerves/images/athasha_terminal.fw
UUID: 88d57e66-f46d-5280-bdf9-7b1ecf8d25c3


Uploading to athasha-ee0c...

fwup: Upgrading partition B
 58% [====================                ] 13.53 MB in / 15.53 MB out^C     
BREAK: (a)bort (A)bort with dump (c)ontinue (p)roc info (i)nfo
       (l)oaded (v)ersion (k)ill (D)b-tables (d)istribution

Expected behavior

Upload task should complete.

[samuelventura]

From time to time I also get write verification errors. This below happened at around 35%. Using the generated scrript makes no difference. SD card is totally new.

samuel@build:~/src/athasha/terminal$ MIX_TARGET=rpi3 mix upload athasha-ee0c
==> nerves
==> athasha_terminal

Nerves environment
  MIX_TARGET:   rpi3
  MIX_ENV:      dev

Path: /home/samuel/src/athasha/terminal/_build/rpi3_dev/nerves/images/athasha_terminal.fw
UUID: 88d57e66-f46d-5280-bdf9-7b1ecf8d25c3


Uploading to athasha-ee0c...

fwup: Upgrading partition B
fwup: write verification failed at offset 26476544
** (Mix) Unexpected exit from ssh (:epipe)

This is known to happen when ssh interactively prompts you for a
passphrase. The following are workarounds:

1. Load your private key identity into the ssh agent by running
   `ssh-add`

2. Use the `upload.sh` script. Create one by running
   `mix firmware.gen.script`.

And a success after that:

samuel@build:~/src/athasha/terminal$ MIX_TARGET=rpi3 mix upload athasha-ee0c
==> nerves
==> athasha_terminal

Nerves environment
  MIX_TARGET:   rpi3
  MIX_ENV:      dev

Path: /home/samuel/src/athasha/terminal/_build/rpi3_dev/nerves/images/athasha_terminal.fw
UUID: 88d57e66-f46d-5280-bdf9-7b1ecf8d25c3


Uploading to athasha-ee0c...

fwup: Upgrading partition B
100% [====================================] 24.73 MB in / 26.76 MB out       
Success!
Elapsed time: 5.733 s

[fhunleth]

@samuelventura I've only seen this happen with bad MicroSD cards and with MicroSD card sockets that have really been used a lot. There is one brand that had many failures reported a year ago and that was Kootion. We never figured out why. I haven't heard issues with the major brands. I primarily use Kingston and Sandisk. What brand are you using?

[samuelventura]

I am using ADATA and SanDisk mainly. I manually uploaded the fw file and ran fwup from nerves shell with same results. This is definitely hw related. I'm closing. Thank you.

Extra observations:

• Each attempt gets a little farther
• scp failed to transfer the fw file but successfully transferred a small text file
• sftp successfully transferred the fw file

iex(5)> cmd "fwup -aU -i /tmp/athasha_terminal.fw -d /dev/mmcblk0 -t upgrade"
fwup: Upgrading partition B
 51% [==================                  ] 11.68 MB in / 13.67 MB out143    
iex(6)> cmd "fwup -aU -i /tmp/athasha_terminal.fw -d /dev/mmcblk0 -t upgrade"
fwup: Upgrading partition B
 61% [=====================               ] 14.34 MB in / 16.33 MB out143    
iex(7)> cmd "fwup -aU -i /tmp/athasha_terminal.fw -d /dev/mmcblk0 -t upgrade"
fwup: Upgrading partition B
 65% [=======================             ] 15.42 MB in / 17.41 MB out143    
iex(8)> cmd "fwup -aU -i /tmp/athasha_terminal.fw -d /dev/mmcblk0 -t upgrade"
fwup: Upgrading partition B
100% [====================================] 24.73 MB in / 26.77 MB out       
Success!
Elapsed time: 5.665 s