chia init only fully completes 1 drive when attempting multiple partition and formats

Question

chia init only fully completes 1 drive when attempting multiple partition and formats

Mathis436 opened this issue a year ago · 12 comments

sudo ./chiainit --fstype ext4 --init sdy sdab sdw sdu sds sdq sdz sdx sdaa sdv sdt sdr

Operation Summary
Drives : sdy sdab sdw sdu sds sdq sdz sdx sdaa sdv sdt sdr
Excluded :
Filesystem : ext4
Actions : init

Do you want to continue? (Y/n)
Y
WARNING: You have selected a destructive action (wipe, format, or init).
THIS WILL DESTROY ALL YOUR DATA ON THE SPECIFIED DRIVE(S).
sdy sdab sdw sdu sds sdq sdz sdx sdaa sdv sdt sdr

To proceed, type 'YES I SACRIFICE THIS DATA' and press Enter.
YES I SACRIFICE THIS DATA

Drive /dev/sdy:
Wiping /dev/sdy...
/dev/sdy: 8 bytes were erased at offset 0x00001000 (gpt): 45 46 49 20 50 41 52 54
/dev/sdy: 8 bytes were erased at offset 0x74702555000 (gpt): 45 46 49 20 50 41 52 54
/dev/sdy: 2 bytes were erased at offset 0x000001fe (PMBR): 55 aa
/dev/sdy: calling ioctl to re-read partition table: Success
Formatting /dev/sdy...
mke2fs 1.47.0 (5-Feb-2023)
/dev/sdy1 contains a zfs_member file system labelled 'mmcuplot2'
Creating filesystem with 1953506304 4k blocks and 1907744 inodes
Filesystem UUID: c226291a-5ae3-4e4c-b4a5-eb71c9dab9be
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968,
102400000, 214990848, 512000000, 550731776, 644972544, 1934917632

Allocating group tables: done
Writing inode tables: done
Creating journal (262144 blocks): done
Writing superblocks and filesystem accounting information: done

Labeling /dev/sdy1...
Serial Number: R5GG0AUV
UUID: UNKNOWN
New label:

Drive /dev/sdab:
Wiping /dev/sdab...
/dev/sdab: 8 bytes were erased at offset 0x00001000 (gpt): 45 46 49 20 50 41 52 54
/dev/sdab: 8 bytes were erased at offset 0x74702555000 (gpt): 45 46 49 20 50 41 52 54
/dev/sdab: 2 bytes were erased at offset 0x000001fe (PMBR): 55 aa
/dev/sdab: calling ioctl to re-read partition table: Success
Formatting /dev/sdab...
Labeling /dev/sdab1...
Serial Number: R5G9WN2V
UUID: UNKNOWN
New label:

Drive /dev/sdw:
Wiping /dev/sdw...
/dev/sdw: 8 bytes were erased at offset 0x00001000 (gpt): 45 46 49 20 50 41 52 54
/dev/sdw: 8 bytes were erased at offset 0x74702555000 (gpt): 45 46 49 20 50 41 52 54
/dev/sdw: 2 bytes were erased at offset 0x000001fe (PMBR): 55 aa
/dev/sdw: calling ioctl to re-read partition table: Success
Formatting /dev/sdw...
Labeling /dev/sdw1...
Serial Number: R5GEN2VV
UUID: UNKNOWN
New label:

Drive /dev/sdu:
Wiping /dev/sdu...
/dev/sdu: 8 bytes were erased at offset 0x00001000 (gpt): 45 46 49 20 50 41 52 54
/dev/sdu: 8 bytes were erased at offset 0x74702555000 (gpt): 45 46 49 20 50 41 52 54
/dev/sdu: 2 bytes were erased at offset 0x000001fe (PMBR): 55 aa
/dev/sdu: calling ioctl to re-read partition table: Success
Formatting /dev/sdu...
Labeling /dev/sdu1...
Serial Number: R5GEJMBV
UUID: UNKNOWN
New label:

Drive /dev/sds:
Wiping /dev/sds...
/dev/sds: 8 bytes were erased at offset 0x00001000 (gpt): 45 46 49 20 50 41 52 54
/dev/sds: 8 bytes were erased at offset 0x74702555000 (gpt): 45 46 49 20 50 41 52 54
/dev/sds: 2 bytes were erased at offset 0x000001fe (PMBR): 55 aa
/dev/sds: calling ioctl to re-read partition table: Success
Formatting /dev/sds...
Labeling /dev/sds1...
Serial Number: R5GBLVZV
UUID: UNKNOWN
New label:

Drive /dev/sdq:
Wiping /dev/sdq...
/dev/sdq: 8 bytes were erased at offset 0x00001000 (gpt): 45 46 49 20 50 41 52 54
/dev/sdq: 8 bytes were erased at offset 0x74702555000 (gpt): 45 46 49 20 50 41 52 54
/dev/sdq: 2 bytes were erased at offset 0x000001fe (PMBR): 55 aa
/dev/sdq: calling ioctl to re-read partition table: Success
Formatting /dev/sdq...
Labeling /dev/sdq1...
Serial Number: R5G1HRKV
UUID: UNKNOWN
New label:

Drive /dev/sdz:
Wiping /dev/sdz...
/dev/sdz: 8 bytes were erased at offset 0x00001000 (gpt): 45 46 49 20 50 41 52 54
/dev/sdz: 8 bytes were erased at offset 0x74702555000 (gpt): 45 46 49 20 50 41 52 54
/dev/sdz: 2 bytes were erased at offset 0x000001fe (PMBR): 55 aa
/dev/sdz: calling ioctl to re-read partition table: Success
Formatting /dev/sdz...
Labeling /dev/sdz1...
Serial Number: R5GEU6AV
UUID: UNKNOWN
New label:

Drive /dev/sdx:
Wiping /dev/sdx...
/dev/sdx: 8 bytes were erased at offset 0x00001000 (gpt): 45 46 49 20 50 41 52 54
/dev/sdx: 8 bytes were erased at offset 0x74702555000 (gpt): 45 46 49 20 50 41 52 54
/dev/sdx: 2 bytes were erased at offset 0x000001fe (PMBR): 55 aa
/dev/sdx: calling ioctl to re-read partition table: Success
Formatting /dev/sdx...
Labeling /dev/sdx1...
Serial Number: R5GBLSLV
UUID: UNKNOWN
New label:

Drive /dev/sdaa:
Wiping /dev/sdaa...
/dev/sdaa: 8 bytes were erased at offset 0x00001000 (gpt): 45 46 49 20 50 41 52 54
/dev/sdaa: 8 bytes were erased at offset 0x74702555000 (gpt): 45 46 49 20 50 41 52 54
/dev/sdaa: 2 bytes were erased at offset 0x000001fe (PMBR): 55 aa
/dev/sdaa: calling ioctl to re-read partition table: Success
Formatting /dev/sdaa...
Labeling /dev/sdaa1...
Serial Number: 7HK1E73F
UUID: UNKNOWN
New label:

Drive /dev/sdv:
Wiping /dev/sdv...
/dev/sdv: 8 bytes were erased at offset 0x00001000 (gpt): 45 46 49 20 50 41 52 54
/dev/sdv: 8 bytes were erased at offset 0x74702555000 (gpt): 45 46 49 20 50 41 52 54
/dev/sdv: 2 bytes were erased at offset 0x000001fe (PMBR): 55 aa
/dev/sdv: calling ioctl to re-read partition table: Success
Formatting /dev/sdv...
Labeling /dev/sdv1...
Serial Number: R5GBW83V
UUID: UNKNOWN
New label:

Drive /dev/sdt:
Wiping /dev/sdt...
/dev/sdt: 8 bytes were erased at offset 0x00001000 (gpt): 45 46 49 20 50 41 52 54
/dev/sdt: 8 bytes were erased at offset 0x74702555000 (gpt): 45 46 49 20 50 41 52 54
/dev/sdt: 2 bytes were erased at offset 0x000001fe (PMBR): 55 aa
/dev/sdt: calling ioctl to re-read partition table: Success
Formatting /dev/sdt...
Labeling /dev/sdt1...
Serial Number: R5GBPL1V
UUID: UNKNOWN
New label:

Drive /dev/sdr:
Wiping /dev/sdr...
/dev/sdr: 8 bytes were erased at offset 0x00001000 (gpt): 45 46 49 20 50 41 52 54
/dev/sdr: 8 bytes were erased at offset 0x74702555000 (gpt): 45 46 49 20 50 41 52 54
/dev/sdr: 2 bytes were erased at offset 0x000001fe (PMBR): 55 aa
/dev/sdr: calling ioctl to re-read partition table: Success
Formatting /dev/sdr...
Labeling /dev/sdr1...
Serial Number: R5GG7BVV
UUID: UNKNOWN
New label:

Operation complete for all specified drives.

Answer 1 · 2023-11-29T16:15:48.000Z

as the above shows. only 1 drive went through the whole process. I did receive Python errors during install. (see below)

error: externally-managed-environment

× This environment is externally managed
mq> To install Python packages system-wide, try apt install
python3-xyz, where xyz is the package you are trying to
install.

If you wish to install a non-Debian-packaged Python package,
create a virtual environment using python3 -m venv path/to/venv.
Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
sure you have python3-full installed.

If you wish to install a non-Debian packaged Python application,
it may be easiest to use pipx install xyz, which will manage a
virtual environment for you. Make sure you have pipx installed.

See /usr/share/doc/python3.11/README.venv for more information.

note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
hint: See PEP 668 for the detailed specification.
error: externally-managed-environment

× This environment is externally managed
mq> To install Python packages system-wide, try apt install
python3-xyz, where xyz is the package you are trying to
install.

If you wish to install a non-Debian-packaged Python package,
create a virtual environment using python3 -m venv path/to/venv.
Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
sure you have python3-full installed.

If you wish to install a non-Debian packaged Python application,
it may be easiest to use pipx install xyz, which will manage a
virtual environment for you. Make sure you have pipx installed.

See /usr/share/doc/python3.11/README.venv for more information.

note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
hint: See PEP 668 for the detailed specification.

Answer 2 · 2023-11-29T16:16:12.000Z

any help would be appreciated.

Answer 3 · 2023-11-29T18:01:55.000Z

Hi Mathis436, thanks for you feedback.

I have just installed the lastest version on a test system here and did not get the error you mentionned.
chiainit does not have any dependencies on python, so we can skip the errors you got during installation (at least for now).

Could you show the output of lsblk, please?
What kind of system are you running this on?
Are you running the latest version of chiagarden?

Answer 4 · 2023-11-29T18:17:29.000Z

Lsbk is not copying correctly from the terminal.

it is a Ubuntu server 22.04 just freshly loaded
approximately 180 hdd split by 15 per array
the drives in question were apart of a zfs pool about 12 per pool
latest version of chia garden

Answer 5 · 2023-11-29T18:18:12.000Z

if I run the script with 1 drive at a time all is good

Answer 6 · 2023-11-30T03:18:36.000Z

I didn't understand what you mean by "lsbk is not copying correctly from the terminal"
can you show the output of "lsblk"?

Answer 7 · 2023-12-04T13:48:15.000Z

Mathis436 commented a year ago

Answer 8 · 2023-12-04T14:23:35.000Z

Maybe try unmounting the disks first. It doesnt quite explain the output you're seeing described at the beginning of this issue, but I know that chiainit wont operate on mounted drives (for safety reasons)

for your setup you can use something like this:

grep -o '/mnt/bay[0-9]*' /proc/mounts | xargs -I {} umount {}

Answer 9 · 2023-12-06T16:30:30.000Z

I've updated the chiainit script.

https://chiaforum.com/t/chiagarden-a-toolkit-for-post-farming-on-linux/20919/34?u=efnats

This new version isnt as strict on mounted disks. It will still refuse to process, but it'll give better error output.

Answer 10 · 2023-12-06T16:41:16.000Z

Thank you for the update. I think the disks were re shuffled after a reboot on the system I'm working with. I reviewed the out put of lsblk that I posted and saw that some of the disks were in a different position in the dev tree than before. The original issue is still there, the drives that are problematic are ones that I remove from a zfs pool. They are not mounted and the pool was destroyed using 'zpool destroy' command. I am about to try to run chia init on a set of drives that I just removed from zfs. I will update with the results using the most recent script.

…

On Wed, Dec 6, 2023, 10:30 AM efnats ***@***.***> wrote: I've updated the chiainit script. https://chiaforum.com/t/chiagarden-a-toolkit-for-post-farming-on-linux/20919/34?u=efnats This new version isnt as strict on mounted disks. It will still refuse to process, but it'll give better error output. — Reply to this email directly, view it on GitHub <#2 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AQGQTIOI3QUOVEQAMDEESY3YICMTBAVCNFSM6AAAAAA77XMKDCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNBTGI2DOMBZGE> . You are receiving this because you authored the thread.Message ID: ***@***.***>

Answer 11 · 2023-12-06T16:44:49.000Z

These are the drive sets I will be working with:

william@harvester56:$ blkid | grep -E mmcuplot3
/dev/sdds2: LABEL="mmcuplot3" UUID="731586632537411585" UUID_SUB="15875005551098520552" BLOCK_SIZE="4096" TYPE="zfs_member" PARTUUID="c3bf6088-1b7d-4459-bd60-dc22947666b6"
/dev/sddq2: LABEL="mmcuplot3" UUID="731586632537411585" UUID_SUB="18188458188632961107" BLOCK_SIZE="4096" TYPE="zfs_member" PARTUUID="5434c876-f7b5-4acc-8f7e-7d86d92c373d"
/dev/sddn2: LABEL="mmcuplot3" UUID="731586632537411585" UUID_SUB="9183859404714344735" BLOCK_SIZE="4096" TYPE="zfs_member" PARTUUID="fde8f798-3902-4608-b42e-4d3792f1bcc3"
/dev/sddv2: LABEL="mmcuplot3" UUID="731586632537411585" UUID_SUB="8198670406398713545" BLOCK_SIZE="4096" TYPE="zfs_member" PARTUUID="3305e013-7fa8-4c4b-97fa-6cb92193e581"
/dev/sddt2: LABEL="mmcuplot3" UUID="731586632537411585" UUID_SUB="4874665968607209161" BLOCK_SIZE="4096" TYPE="zfs_member" PARTUUID="b5ea83c5-891f-4a66-9013-caf958b89d48"
/dev/sddr2: LABEL="mmcuplot3" UUID="731586632537411585" UUID_SUB="1443365415624434724" BLOCK_SIZE="4096" TYPE="zfs_member" PARTUUID="e5a1584a-d384-4b60-b282-d09998856415"
/dev/sddp2: LABEL="mmcuplot3" UUID="731586632537411585" UUID_SUB="12082118715416661166" BLOCK_SIZE="4096" TYPE="zfs_member" PARTUUID="7407cc8e-95af-46a2-b400-6258832a0486"
/dev/sddy2: LABEL="mmcuplot3" UUID="731586632537411585" UUID_SUB="8771935977194457155" BLOCK_SIZE="4096" TYPE="zfs_member" PARTUUID="c52814f8-c334-485f-935e-4dcb9a107d20"
/dev/sddo2: LABEL="mmcuplot3" UUID="731586632537411585" UUID_SUB="17416634591345524616" BLOCK_SIZE="4096" TYPE="zfs_member" PARTUUID="94852135-9529-49b7-ba8d-455ed217a21c"
/dev/sddw2: LABEL="mmcuplot3" UUID="731586632537411585" UUID_SUB="14491296879361635987" BLOCK_SIZE="4096" TYPE="zfs_member" PARTUUID="b16cc2f9-5cd9-4825-86fd-b0375e38a2f2"
/dev/sddm2: LABEL="mmcuplot3" UUID="731586632537411585" UUID_SUB="5196215417963777085" BLOCK_SIZE="4096" TYPE="zfs_member" PARTUUID="c4dcab04-c0e6-48b3-8154-e967f0967256"
/dev/sddu2: LABEL="mmcuplot3" UUID="731586632537411585" UUID_SUB="9364638646207385226" BLOCK_SIZE="4096" TYPE="zfs_member" PARTUUID="c81d9488-ddaa-42dc-9d8b-525272aae876"
william@harvester56:$ blkid | grep -E mmcuplot4
/dev/sdei2: LABEL="mmcuplot4" UUID="12257070796734361074" UUID_SUB="6747168187267872197" BLOCK_SIZE="4096" TYPE="zfs_member" PARTUUID="993d4907-6f9c-49f5-954a-62b360403983"
/dev/sdeg2: LABEL="mmcuplot4" UUID="12257070796734361074" UUID_SUB="3357397129131831856" BLOCK_SIZE="4096" TYPE="zfs_member" PARTUUID="753cdbc5-7169-44c6-8f3a-95f61aa9e7ed"
/dev/sdee2: LABEL="mmcuplot4" UUID="12257070796734361074" UUID_SUB="14906228786015713968" BLOCK_SIZE="4096" TYPE="zfs_member" PARTUUID="41057ab1-f818-4059-bd21-dccc534a994e"
/dev/sden2: LABEL="mmcuplot4" UUID="12257070796734361074" UUID_SUB="7016490663483222203" BLOCK_SIZE="4096" TYPE="zfs_member" PARTUUID="a581ddbf-9f53-4a72-aded-aeace3a06a84"
/dev/sdec2: LABEL="mmcuplot4" UUID="12257070796734361074" UUID_SUB="9910297281975193445" BLOCK_SIZE="4096" TYPE="zfs_member" PARTUUID="d3b8210f-7249-4e41-8fba-54b29a0ce818"
/dev/sdel2: LABEL="mmcuplot4" UUID="12257070796734361074" UUID_SUB="11841322529742156781" BLOCK_SIZE="4096" TYPE="zfs_member" PARTUUID="782b1267-9661-4840-8128-21a34297b0ee"
/dev/sdej2: LABEL="mmcuplot4" UUID="12257070796734361074" UUID_SUB="6071811071664839850" BLOCK_SIZE="4096" TYPE="zfs_member" PARTUUID="a2e30de8-f579-4b86-ab3f-50acbe9df8d8"
/dev/sdeh2: LABEL="mmcuplot4" UUID="12257070796734361074" UUID_SUB="11856261972681364045" BLOCK_SIZE="4096" TYPE="zfs_member" PARTUUID="cab581b2-a1dd-4128-b98d-d0e75b2c7b4b"
/dev/sdef2: LABEL="mmcuplot4" UUID="12257070796734361074" UUID_SUB="10455415265497263325" BLOCK_SIZE="4096" TYPE="zfs_member" PARTUUID="76a9064a-84e6-4ec8-aca0-3f66fbe55b60"
/dev/sded2: LABEL="mmcuplot4" UUID="12257070796734361074" UUID_SUB="3492747265520844990" BLOCK_SIZE="4096" TYPE="zfs_member" PARTUUID="65bfc961-beb3-4105-91e1-514c4d5b927b"
/dev/sdeb2: LABEL="mmcuplot4" UUID="12257070796734361074" UUID_SUB="14204935648313710617" BLOCK_SIZE="4096" TYPE="zfs_member" PARTUUID="a0a82c0b-0704-463c-bcbc-34f06bf2e590"
/dev/sdek2: LABEL="mmcuplot4" UUID="12257070796734361074" UUID_SUB="16560546347906069205" BLOCK_SIZE="4096" TYPE="zfs_member" PARTUUID="0e350908-8639-4bab-8a85-48f66486d2d9"
william@harvester56:~$ blkid | grep -E mmcuplot6
/dev/sdfi2: LABEL="mmcuplot6" UUID="5652974470699221253" UUID_SUB="9500366803252952534" BLOCK_SIZE="4096" TYPE="zfs_member" PARTUUID="db8c8948-4026-4077-ad4e-26ba614407d0"
/dev/sdfq2: LABEL="mmcuplot6" UUID="5652974470699221253" UUID_SUB="14514779525521540965" BLOCK_SIZE="4096" TYPE="zfs_member" PARTUUID="8fcda900-94e0-4761-9a0a-c61449261c81"
/dev/sdfg2: LABEL="mmcuplot6" UUID="5652974470699221253" UUID_SUB="11414995663284118089" BLOCK_SIZE="4096" TYPE="zfs_member" PARTUUID="a2e65b5f-8342-4c4b-942a-a881c0ee74e0"
/dev/sdfn2: LABEL="mmcuplot6" UUID="5652974470699221253" UUID_SUB="2107388543361531863" BLOCK_SIZE="4096" TYPE="zfs_member" PARTUUID="0a361b93-bca7-4f64-a493-b21b1bd3ebc8"
/dev/sdfl2: LABEL="mmcuplot6" UUID="5652974470699221253" UUID_SUB="8794879725291272290" BLOCK_SIZE="4096" TYPE="zfs_member" PARTUUID="075a613e-6248-4eac-8b30-3d17deb4e04e"
/dev/sdfj2: LABEL="mmcuplot6" UUID="5652974470699221253" UUID_SUB="5890401749611128605" BLOCK_SIZE="4096" TYPE="zfs_member" PARTUUID="c3412e79-8f8b-4c0b-92cf-faeb699546cd"
/dev/sdfr2: LABEL="mmcuplot6" UUID="5652974470699221253" UUID_SUB="2253073252408298285" BLOCK_SIZE="4096" TYPE="zfs_member" PARTUUID="9adddd00-d881-4521-82c6-4f664cf91c92"
/dev/sdfh2: LABEL="mmcuplot6" UUID="5652974470699221253" UUID_SUB="3783095676991772588" BLOCK_SIZE="4096" TYPE="zfs_member" PARTUUID="84fbffe2-69ea-492a-a5c9-0f947490143b"
/dev/sdfp2: LABEL="mmcuplot6" UUID="5652974470699221253" UUID_SUB="430937743258217499" BLOCK_SIZE="4096" TYPE="zfs_member" PARTUUID="f0437943-0ec8-40a6-8f21-a3b88021cdb9"
/dev/sdff2: LABEL="mmcuplot6" UUID="5652974470699221253" UUID_SUB="15948115014804191267" BLOCK_SIZE="4096" TYPE="zfs_member" PARTUUID="2ce57c9f-3cc8-4acc-8db2-66dc3eb89395"
/dev/sdfo2: LABEL="mmcuplot6" UUID="5652974470699221253" UUID_SUB="4204737998571487521" BLOCK_SIZE="4096" TYPE="zfs_member" PARTUUID="4c25c09d-3d5d-442e-bf3f-5eb51f4146d1"
/dev/sdfm2: LABEL="mmcuplot6" UUID="5652974470699221253" UUID_SUB="13880043201164852661" BLOCK_SIZE="4096" TYPE="zfs_member" PARTUUID="9593b255-a02a-4ac1-9e43-16b92b654bec"
/dev/sdfk2: LABEL="mmcuplot6" UUID="5652974470699221253" UUID_SUB="12786961627030989449" BLOCK_SIZE="4096" TYPE="zfs_member" PARTUUID="04cb832c-ed7a-4956-96e0-c0cc808ee934"
/dev/sdfs2: LABEL="mmcuplot6" UUID="5652974470699221253" UUID_SUB="10518637236739477654" BLOCK_SIZE="4096" TYPE="zfs_member" PARTUUID="353f1412-5fc1-49e7-87d5-e755b531fbce"

Answer 12 · 2023-12-06T16:58:24.000Z

All seems to be working now. It is on the 4th drive and all is going as expected!
If all the drives Finnish without an issue i will close this issue.