Kernel panic - not syncing: Out of memory and no killable processes
shawnhsiao97 opened this issue · 10 comments
shawnhsiao97 commented
Hi guys,
The kernel panic while loading kernel at boot if a uImage size larger than 12MB
I confused what a little size was cause OOM
because I thought the DDR size is 64 MB it has should enough memory space
I follow document flash uboot and kernel into qspiNAND flash and then read kernel from qspi flash to DDR
[ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name
Kernel panic - not syncing: Out of memory and no killable processes
Dose anybody know what cause it and can I just boot kernel in DDR and increase the uImage size?
ychuang3 commented
Which type of board are you using and select which defconfig?
shawnhsiao97 commented
Which type of board are you using and select which defconfig?
Hi @ychuang3 thank you for reply
I used NUC980DK61YC and No I didn't used nuc980_iot_defconfig Instead used need of manuconfig for myself
supplementary log
NUC980 IBR 201SF: 38797312 bytes @ 0x200000 Read: OK
## Booting kernel from Legacy Image at 00007fc0 ...
Image Name: Linux-4.4.207+
Image Type: ARM Linux Kernel Image (uncompressed)
Data Size: 16560776 Bytes = 15.8 MiB
Load Address: 00008000
Entry Point: 00008000
Verifying Checksum ... OK
XIP Kernel Image ... OK
Starting kernel ...
SF: Detected W25N01GV with page size 2 KiB, erase size 128 KiB, total 128 MiB
SF: DeviceBooting Linux on physical CPU 0x0
Linux version 4.4.207+ (r
NUC980 IBR 20180813
Boot from SPI-NAND
DDR-OK
finish SPI dow▒load
SPL load main U-Boot from SPI NAND Flash! (Mar 28 2022 10:33:25)
U-Boot 2016.11-gbf0ff9f-dirty (Mar 28 2022 - 10:33:25 +0800)
CPU: NUC980
Board: NUC980
DRAM: 64 MiB
NAND: NAND Flash not found !
NUC980 NAND CONTROLLER IS NOT SUPPORT THE PAGE SIZE. (0, 0)
0 MiB
SF: Detected W25N01GV with page size 2 KiB, erase size 128 KiB, total 128 MiB
In: serial
Out: serial
Err: serial
Net: Net Initialization Skipped
No ethernet found.
Hit any key to stop autoboot: 0
SF: Detected W25N01GV with page size 2 KiB, erase size 128 KiB, total 128 MiB
device 0 offset 0x200000, size 0x2500000
SF: 38797312 bytes @ 0x200000 Read: OK
## Booting kernel from Legacy Image at 00007fc0 ...
Image Name: Linux-4.4.207+
Image Type: ARM Linux Kernel Image (uncompressed)
Data Size: 16560776 Bytes = 15.8 MiB
Load Address: 00008000
Entry Point: 00008000
Verifying Checksum ... OK
XIP Kernel Image ... OK
Starting kernel ...
SF: Detected W25N01GV with page size 2 KiB, erase size 128 KiB, total 128 MiB
SF: DeviceBooting Linux on physical CPU 0x0
Linux version 4.4.207+ (root@shawn-Default-string) (gcc version 11.2.0 (Ubuntu 11.2.0-5ubuntu1) ) #25 PREEMPT Tue Apr 12 16:51:00 CST 2022
CPU: ARM926EJ-S [41069265] revision 5 (ARMv5TEJ), cr=0005317f
CPU: VIVT data cache, VIVT instruction cache
Machine: NUC980
Memory policy: Data cache writeback
Built 1 zonelists in Zone order, mobility grouping on. Total pages: 16256
Kernel command line: root=/dev/ram0 console=ttyS0,115200n8 rdinit=/sbin/init mem=64M
PID hash table entries: 256 (order: -2, 1024 bytes)
Dentry cache hash table entries: 8192 (order: 3, 32768 bytes)
Inode-cache hash table entries: 4096 (order: 2, 16384 bytes)
Memory: 45372K/65536K available (3962K kernel code, 284K rwdata, 1268K rodata, 13776K init, 216K bss, 20164K reserved, 0K cma-reserved)
Virtual kernel memory layout:
vector : 0xffff0000 - 0xffff1000 ( 4 kB)
fixmap : 0xffc00000 - 0xfff00000 (3072 kB)
vmalloc : 0xc4800000 - 0xff800000 ( 944 MB)
lowmem : 0xc0000000 - 0xc4000000 ( 64 MB)
modules : 0xbf000000 - 0xc0000000 ( 16 MB)
.text : 0xc0008000 - 0xc0523e54 (5232 kB)
.init : 0xc0524000 - 0xc1298000 (13776 kB)
.data : 0xc1298000 - 0xc12df180 ( 285 kB)
.bss : 0xc12df180 - 0xc1315310 ( 217 kB)
SLUB: HWalign=32, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
Preemptible hierarchical RCU implementation.
Build-time adjustment of leaf fanout to 32.
NR_IRQS:545
clocksource: nuc980-timer5: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 62215505635 ns
sched_clock: 24 bits at 120kHz, resolution 8333ns, wraps every 69905062489ns
Console: colour dummy device 80x30
console [ttyS0] enabled
Calibrating delay loop... 148.88 BogoMIPS (lpj=744448)
pid_max: default: 32768 minimum: 301
Mount-cache hash table entries: 1024 (order: 0, 4096 bytes)
Mountpoint-cache hash table entries: 1024 (order: 0, 4096 bytes)
CPU: Testing write buffer coherency: ok
Setting up static identity map for 0x8400 - 0x843c
devtmpfs: initialized
clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
futex hash table entries: 256 (order: -1, 3072 bytes)
pinctrl core: initialized pinctrl subsystem
NET: Registered protocol family 16
DMA: preallocated 256 KiB pool for atomic coherent allocations
SCSI subsystem initialized
usbcore: registered new interface driver usbfs
usbcore: registered new interface driver hub
usbcore: registered new device driver usb
Advanced Linux Sound Architecture Driver Initialized.
clocksource: Switched to clocksource nuc980-timer5
NET: Registered protocol family 2
TCP established hash table entries: 1024 (order: 0, 4096 bytes)
TCP bind hash table entries: 1024 (order: 0, 4096 bytes)
TCP: Hash tables configured (established 1024 bind 1024)
UDP hash table entries: 256 (order: 0, 4096 bytes)
UDP-Lite hash table entries: 256 (order: 0, 4096 bytes)
NET: Registered protocol family 1
swapper invoked oom-killer: gfp_mask=0x24200c2, order=0, oom_score_adj=0
CPU: 0 PID: 1 Comm: swapper Not tainted 4.4.207+ #25
Hardware name: NUC980
Backtrace:
[<c0014e48>] (dump_backtrace) from [<c0015144>] (show_stack+0x18/0x1c)
r7:00002c4f r6:00000000 r5:c3841abc r4:00000000
[<c001512c>] (show_stack) from [<c018c538>] (dump_stack+0x20/0x28)
[<c018c518>] (dump_stack) from [<c00b6f7c>] (dump_header.constprop.0+0x50/0x184)
[<c00b6f2c>] (dump_header.constprop.0) from [<c007f188>] (out_of_memory.part.0+0x2bc/0x300)
r10:c129bd58 r9:c129bc18 r8:00000000 r7:00002c4f r6:00000000 r5:c3841abc
r4:c3936674
[<c007eecc>] (out_of_memory.part.0) from [<c007f268>] (out_of_memory+0x5c/0x6c)
r10:00000000 r9:024200c2 r8:00000031 r7:00000040 r6:c129a0d8 r5:00000000
r4:c3841abc
[<c007f20c>] (out_of_memory) from [<c0083124>] (__alloc_pages_nodemask+0x118/0x984)
r4:00000000
[<c008300c>] (__alloc_pages_nodemask) from [<c007b148>] (pagecache_get_page+0xf8/0x220)
r10:c129a1d8 r9:c12def00 r8:00000031 r7:c362f9bc r6:024200c2 r5:0000000f
r4:00000000
[<c007b050>] (pagecache_get_page) from [<c007b294>] (grab_cache_page_write_begin+0x24/0x38)
r10:00000000 r9:c03ecd00 r8:c00e2484 r7:c362f9bc r6:00031000 r5:00031000
r4:000008c4
[<c007b270>] (grab_cache_page_write_begin) from [<c00e24a8>] (simple_write_begin+0x24/0x170)
r5:00031000 r4:000008c4
[<c00e2484>] (simple_write_begin) from [<c007a404>] (generic_perform_write+0xd4/0x1d0)
r9:c03ecd00 r8:c00e2484 r7:c362f9bc r6:c3841c94 r5:00031000 r4:000008c4
[<c007a330>] (generic_perform_write) from [<c007c7d8>] (__generic_file_write_iter+0x168/0x208)
r10:c3841c94 r9:ffffffff r8:c39326e0 r7:c362f9bc r6:00000000 r5:c3841ca8
r4:00000000
[<c007c670>] (__generic_file_write_iter) from [<c007c9e4>] (generic_file_write_iter+0x16c/0x268)
r10:00000000 r9:ffffffff r8:c39326e0 r7:c362f964 r6:c3841c94 r5:c3841ca8
r4:00008000
[<c007c878>] (generic_file_write_iter) from [<c00b9d08>] (__vfs_write+0xb0/0xe0)
r10:c38a3200 r9:00000000 r8:000298c4 r7:c3841d08 r6:00000000 r5:c3841d08
r4:c39326e0
[<c00b9c58>] (__vfs_write) from [<c00ba1d8>] (vfs_write+0x94/0xf4)
r7:c3841d08 r6:c3990000 r5:c39326e0 r4:00008000
[<c00ba144>] (vfs_write) from [<c00ba850>] (SyS_write+0x54/0xc8)
r7:00008000 r6:c3990000 r5:c39326e0 r4:c39326e0
[<c00ba7fc>] (SyS_write) from [<c0526e40>] (xwrite+0x34/0x6c)
r9:c05448b8 r8:00000007 r7:00000000 r6:c3990000 r5:00008000 r4:00000000
[<c0526e0c>] (xwrite) from [<c0526f20>] (do_copy+0xa8/0x104)
r7:c04b188c r6:00008000 r5:00008000 r4:c05448b8
[<c0526e78>] (do_copy) from [<c052638c>] (write_buffer+0x2c/0x44)
r5:00008000 r4:c05448b8
[<c0526360>] (write_buffer) from [<c05263e0>] (flush_buffer+0x3c/0xa0)
r5:00008000 r4:c3990000
[<c05263a4>] (flush_buffer) from [<c0534958>] (__gunzip+0x2b4/0x378)
r9:c0534690 r8:c0526168 r7:c05263a4 r6:c3990000 r5:c054bae4 r4:00008000
[<c05346a4>] (__gunzip) from [<c0534a50>] (gunzip+0x34/0x3c)
r10:c04b188c r9:00000000 r8:00000000 r7:00000000 r6:00d4c29c r5:c054bae4
r4:c05448b8
[<c0534a1c>] (gunzip) from [<c0526724>] (unpack_to_rootfs+0x178/0x2b4)
[<c05265ac>] (unpack_to_rootfs) from [<c0527048>] (populate_rootfs+0x20/0x24c)
r10:c1297d80 r9:c0521dd8 r8:00000000 r7:c0527028 r6:c129ba40 r5:c39235a0
r4:c129ba40
[<c0527028>] (populate_rootfs) from [<c0009574>] (do_one_initcall+0x88/0x1fc)
r10:c05244e8 r9:c0521dd8 r8:00000000 r7:c0527028 r6:c129ba40 r5:c39235a0
r4:c129ba40
[<c00094ec>] (do_one_initcall) from [<c0524de8>] (kernel_init_freeable+0x10c/0x1c8)
r9:c0521dd8 r8:c054482c r7:c054484c r6:c12df180 r5:0000007b r4:c054b84c
[<c0524cdc>] (kernel_init_freeable) from [<c03e0cec>] (kernel_init+0x10/0xe8)
r10:00000000 r9:00000000 r8:00000000 r7:00000000 r6:00000000 r5:c03e0cdc
r4:00000000
[<c03e0cdc>] (kernel_init) from [<c000f6f8>] (ret_from_fork+0x14/0x3c)
r5:c03e0cdc r4:00000000
Mem-Info:
active_anon:0 inactive_anon:0 isolated_anon:0
active_file:747 inactive_file:0 isolated_file:0
unevictable:9436 dirty:0 writeback:0 unstable:0
slab_reclaimable:561 slab_unreclaimable:238
mapped:0 shmem:0 pagetables:0 bounce:0
free:213 free_pcp:0 free_cma:0
Normal free:852kB min:852kB low:1064kB high:1276kB active_anon:0kB inactive_anon:0kB active_file:2988kB inactive_file:0kB unevictable:37744kB isolated(anon):0kB isolated(file):0kB present:65536kB managed:45372kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:2244kB slab_unreclaimable:952kB kernel_stack:136kB pagetables:0kB unstable:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:37744 all_unreclaimable? yes
lowmem_reserve[]: 0 0
Normal: 3*4kB (UME) 3*8kB (UME) 1*16kB (E) 3*32kB (UE) 3*64kB (UE) 2*128kB (UE) 1*256kB (E) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 852kB
10183 total pagecache pages
16384 pages RAM
0 pages HighMem/MovableOnly
5041 pages reserved
[ pid ] uid tgid total_vm rss nr_ptes nr_pmds swapents oom_score_adj name
Kernel panic - not syncing: Out of memory and no killable processes...
CPU: 0 PID: 1 Comm: swapper Not tainted 4.4.207+ #25
Hardware name: NUC980
Backtrace:
[<c0014e48>] (dump_backtrace) from [<c0015144>] (show_stack+0x18/0x1c)
r7:00002c4f r6:00000000 r5:c3841abc r4:c12df428
[<c001512c>] (show_stack) from [<c018c538>] (dump_stack+0x20/0x28)
[<c018c518>] (dump_stack) from [<c0078bb8>] (panic+0xa8/0x228)
[<c0078b14>] (panic) from [<c007f190>] (out_of_memory.part.0+0x2c4/0x300)
r3:00000000 r2:000000d0 r1:c12a1d1c r0:c04b9058
r7:00002c4f
[<c007eecc>] (out_of_memory.part.0) from [<c007f268>] (out_of_memory+0x5c/0x6c)
r10:00000000 r9:024200c2 r8:00000031 r7:00000040 r6:c129a0d8 r5:00000000
r4:c3841abc
[<c007f20c>] (out_of_memory) from [<c0083124>] (__alloc_pages_nodemask+0x118/0x984)
r4:00000000
[<c008300c>] (__alloc_pages_nodemask) from [<c007b148>] (pagecache_get_page+0xf8/0x220)
r10:c129a1d8 r9:c12def00 r8:00000031 r7:c362f9bc r6:024200c2 r5:0000000f
r4:00000000
[<c007b050>] (pagecache_get_page) from [<c007b294>] (grab_cache_page_write_begin+0x24/0x38)
r10:00000000 r9:c03ecd00 r8:c00e2484 r7:c362f9bc r6:00031000 r5:00031000
r4:000008c4
[<c007b270>] (grab_cache_page_write_begin) from [<c00e24a8>] (simple_write_begin+0x24/0x170)
r5:00031000 r4:000008c4
[<c00e2484>] (simple_write_begin) from [<c007a404>] (generic_perform_write+0xd4/0x1d0)
r9:c03ecd00 r8:c00e2484 r7:c362f9bc r6:c3841c94 r5:00031000 r4:000008c4
[<c007a330>] (generic_perform_write) from [<c007c7d8>] (__generic_file_write_iter+0x168/0x208)
r10:c3841c94 r9:ffffffff r8:c39326e0 r7:c362f9bc r6:00000000 r5:c3841ca8
r4:00000000
[<c007c670>] (__generic_file_write_iter) from [<c007c9e4>] (generic_file_write_iter+0x16c/0x268)
r10:00000000 r9:ffffffff r8:c39326e0 r7:c362f964 r6:c3841c94 r5:c3841ca8
r4:00008000
[<c007c878>] (generic_file_write_iter) from [<c00b9d08>] (__vfs_write+0xb0/0xe0)
r10:c38a3200 r9:00000000 r8:000298c4 r7:c3841d08 r6:00000000 r5:c3841d08
r4:c39326e0
[<c00b9c58>] (__vfs_write) from [<c00ba1d8>] (vfs_write+0x94/0xf4)
r7:c3841d08 r6:c3990000 r5:c39326e0 r4:00008000
[<c00ba144>] (vfs_write) from [<c00ba850>] (SyS_write+0x54/0xc8)
r7:00008000 r6:c3990000 r5:c39326e0 r4:c39326e0
[<c00ba7fc>] (SyS_write) from [<c0526e40>] (xwrite+0x34/0x6c)
r9:c05448b8 r8:00000007 r7:00000000 r6:c3990000 r5:00008000 r4:00000000
[<c0526e0c>] (xwrite) from [<c0526f20>] (do_copy+0xa8/0x104)
r7:c04b188c r6:00008000 r5:00008000 r4:c05448b8
[<c0526e78>] (do_copy) from [<c052638c>] (write_buffer+0x2c/0x44)
r5:00008000 r4:c05448b8
[<c0526360>] (write_buffer) from [<c05263e0>] (flush_buffer+0x3c/0xa0)
r5:00008000 r4:c3990000
[<c05263a4>] (flush_buffer) from [<c0534958>] (__gunzip+0x2b4/0x378)
r9:c0534690 r8:c0526168 r7:c05263a4 r6:c3990000 r5:c054bae4 r4:00008000
[<c05346a4>] (__gunzip) from [<c0534a50>] (gunzip+0x34/0x3c)
r10:c04b188c r9:00000000 r8:00000000 r7:00000000 r6:00d4c29c r5:c054bae4
r4:c05448b8
[<c0534a1c>] (gunzip) from [<c0526724>] (unpack_to_rootfs+0x178/0x2b4)
[<c05265ac>] (unpack_to_rootfs) from [<c0527048>] (populate_rootfs+0x20/0x24c)
r10:c1297d80 r9:c0521dd8 r8:00000000 r7:c0527028 r6:c129ba40 r5:c39235a0
r4:c129ba40
[<c0527028>] (populate_rootfs) from [<c0009574>] (do_one_initcall+0x88/0x1fc)
r10:c05244e8 r9:c0521dd8 r8:00000000 r7:c0527028 r6:c129ba40 r5:c39235a0
r4:c129ba40
[<c00094ec>] (do_one_initcall) from [<c0524de8>] (kernel_init_freeable+0x10c/0x1c8)
r9:c0521dd8 r8:c054482c r7:c054484c r6:c12df180 r5:0000007b r4:c054b84c
[<c0524cdc>] (kernel_init_freeable) from [<c03e0cec>] (kernel_init+0x10/0xe8)
r10:00000000 r9:00000000 r8:00000000 r7:00000000 r6:00000000 r5:c03e0cdc
r4:00000000
[<c03e0cdc>] (kernel_init) from [<c000f6f8>] (ret_from_fork+0x14/0x3c)
r5:c03e0cdc r4:00000000
---[ end Kernel panic - not syncing: Out of memory and no killable processes...
random: nonblocking pool is initialized
ychuang3 commented
Please make kernel with the NUC980 defconfig.
shawnhsiao97 commented
Please make kernel with the NUC980 defconfig.
Hi @ychuang3 , I have made kernel with the NUC980 defonfig now.
but unfortunately still the same
ychuang3 commented
(1) git clone git@github.com:OpenNuvoton/NUC970_Buildroot.git
(2) make nuvoton_nuc980_iot_defconfig
(3) make
(4) use NuWriter to download output/images/Image
Can it work?
shawnhsiao97 commented
Hi @ychuang3 thank you for reply
Yes, it can work.
but my problem is what uImage size over 12MB will cause kernel panic
AFAIK I think the EVB have 64MB of memory in a DDR, therefore it shouldn't happen
Follow log mentioned above, the reason for the kernel panic is what it cannot allocate the memory when rootfs decompressed by kernel I think.
May be there is caused by memory allocation, but I do not have any idea solve it, do you have it?
ychuang3 commented
My kernel image size is 12 MB, it works well.
Would you upload your .config?
shawnhsiao97 commented
My kernel image size is 12 MB, it works well. Would you upload your .config?
my .config is the same with nuvoton_nuc980_iot_defconfig
but I had addition some libraries, tools and others, there will increase my rootfs size.
ychuang3 commented
You can move rootfs from ramfs to other stoage such as NAND.
shawnhsiao97 commented
You can move rootfs from ramfs to other stoage such as NAND.
I would like use DDR if I can, I'm confused why I can only use so little space size?