armbian/linux-rockchip

rk3528-mali450 does not work properly with lima driver

Closed this issue · 15 comments

eternal-Eason commented 
  █ █ █ █ █ █ █ █ █ █ █      OS: Armbian (23.08.0-trunk) aarch64 
 ███████████████████████     Host: Rockchip RK3528

▄▄██ ██▄▄ Kernel: 5.10.160
▄▄██ ███████████ ██▄▄ Uptime: 7 mins
▄▄██ ██ ██ ██▄▄ Packages: 1420 (dpkg)
▄▄██ ██ ██ ██▄▄ Shell: bash 5.1.16
▄▄██ ██ ██ ██▄▄ Resolution: 1920x1080
▄▄██ █████████████ ██▄▄ Terminal: /dev/ttyFIQ0
▄▄██ ██ ██ ██▄▄ CPU: (4) @ 2.016GHz
▄▄██ ██ ██ ██▄▄ Memory: 647MiB / 3912MiB

Hello, rk3528 is currently seeing mail450 after loading the lima driver with the open source library mesa.
Desktop rendering is a little smoother, but the screen sometimes flickers.

root@rk35xx:~# glmark2-es2-drm =======================================================
glmark2 2021.02 =======================================================
OpenGL Information
GL_VENDOR: Mesa
GL_RENDERER: Mali450
GL_VERSION: OpenGL ES 2.0 Mesa 23.0.4 =======================================================
[build] use-vbo=false:Error: Failed to set crtc: -13
Error: Failed to set crtc: -13
Error: Failed to set crtc: -13
Error: Failed to set crtc: -13
Segmentation fault

=======================================================

glmark2
glmark2 2021.02 =======================================================
OpenGL Information
GL_VENDOR: Mesa
GL_RENDERER: Mali450
GL_VERSION: 2.1 Mesa 23.0.4 =======================================================
[build] use-vbo=false: FPS: 67 FrameTime: 14.925 ms
[build] use-vbo=true: FPS: 62 FrameTime: 16.129 ms
[texture] texture-filter=nearest: FPS: 62 FrameTime: 16.129 ms
[texture] texture-filter=linear: FPS: 62 FrameTime: 16.129 ms
Segmentation fault

How to debug the problem, can guide it? thank you.

amazingfate commented

Do you get any kernel output when the Segmentation fault happens? I guess this is devicetree related.

eternal-Eason commented

Thanks for your reply. According to the prompt, I found that the kernel reported an error.

root@rk35xx:# echo 8 8 8 8 > /proc/sys/kernel/printk
root@rk35xx:# glmark2-es2-drm
glmark2 2021.02

OpenGL Information
GL_VENDOR:     Mesa
GL_RENDERER:   Mali450
GL_VERSION:    OpenGL ES 2.0 Mesa 23.0.4-0ubuntu1~22.04.1

=======================================================
[ 84.739495] alloc_contig_range: [fa550, fa558) PFNs busy
[ 84.739757] alloc_contig_range: [fa558, fa560) PFNs busy
[build] use-vbo=false:Error: Failed to set crtc: -13
Error: Failed to set crtc: -13
Error: Failed to set crtc: -13
Error: Failed to set crtc: -13
Segmentation fault
root@rk35xx:~# [ 103.121581] alloc_contig_range: [fa550, fa558) PFNs busy
[ 103.121850] alloc_contig_range: [fa558, fa560) PFNs busy
[ 103.181886] alloc_contig_range: [fa550, fa558) PFNs busy
[ 103.182006] alloc_contig_range: [fa558, fa560) PFNs busy
[ 103.241589] alloc_contig_range: [fa550, fa558) PFNs busy
[ 103.241708] alloc_contig_range: [fa558, fa560) PFNs busy
[ 103.301452] alloc_contig_range: [fa550, fa558) PFNs busy
[ 103.301569] alloc_contig_range: [fa558, fa560) PFNs busy
[ 103.362125] alloc_contig_range: [fa550, fa558) PFNs busy

Here's the gpu part of my device tree

&gpu{
interrupt-names = "gp", "gpmmu", "pp",
"pp0", "ppmmu0", "pp1", "ppmmu1";
clock-names = "bus", "core";
//clocks = <&cru ACLK_GPU>, <&cru ACLK_GPU>;
//clocks = <&scmi_clk SCMI_CLK_GPU>, <&cru ACLK_GPU_MALI>, <&cru PCLK_GPU_ROOT>;
clocks = <&cru ACLK_GPU_MALI>, <&cru ACLK_GPU_MALI>;
};

eternal-Eason commented

Do you get any kernel output when the Segmentation fault happens? I guess this is devicetree related.

Do you get any kernel output when the Segmentation fault happens? I guess this is devicetree related.

diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 38685860d1ab..59cfc7859f85 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -8911,6 +8911,7 @@ int alloc_contig_range(unsigned long start, unsigned long end,
if (test_pages_isolated(outer_start, end, 0, &info->failed_pfn)) {
pr_info_ratelimited("%s: [%lx, %lx) PFNs busy\n",
func, outer_start, end);

  •           dump_stack();   
          ret = -EBUSY;
          info->err |= ACR_ERR_TEST;
          goto done;

root@rk35xx:~# [ 34.860099] vccio_sd: disabling
[ 48.783010] alloc_contig_range: [fa700, fa708) PFNs busy
[ 48.783046] CPU: 2 PID: 2697 Comm: gnome-shell Not tainted 5.10.160 #22
[ 48.783051] Hardware name: Rockchip RK3528 X88PRO DDR4 V10 Board (DT)
[ 48.783056] Call trace:
[ 48.783070] dump_backtrace+0x0/0x1d0
[ 48.783077] show_stack+0x1c/0x24
[ 48.783086] dump_stack_lvl+0xcc/0xf0
[ 48.783092] dump_stack+0x14/0x30
[ 48.783100] alloc_contig_range+0x564/0x5a0
[ 48.783108] cma_alloc+0x2d8/0x714
[ 48.783114] dma_alloc_contiguous+0x70/0xb0
[ 48.783119] __dma_direct_alloc_pages+0x6c/0x1bc
[ 48.783124] dma_direct_alloc+0x78/0x32c
[ 48.783131] dma_alloc_attrs+0x80/0xec
[ 48.783138] lima_vm_map_page+0x88/0xd4
[ 48.783143] lima_vm_bo_add+0x16c/0x22c
[ 48.783148] lima_gem_object_open+0x28/0x30
[ 48.783154] drm_gem_handle_create_tail+0xf4/0x214
[ 48.783160] drm_gem_handle_create+0x40/0x50
[ 48.783165] lima_gem_create_handle+0x70/0x110
[ 48.783172] lima_ioctl_gem_create+0x3c/0x50
[ 48.783177] drm_ioctl_kernel+0xc4/0x114
[ 48.783182] drm_ioctl+0x228/0x480
[ 48.783189] __arm64_sys_ioctl+0xb4/0xd4
[ 48.783196] el0_svc_common.constprop.0+0x8c/0x230
[ 48.783202] do_el0_svc+0x20/0x30
[ 48.783209] el0_svc+0x1c/0x2c
[ 48.783215] el0_sync_handler+0xa8/0xac
[ 48.783220] el0_sync+0x158/0x180

amazingfate commented

Do you have reserved memory (CMA) in devicetree?
FYI: https://whycan.com/t_9594.html#p87994

eternal-Eason commented

Do you have reserved memory (CMA) in devicetree? FYI: https://whycan.com/t_9594.html#p87994

CONFIG_CMA_SIZE_MBYTES=128

I added the CMA via the kernel configuration file

CmaTotal: 131072 kB
CmaAllocated: 3256 kB
CmaReleased: 127816 kB
CmaFree: 72 kB

[ 0.931353] cma: Reserved 128 MiB at 0x00000000edc00000

It still prints: alloc_contig_range: [edd60, edd68) PFNs busy

sputnik2019 commented

@eternal-Eason How do you install the open source library？mesa？I can't drive up the mali450 on the rk3528 right now, even with MESA installed！

eternal-Eason commented

@eternal-Eason How do you install the open source library？mesa？I can't drive up the mali450 on the rk3528 right now, even with MESA installed！

You don't need to install Mesa as it is already pre-installed in Armbian. However, you need to enable the Lima driver in the kernel.

sputnik2019 commented

@eternal-Eason when enable CONFIG_DRM_LIMA=m and modify dts
&gpu {
mali-supply = <&vdd_gpu>;
status = "okay";
interrupt-names = "gp",
"gpmmu",
"pp",
"pp0",
"ppmmu0",
"pp1",
"ppmmu1";
clocks = <&cru ACLK_GPU_MALI>, <&cru ACLK_GPU_MALI>;
clock-names = "bus", "core";
};

linux boot fail
[ 9.299840] pstate: 60000005 (nZCv daif -PAN -UAO -TCO BTYPE=--)
[ 9.300492] pc : rk_platform_power_off_gpu+0x24/0x80
[ 9.301031] lr : mali_runtime_suspend+0x2c/0x5c
[ 9.301520] sp : ffff80000a4cbc00
[ 9.301881] x29: ffff80000a4cbc00 x28: 0000000000000000
[ 9.302458] x27: ffff0000032951e0 x26: 0000000000000003
[ 9.303035] x25: 00000000e449b3a6 x24: ffff000007198b00
[ 9.303612] x23: 0000000000000000 x22: ffff80000a116950
[ 9.304188] x21: ffff8000089296f4 x20: 0000000000000000
[ 9.304765] x19: ffff000003295010 x18: 0000000000000000
[ 9.305341] x17: 0000000000000000 x16: 0000000000000000
[ 9.305917] x15: 0000000000000000 x14: ffff80000b259220
[ 9.306491] x13: ffff80000b259220 x12: ffff00000aa7b3b0
[ 9.307067] x11: ffff800009e0e098 x10: 0000000000000008
[ 9.307644] x9 : ffff800008871df0 x8 : 0000000000000000
[ 9.308221] x7 : 0000000000000000 x6 : ffff000002c00000
[ 9.308800] x5 : ffff000002914000 x4 : 0000000000000000
[ 9.309377] x3 : ffff000003230f40 x2 : 0000000000000000
[ 9.309954] x1 : ffff000003230f40 x0 : ffff80000a1f1000
[ 9.310537] Call trace:
[ 9.310833] rk_platform_power_off_gpu+0x24/0x80
[ 9.311165] hid-generic 0003:046D:C534.0001: input,hidraw0: USB HID v1.11 Keyboard [Logitech USB Receiver] on usb-ff140000.usb-1/input0
[ 9.311356] mali_runtime_suspend+0x2c/0x5c
[ 9.313135] genpd_runtime_suspend+0xd0/0x200
[ 9.313633] __rpm_callback+0x48/0x120
[ 9.314059] rpm_callback+0x7c/0x88
[ 9.314458] rpm_suspend+0x2a8/0x464
[ 9.314863] pm_runtime_work+0x90/0xa8
[ 9.315276] process_one_work+0x1e0/0x298
[ 9.315714] worker_thread+0x1e0/0x278
[ 9.316124] kthread+0xf4/0x104
[ 9.316469] ret_from_fork+0x10/0x30
[ 9.316860]

eternal-Eason commented

Similar question, I have already asked the RK developer, and they suggested that I update to the latest RK3528 Linux SDK, but I will try it when I have time.

sputnik2019 commented

@eternal-Eason Is your problem solved？

eternal-Eason commented

@sputnik2019 I'm using Armbian with Debian 11 system, and the screen is no longer lagging. The error message 'alloc_contig_range: [fa558, fa560) PFNs busy' is no longer displayed.

1703815921214

sputnik2019 commented

Which branch of the kernel you are using？rk-5.10-rkr6? I can't start the kernel with this branch when enable lima driver！@eternal-Eason

sputnik2019 commented

@eternal-Eason Can you share some of your changes?

eternal-Eason commented

@sputnik2019 You need to address the kernel error log information when loading the lima driver. Either the rk-5.10-rkr6 branch or the rk-5.10-rkr5 branch is acceptable.

amazingfate commented

[ 9.300492] pc : rk_platform_power_off_gpu+0x24/0x80

This is log from arm mali driver, try disabling it first.