So I have embarked on an odd project, just to see if it can be done. I am trying to get my AMD RX580 to work with the Banana Pi R4 but am running into issues with trying to map a big enough BAR space. I have been following Jeff Geerling’s attempts to do similar things on the raspberry pi for inspiration. Increase the BAR memory address space for PCIe devices on the Raspberry Pi Compute Module 4 · GitHub
It looks like I need 512MB to get rid of any BAR issues, and the only way to get that to fit is to move up in memory. I tried 0x16 instead of 0x28 set in the mt7988a.dtsi file:
ranges = <0x81000000 0x00 0x28000000 0x00
0x28000000 0x00 0x00200000>,
<0x82000000 0x00 0x28200000 0x00
0x28200000 0x00 0x07e00000>;
[ 3.384469] pci 0002:00:00.0: BAR 9: no space for [mem size 0x18000000 64bit pref]
[ 3.392027] pci 0002:00:00.0: BAR 9: failed to assign [mem size 0x18000000 64bit pref]
[ 3.399935] pci 0002:00:00.0: BAR 8: assigned [mem 0x28200000-0x282fffff]
[ 3.406714] pci 0002:00:00.0: BAR 0: assigned [mem 0x28300000-0x28307fff 64bit]
[ 3.414018] pci 0002:00:00.0: BAR 7: assigned [io 0x200000-0x200fff]
[ 3.420450] pci 0002:01:00.0: BAR 0: no space for [mem size 0x10000000 64bit pref]
[ 3.428008] pci 0002:01:00.0: BAR 0: failed to assign [mem size 0x10000000 64bit pref]
[ 3.435914] pci 0002:01:00.0: BAR 2: no space for [mem size 0x00200000 64bit pref]
[ 3.443471] pci 0002:01:00.0: BAR 2: failed to assign [mem size 0x00200000 64bit pref]
[ 3.451374] pci 0002:01:00.0: BAR 5: assigned [mem 0x28200000-0x2823ffff]
[ 3.458155] pci 0002:01:00.0: BAR 6: assigned [mem 0x28240000-0x2825ffff pref]
[ 3.465366] pci 0002:01:00.1: BAR 0: assigned [mem 0x28260000-0x28263fff 64bit]
[ 3.472675] pci 0002:01:00.0: BAR 4: assigned [io 0x200000-0x2000ff]
[ 3.479106] pci 0002:00:00.0: PCI bridge to [bus 01]
[ 3.484065] pci 0002:00:00.0: bridge window [io 0x200000-0x200fff]
[ 3.490495] pci 0002:00:00.0: bridge window [mem 0x28200000-0x282fffff]
This does solve the BAR issue, but it seems to cause another issue. I can no longer talk to the ROM BAR when attempting modprobe amdgpu.
Before moving to 0x16:
root@OpenWrt:~# modprobe amdgpu
[ 223.070487] [drm] amdgpu kernel modesetting enabled.
[ 223.075582] amdgpu 0002:01:00.0: assign IRQ: got 115
[ 223.080689] amdgpu 0002:01:00.0: enabling device (0000 -> 0003)
[ 223.086629] [drm] initializing kernel modesetting (POLARIS10 0x1002:0x67DF 0x1458:0x22FC 0xE7).
[ 223.095346] [drm] register mmio base: 0x28200000
[ 223.099950] [drm] register mmio size: 262144
[ 223.104323] [drm] add ip block number 0 <vi_common>
[ 223.109190] [drm] add ip block number 1 <gmc_v8_0>
[ 223.113984] [drm] add ip block number 2 <tonga_ih>
[ 223.118763] [drm] add ip block number 3 <gfx_v8_0>
[ 223.123545] [drm] add ip block number 4 <sdma_v3_0>
[ 223.128411] [drm] add ip block number 5 <powerplay>
[ 223.133280] [drm] add ip block number 6 <dm>
[ 223.137538] [drm] add ip block number 7 <uvd_v6_0>
[ 223.142319] [drm] add ip block number 8 <vce_v3_0>
[ 223.355252] amdgpu 0002:01:00.0: amdgpu: Fetched VBIOS from ROM BAR
[ 223.361514] amdgpu: ATOM BIOS: xxx-xxx-xxx
[ 223.365630] [drm] UVD is enabled in VM mode
[ 223.369801] [drm] UVD ENC is enabled in VM mode
[ 223.374326] [drm] VCE enabled in VM mode
[ 223.378238] amdgpu 0002:01:00.0: amdgpu: Trusted Memory Zone (TMZ) feature not supported
[ 223.386321] amdgpu 0002:01:00.0: amdgpu: PCIE atomic ops is not supported
[ 223.393107] [drm] GPU posting now...
[ 223.528769] [drm] vm size is 64 GB, 2 levels, block size is 10-bit, fragment size is 9-bit
[ 223.557315] amdgpu 0002:01:00.0: amdgpu: VRAM: 8192M 0x000000F400000000 - 0x000000F5FFFFFFFF (8192M used)
[ 223.566885] amdgpu 0002:01:00.0: amdgpu: GART: 256M 0x000000FF00000000 - 0x000000FF0FFFFFFF
[ 223.575235] [drm] Detected VRAM RAM=8192M, BAR=0M
[ 223.579928] [drm] RAM width 256bits GDDR5
[ 223.584060] [drm] amdgpu: 8192M of VRAM memory ready
[ 223.589015] [drm] amdgpu: 1964M of GTT memory ready.
[ 223.594020] [drm] GART: num cpu pages 65536, num gpu pages 65536
[ 223.602023] [drm] PCIE GART of 256M enabled (table at 0x000000F5FFF80000).
[ 223.609009] mtk-pcie-gen3 11290000.pcie: msi#0x1 address_hi 0x0 address_lo 0x11290c00 data 1
[ 223.636698] [drm] Chained IB support enabled!
[ 223.668590] amdgpu: hwmgr_sw_init smu backed is polaris10_smu
[ 223.736536] [drm] Found UVD firmware Version: 1.130 Family ID: 16
[ 223.761840] [drm] Found VCE firmware Version: 53.26 Binary ID: 3
[ 223.769200] amdgpu 0002:01:00.0: enabling bus mastering
[ 223.838080] [drm:amdgpu_mm_wdoorbell [amdgpu]] *ERROR* writing beyond doorbell aperture: 0x000001e8!
[ 228.114505] amdgpu: SMU load firmware failed
[ 228.118773] amdgpu: fw load failed
[ 228.122172] amdgpu: smu firmware loading failed
[ 228.126695] amdgpu 0002:01:00.0: amdgpu: amdgpu_device_ip_init failed
[ 228.133130] amdgpu 0002:01:00.0: amdgpu: Fatal error during GPU init
[ 228.139471] amdgpu 0002:01:00.0: amdgpu: amdgpu: finishing device.
After moving to 0x16
root@OpenWrt:/lib/modules/6.6.92# modprobe amdgpu
[ 95.048985] [drm] amdgpu kernel modesetting enabled.
[ 95.054057] amdgpu 0002:01:00.0: assign IRQ: got 115
[ 95.059205] amdgpu 0002:01:00.0: enabling device (0000 -> 0003)
[ 95.065130] [drm] initializing kernel modesetting (POLARIS10 0x1002:0x67DF 0x1458:0x22FC 0xE7).
[ 95.073910] [drm] register mmio base: 0x16000000
[ 95.078523] [drm] register mmio size: 262144
[ 95.082874] [drm] add ip block number 0 <vi_common>
[ 95.087755] [drm] add ip block number 1 <gmc_v8_0>
[ 95.092534] [drm] add ip block number 2 <tonga_ih>
[ 95.097433] [drm] add ip block number 3 <gfx_v8_0>
[ 95.102211] [drm] add ip block number 4 <sdma_v3_0>
[ 95.107094] [drm] add ip block number 5 <powerplay>
[ 95.111960] [drm] add ip block number 6 <dm>
[ 95.116227] [drm] add ip block number 7 <uvd_v6_0>
[ 95.121015] [drm] add ip block number 8 <vce_v3_0>
[ 95.125875] amdgpu 0002:01:00.0: Invalid PCI ROM header signature: expecting 0xaa55, got 0x0000
[ 95.134593] amdgpu 0002:01:00.0: Invalid PCI ROM header signature: expecting 0xaa55, got 0x0000
[ 95.143309] amdgpu 0002:01:00.0: amdgpu: Unable to locate a BIOS ROM
[ 95.149657] amdgpu 0002:01:00.0: amdgpu: Fatal error during GPU init
[ 95.156002] amdgpu 0002:01:00.0: amdgpu: amdgpu: finishing device.
[ 95.162189] amdgpu: probe of 0002:01:00.0 failed with error -22
I have tried disabling all other pcie devices in the mt7988a-banana-bpi-r4.dtsi file, and then setting pcie3(where my m.2 to pci-e adapter is plugged in) to the memory range of pcie2 since it starts at 0x20, but I seem to get the same Invalid PCI ROM header error.
I saw in the documentation that the BAR size can be 256MB, so maybe it is just impossible for me to stretch them to 512MB?
0x2000_0000 0x27FF_FFFF 256MB PCIE2 data block (64-bit address)
0x2800_0000 0x2FFF_FFFF 256MB PCIE3 data block (64-bit address)
0x3000_0000 0x37FF_FFFF 256MB PCIE0 data block (64-bit address)
0x3800_0000 0x3FFF_FFFF 256MB PCIE1 data block (64-bit address)
But when I lspci -vv I do see this line which makes me think it is possible.
Capabilities: [200 v1] Physical Resizable BAR
BAR 0: current size: 256MB, supported: 256MB 512MB 1GB 2GB 4GB 8GB
Here is a copy of /proc/iomem I was using to try to find enough room to fit a 512MB BAR space.
So really my main question is does anyone know why I suddenly cannot access ROM BAR space when I move to either 0x16 or 0x20? Also if anyone knows of anywhere to read up more on this kind of stuff I’d appreciate it. Hopefully this dumb project makes someone laugh.
So far no luck in expanding the BAR size to 512MB, but 256MB expansion does work. I ended up swapping out the RX580 with an Nvidia GTX660, a much older gpu that I had lying around. Luckily this gpu only required a 256MB bar, so modifying mt7988a.dtsi and changing the range to:
Next came hours and hours of tinkering and compiling the video packages so I could add Wayland and Weston to the device. I ended up having to use the main branch of Openwrt as I ran into a missing libdrm-nouveau dependency issue.
Once Wayland, Weston, and all other required drivers and software were installed I attempted to start Weston, but ran into an interesting error:
root@OpenWrt:~# weston
Date: 2025-06-19 GMT
[16:53:07.411] weston 14.0.1
https://wayland.freedesktop.org
Bug reports to: https://gitlab.freedesktop.org/wayland/weston/issues/
Build: 14.0.1
[16:53:07.411] Command line: weston
[16:53:07.411] OS: Linux, 6.6.93, #0 SMP Mon Jun 16 21:10:54 2025, aarch64
[16:53:07.411] Flight recorder: enabled
[16:53:07.411] Starting with no config file.
[16:53:07.412] Output repaint window is 7 ms maximum.
[16:53:07.412] WARNING: Weston was started from /dev/ttyS0. Messages will be dropped if not written to a file.
Use --log for capturing Weston logs to a file.
[16:53:07.412] Loading module '/usr/lib/libweston-14/drm-backend.so'
[16:53:07.413] initializing drm backend
[16:53:07.413] Trying libseat launcher...
[16:53:07.413] [libseat/libseat.c:73] Seat opened with backend 'seatd'
[16:53:07.413] [libseat/backend/seatd.c:212] Enabling seat
[16:53:07.414] libseat: session control granted
[16:53:07.428] using /dev/dri/card0
[16:53:07.429] DRM: does not support atomic modesetting
[16:53:07.429] DRM: supports GBM modifiers
[16:53:07.429] DRM: does not support Atomic async page flip
[16:53:07.429] DRM: supports picture aspect ratio
[16:53:07.490] Loading module '/usr/lib/libweston-14/gl-renderer.so'
[16:53:07.495] Using rendering device: /dev/dri/renderD128
[16:53:07.495] EGL version: 1.5
[16:53:07.495] EGL vendor: Mesa Project
[16:53:07.495] EGL client APIs: OpenGL OpenGL_ES
[16:53:07.495] warning: Disabling render GPU timeline and explicit synchronization due to missing EGL_ANDROID_native_fence_sync extension
[16:53:07.495] EGL features:
EGL Wayland extension: yes
context priority: no
buffer age: yes
partial update: no
swap buffers with damage: no
configless context: yes
surfaceless context: yes
dmabuf support: modifiers
[16:53:07.501] GL version: OpenGL ES 3.2 Mesa 25.0.5
[16:53:07.501] GLSL version: OpenGL ES GLSL ES 3.20
[16:53:07.501] GL vendor: Mesa
[16:53:07.501] GL renderer: NVE6
[16:53:07.534] GL ES 3.2 - renderer features:
read-back format: ARGB8888
glReadPixels supports y-flip: yes
glReadPixels supports PBO: yes
wl_shm 10 bpc formats: yes
wl_shm 16 bpc formats: yes
wl_shm half-float formats: yes
internal R and RG formats: yes
OES_EGL_image_external: yes
[16:53:07.534] Using GL renderer
[16:53:07.557] warning: no input devices on entering Weston. Possible causes:
- no permissions to read /dev/input/event*
- seats misconfigured (Weston backend option 'seat', udev device property ID_SEAT)
[16:53:07.557] failed to create input devices
[ 313.377683] nouveau 0002:01:00.0: seatd[2123]: failed to idle channel 2 [seatd[2123]]
[16:53:22.587] fatal: failed to create compositor backend
Segmentation fault
Since the device did not see an input device, it crashed. I ended up having to add these packages:
kmod-input-core
kmod-input-evdev
kmod-hid
kmod-hid-generic
kmod-usb-hid
I also earlier had to install these to get Weston to install correctly:
kmod-sound-core
alsa-lib
Once the new packages were installed and a USB mouse was plugged in I got:
root@OpenWrt:~# weston
Date: 2025-06-19 GMT
[18:10:46.841] weston 14.0.1
https://wayland.freedesktop.org
Bug reports to: https://gitlab.freedesktop.org/wayland/weston/issues/
Build: 14.0.1
[18:10:46.842] Command line: weston
[18:10:46.842] OS: Linux, 6.6.93, #0 SMP Mon Jun 16 21:10:54 2025, aarch64
[18:10:46.842] Flight recorder: enabled
[18:10:46.842] Starting with no config file.
[18:10:46.842] Output repaint window is 7 ms maximum.
[18:10:46.842] WARNING: Weston was started from /dev/ttyS0. Messages will be dropped if not written to a file.
Use --log for capturing Weston logs to a file.
[18:10:46.842] Loading module '/usr/lib/libweston-14/drm-backend.so'
[18:10:46.844] initializing drm backend
[18:10:46.844] Trying libseat launcher...
[18:10:46.844] [libseat/libseat.c:73] Seat opened with backend 'seatd'
[18:10:46.844] [libseat/backend/seatd.c:212] Enabling seat
[18:10:46.844] libseat: session control granted
[18:10:46.861] using /dev/dri/card0
[18:10:46.861] DRM: does not support atomic modesetting
[18:10:46.861] DRM: supports GBM modifiers
[18:10:46.861] DRM: does not support Atomic async page flip
[18:10:46.861] DRM: supports picture aspect ratio
[18:10:47.000] Loading module '/usr/lib/libweston-14/gl-renderer.so'
[18:10:47.[ 233.661016] weston[4237]: memfd_create() called without MFD_EXEC or MFD_NOEXEC_SEAL set
005] Using rendering device: /dev/dri/renderD128
[18:10:47.005] EGL version: 1.5
[18:10:47.005] EGL vendor: Mesa Project
[18:10:47.005] EGL client APIs: OpenGL OpenGL_ES
[18:10:47.005] warning: Disabling render GPU timeline and explicit synchronization due to missing EGL_ANDROID_native_fence_sync extension
[18:10:47.005] EGL features:
EGL Wayland extension: yes
context priority: no
buffer age: yes
partial update: no
swap buffers with damage: no
configless context: yes
surfaceless context: yes
dmabuf support: modifiers
[18:10:47.037] GL version: OpenGL ES 3.2 Mesa 25.0.5
[18:10:47.037] GLSL version: OpenGL ES GLSL ES 3.20
[18:10:47.037] GL vendor: Mesa
[18:10:47.038] GL renderer: NVE6
[18:10:47.104] GL ES 3.2 - renderer features:
read-back format: ARGB8888
glReadPixels supports y-flip: yes
glReadPixels supports PBO: yes
wl_shm 10 bpc formats: yes
wl_shm 16 bpc formats: yes
wl_shm half-float formats: yes
internal R and RG formats: yes
OES_EGL_image_external: yes
[18:10:47.105] Using GL renderer
[18:10:47.195] event0 - Logitech USB-PS/2 Optical Mouse: is tagged by udev as: Mouse
[18:10:47.196] event0 - Logitech USB-PS/2 Optical Mouse: device is a pointer
[18:10:47.197] libinput: configuring device "Logitech USB-PS/2 Optical Mouse".
[18:10:47.209] DRM: head 'DVI-I-1' found, connector 85 is disconnected.
[18:10:47.285] DRM: EDID for the following head fails conformity:
Block 1, CTA-861 Extension Block:
Colorimetry Data Block: Reserved bits MD0-MD3 must be 0.
[18:10:47.285] DRM: head 'HDMI-A-1' found, connector 88 is connected, EDID make 'Sony', model 'SONY TV *00', serial '0x01010101'
Supported EOTF modes: SDR
Supported colorimetry modes: default
[18:10:47.443] DRM: head 'DP-1' found, connector 90 is disconnected.
[18:10:47.445] DRM: head 'DVI-D-1' found, connector 93 is disconnected.
[18:10:47.445] Registered plugin API 'weston_drm_output_api_v1' of size 40
[18:10:47.445] Color manager: no-op
protocol support: no
[18:10:47.445] Output 'HDMI-A-1' attempts EOTF mode SDR and colorimetry mode default.
[18:10:47.445] Output 'HDMI-A-1' using color profile: stock sRGB color profile
[18:10:47.446] Chosen EGL config details: id: 33 rgba: 8 8 8 0 buf: 24 dep: 0 stcl: 0 int: 1-1 type: win vis_id: XRGB8888 (0x34325258)
[18:10:47.446] Output HDMI-A-1 (crtc 83) video modes:
[email protected] 256:135, current, 297.0 MHz
[email protected] 16:9, 297.0 MHz
[email protected] 16:9, 297.0 MHz
[email protected] 16:9, 297.0 MHz
[email protected], 148.5 MHz
[email protected] 16:9, 148.5 MHz
[email protected] 16:9, 148.4 MHz
[email protected] 16:9, 74.2 MHz
[email protected] 16:9, 74.2 MHz
[email protected] 16:9, 148.5 MHz
[email protected] 16:9, 74.2 MHz
[email protected] 16:9, 74.2 MHz
[email protected] 16:9, 74.2 MHz
[email protected], 119.0 MHz
[email protected], 108.0 MHz
[email protected], 108.0 MHz
[email protected], 108.0 MHz
[email protected], 74.2 MHz
[email protected] 16:9, 74.2 MHz
[email protected] 16:9, 74.2 MHz
[email protected] 16:9, 74.2 MHz
[email protected] 16:9, 74.2 MHz
[email protected] 16:9, 59.4 MHz
[email protected], 65.0 MHz
[email protected], 40.0 MHz
[email protected] 16:9, 27.0 MHz
[email protected] 4:3, 27.0 MHz
[email protected] 16:9, 27.0 MHz
[email protected] 4:3, 27.0 MHz
[email protected] 16:9, 27.0 MHz
[email protected] 4:3, 27.0 MHz
[email protected] 4:3, 25.2 MHz
[email protected], 25.2 MHz
[email protected] 4:3, 25.2 MHz
[18:10:47.447] associating input device event0 with output HDMI-A-1 (none by udev)
[18:10:47.447] Output 'HDMI-A-1' enabled with head(s) HDMI-A-1
[18:10:47.447] Compositor capabilities:
arbitrary surface rotation: yes
screen capture uses y-flip: yes
cursor planes: yes
arbitrary resolutions: no
view mask clipping: yes
explicit sync: no
color operations: yes
presentation clock: CLOCK_MONOTONIC, id 1
presentation clock resolution: 0.000000001 s
[18:10:47.447] Loading module '/usr/lib/weston/desktop-shell.so'
[18:10:47.447] launching '/usr/libexec/weston-keyboard'
[18:10:47.449] launching '/usr/libexec/weston-desktop-shell'
Everything is detected and running! Except, no output on the screen
My best guess right now is that I am interacting via UART or SSH, so I don’t have a proper terminal for it to start on? I am wondering if I need to add support for the pcie in U-boot like here:
I tried turning on Virtual terminal in make kernel_menuconfig, but am unsure if this did anything or if I just don’t know what I am doing here. Any help is appreciated!
Flashed frank-w’s U-boot which does detect the gpu via pci command, but still not output just yet.
Thanks! I started messing around with the enabling the framebuffer, but am still running into issues.
I can enable DRM under Graphics Support in kernel_menuconfig and also enable legacy fbdev support to ensure I get a /dev/fb0, but am still getting an error about detecting the HDMI connection. If I use a DVI connection I get no error but it still doesn’t seem to work.
[ 6.354253] [drm] Initialized nouveau 1.4.0 20120801 for 0002:01:00.0 on minor 0
[ 6.916550] nouveau 0002:01:00.0: DRM: [DRM/00000003:sor-0002-0f42] [ACQUIRE proto:TMDS head:1 hdmi:1 max_ac_packet:41 rekey:56 scdc:0 hda:1] or:108 link:128 (ret:-22)
[ 8.919488] nouveau 0002:01:00.0: DRM: core notifier timeout
[ 10.919541] nouveau 0002:01:00.0: DRM: base-0: timeout
[ 12.919589] nouveau 0002:01:00.0: DRM: base-1: timeout
[ 12.985039] nouveau 0002:01:00.0: [drm] fb0: nouveaudrmfb frame buffer device
If I go into frame buffer devices and select the Nvidia framebuffer support driver here I get a different error:
I did find information here for getting a USB Displaylink device working with OpenWRT
http://h-wrt.com/en/doc/video
I have a displaylink, so just to see if I was going down the correct path, I followed the instructions on that site and was able to get output onto my monitor!
It’s not what I want, but it is a good step in the right direction. For now I need to figure out why the Nvidia framebuffer driver doesn’t detect my gpu’s architecture. It is possible I have a 3rd party bios flashed on this card, I do remember messing around with that a long time ago. Might be best to plug it into an x86 machine and flash it back to stock just to be safe.
Else, I may need to just ignore the framebuffer driver and focus on the DRM one. Still lots to research, I am very new to this!
Changed it to a http link but I do get DNS issues trying to access it at work vs. at home. Can try Googling for H-WRT as well then go to Documentation then Display.
Could I be missing firmware? With AMD gpu’s I had to enable the driver in kernel_menuconfig but also add the firmware in menuconfig, whereas Nouveau only has the kernel driver and no firmware. I was reading this may be because the firmware is Nvidia’s and therefore cannot be packaged and must be manually added by me? Anyone have experience with installing Nouveau even on x86?
