[BPI-R3] NVME connection issue

[RRKh61]

Hi all.

I am trying to connect NVME to my BPI-R3. At the thirst start (after power up) OpenWRT doesn’t see any pcie devises. In the kernel log I see the following: mtk-pcie-gen3 11280000.pcie: PCIe link down, ltssm reg val: 0x3000003 mtk-pcie-gen3: probe of 11280000.pcie failed with error -110 In “/sys/bus/pci/” doesn’t have any devices. “echo 1 > /sys/bus/pci/rescan” has no affect.

After a soft reboot, the pcie device discovered, everything works fine, and I can mount NVME. But that’s it all until the next power off. I use resent OpenWRT snapshot.

Any opinions or suggestions on how to fix the situation?

[sulaco]

I haven’t tried soft-reboot to see if the device shows up, but after a standard boot, I also don’t see an attached NVME device.

[DeadMeat (Alex R.)]

I had no NMVe issues (at least on cold boot, didn’t tested reboot yet) with a self-biult image followed by:

Banana PI R3 BSP source code

It is all (TF-A, U-Boot, Kernel, DT) already merged upstream in OpenWrt. Unlike the initial BSP supplied by SinoVoip, all hardware features are supported there now (incl. automatic switching of SGMII, 1000Base-X, 2500Base-X for the SFP cages, temperature controlled PWM fan, …). git clone https://git.openwrt.org/openwrt/openwrt.git cd openwrt scripts/feeds update -a scripts/feeds install -a make menuconfig # select MediaTek ARM target, Filogic subtarget, BananaPi R3 board. Then choose packages y…

[frank-w]

You should say what Nvme (vendor and partno) you are using. Bpi-r3 has x2 pcie,maybe your nvme is a v4 and have No x2 compatibility.

[jtxnn678]

No NVME issue on cold boot or whatsoever, I included kmod-nvme in build config.

WD Blue SN550 NVME 500GB

[RRKh61]

I am using Patriot P300P1TBM28 (PCIe x4, SM2263XT controller). As I said, it works, but after a soft reset.

[jtxnn678]

do you compile firmware yourself?

[RRKh61]

No, I used the firmware selector (https://firmware-selector.openwrt.org/?version=SNAPSHOT&target=mediatek%2Ffilogic&id=bananapi_bpi-r3)

[jtxnn678]

base-files busybox ca-bundle dnsmasq dropbear e2fsprogs f2fsck firewall4 fstools kmod-crypto-hw-safexcel kmod-gpio-button-hotplug kmod-hwmon-pwmfan kmod-i2c-gpio kmod-leds-gpio kmod-mt7915e kmod-mt7986-firmware kmod-nft-offload kmod-sfp kmod-usb3 libc libgcc libustream-wolfssl logd mkf2fs mtd netifd nftables odhcp6c odhcpd-ipv6only opkg ppp ppp-mod-pppoe procd procd-seccomp procd-ujail uboot-envtools uci uclient-fetch urandom-seed urngd wpad-basic-wolfssl

Based on the firmware selector, it doesn’t include the kmod-nvme module as built-in. I don’t know if that is the cause of issue, as I just installed M.2 NVME yesterday using a old drive pulled off from my PC, and rebuilt the firmware by include kmod-nvme.

You can try to customize installed package by edit & add kmod-nvme module, then request build, and see if it works.

[sulaco]

I was missing kmod-nvme. Now I see the device as expected, however, now I’m running OpenWRT issues with moving the SD card overlay over to the NVME, but that’s not related to this issue.

Thanks @jtxnn678 for the help.

[RRKh61]

Of course kmod-nvme is installed. I installed it right away (like many other packages) via the LuCI interface.

[jtxnn678]

Of course kmod-nvme is installed.

then happy troubleshooting urself, Im sorry if my suggestion made u looked dumb & offended u. “as I said”, “of cause”… you are too smart and don’t need opinion & suggestion.

[Voidstranger]

Report: Samsung 970 Evo plus nvme ssd (500 gb version) works with BPI-R3 without issues.

[jtxnn678]

Report: Samsung 970 Evo plus nvme ssd (500 gb version) works with BPI-R3 without issues.

Do you have kmod-nvme as built-in (kernel) or just installed from package?

[Voidstranger]

I built openwrt image from source with kmod-nvme built-in.

[jtxnn678]

Then thats correct. the kmod-nvme has to be compiled in kernel in order to solve cold boot nvme not found issue.

[RRKh61]

No matter how to install kmod-nvme. I tried both ways and result is the same.

I think the problem in too long init of NVME controller. Maybe I “won” the silicon lottery or something

I looked the Linux PCIe host controller driver ( pcie-mediatek-gen3.c) and decide modify it to fully comply the PCIe CEM clause 2.2.

Now I haven’t any issue with NVME. The patch in attach.

611-pcie-mediatek-gen3-PERST-for-100ms.patch (677 Байты)

[frank-w]

Thx for the Patch,you should send it to linux kernel mailing list to have it fixed in mainline

[Voidstranger]

Do I need to trim f2fs ssd? Fstrim -a -v command trims 0 bytes on my /dev/nvme0n1.

[RRKh61]

I think trim is needed. Maybe for some reason your drive does not support trim, try “lsblk --discard” it will show support trim.

In my case I’m use ext4 and trim supported. “fstrim -v /mnt/nvme0n1p2” for the first time gives all free spase on ssd is trimmed, after it givs 0 bytes.

But I’m too lazy, and I set the “discard” in “Mount options” so I don’t have to think about it.