Context
Hi guys, I just got my BPI-R4 several days ago. For that, I built a relative large OpenWRT image (~150MB) which should not install into the 128MB NAND flash. (I know I can install it into EMMC by utitlize /tmp and dd command, but using an optane ssd would be better.)
Add nvme support (Failed)
1. Enable NVM Express device / NVM Express PCI device support on menuconfig
- go into the uboot source
./build_dir/target-aarch64_cortex-a53_musl/u-boot-mt7988_bananapi_bpi-r4-snand/u-boot-2024.01 make menuconfig
2. A dive into OpenWRT’s DTS
We can find the OpenWRT’s DTS here:
./target/linux/mediatek/files-6.1/arch/arm64/boot/dts/mediatek
2.1 PCIe pinctrl in mt7988a.dtsi
pcie3_pins: pcie3-pins {
mux {
function = "pcie";
groups = "pcie_1l_1_pereset", "pcie_clk_req_n3",
"pcie_wake_n3_0";
};
};
2.2 PCIe device in mt7988a.dtsi
pcie3: pcie@11290000 {
compatible = "mediatek,mt7988-pcie",
"mediatek,mt7986-pcie",
"mediatek,mt8192-pcie";
reg = <0 0x11290000 0 0x2000>;
reg-names = "pcie-mac";
ranges = <0x81000000 0x00 0x28000000 0x00
0x28000000 0x00 0x00200000>,
<0x82000000 0x00 0x28200000 0x00
0x28200000 0x00 0x07e00000>;
device_type = "pci";
linux,pci-domain = <2>;
interrupts = <GIC_SPI 171 IRQ_TYPE_LEVEL_HIGH>;
bus-range = <0x00 0xff>;
clocks = <&infracfg CLK_INFRA_PCIE_PIPE_P3>,
<&infracfg CLK_INFRA_PCIE_GFMUX_TL_P3>,
<&infracfg CLK_INFRA_PCIE_PERI_26M_CK_P3>,
<&infracfg CLK_INFRA_133M_PCIE_CK_P3>;
clock-names = "pl_250m", "tl_26m", "peri_26m",
"top_133m";
pinctrl-names = "default";
pinctrl-0 = <&pcie3_pins>;
#interrupt-cells = <1>;
interrupt-map-mask = <0 0 0 0x7>;
interrupt-map = <0 0 0 1 &pcie_intc3 0>,
<0 0 0 2 &pcie_intc3 1>,
<0 0 0 3 &pcie_intc3 2>,
<0 0 0 4 &pcie_intc3 3>;
#address-cells = <3>;
#size-cells = <2>;
status = "disabled";
pcie_intc3: interrupt-controller {
#address-cells = <0>;
#interrupt-cells = <1>;
interrupt-controller;
};
};
2.3 PCIe enable entries in mt7988a-bananapi-bpi-r4.dts
/* M.2 key-M SSD */
&pcie3 {
pinctrl-names = "default";
pinctrl-0 = <&pcie3_pins>;
status = "okay";
};
3. Add PCIe3 into uboot’s DTS
We only care about &pcie3 because that is the one for SSD.
3.1 Find the DTS files
Under path
./build_dir/target-aarch64_cortex-a53_musl/u-boot-mt7988_bananapi_bpi-r4-snand/u-boot-2024.01/arch/arm/dts
we can find the following dts file:
and we can find the following build targets from
.config
CONFIG_DEFAULT_FDT_FILE="mediatek/mt7988a-bpi-r4-emmc.dtb"
CONFIG_OF_LIST="mt7988a-bananapi-bpi-r4-emmc"
3.2 The including relationship between DTS files
mt7988a-bananapi-bpi-r4-emmc.dts
→ mt7988a-bananapi-bpi-r4.dtsi
→ → mt7988.dtsi
3.3 Add PCIe3 Device support in mt7988.dtsi
pcie3: pcie@11290000 {
compatible = "mediatek,mt7988-pcie",
"mediatek,mt7986-pcie",
"mediatek,mt8192-pcie";
reg = <0 0x11290000 0 0x2000>;
reg-names = "pcie-mac";
ranges = <0x81000000 0x00 0x28000000 0x00
0x28000000 0x00 0x00200000>,
<0x82000000 0x00 0x28200000 0x00
0x28200000 0x00 0x07e00000>;
device_type = "pci";
linux,pci-domain = <2>;
interrupts = <GIC_SPI 171 IRQ_TYPE_LEVEL_HIGH>;
bus-range = <0x00 0xff>;
clocks = <&infracfg_ao CK_INFRA_PCIE_PIPE_P3>,
<&infracfg_ao CK_INFRA_PCIE_GFMUX_TL_P3>,
<&infracfg_ao CK_INFRA_PCIE_PERI_26M_CK_P3>,
<&infracfg_ao CK_INFRA_133M_PCIE_CK_P3>;
clock-names = "pl_250m", "tl_26m", "peri_26m",
"top_133m";
#interrupt-cells = <1>;
interrupt-map-mask = <0 0 0 0x7>;
interrupt-map = <0 0 0 1 &pcie_intc3 0>,
<0 0 0 2 &pcie_intc3 1>,
<0 0 0 3 &pcie_intc3 2>,
<0 0 0 4 &pcie_intc3 3>;
#address-cells = <3>;
#size-cells = <2>;
status = "disabled";
pcie_intc3: interrupt-controller {
#address-cells = <0>;
#interrupt-cells = <1>;
interrupt-controller;
};
};
There are things to notice when migrate from OpenWRT’s DTS:
- Change
&infracfgto&infracfg_ao: because the label forpinctrlis different. - Change the “CLK_” prefix to
CK_: because the drive for uboot is different than the one in Linux/Openwrt kernal - I removed the
pinctrl-0 = <&pcie3_pins>because I want to defined it later.
3.4 Add PCIe3 pinctrl define in mt7988a-bananapi-bpi-r4.dtsi
Under the pinctrl section, add:
pcie3_pins: pcie3-pins {
mux {
function = "pcie";
groups = "pcie_1l_1_pereset", "pcie_clk_req_n3",
"pcie_wake_n3_0";
};
};
3.5 Add PCIe3 enable in mt7988a-bananapi-bpi-r4-emmc.dts
/* M.2 key-M SSD */
&pcie3 {
pinctrl-names = "default";
pinctrl-0 = <&pcie3_pins>;
status = "okay";
};
4. Compile the fip file
4.1 Compile Uboot first
Simply run the following command and be sure to replace the placeholder:
make clean; make CROSS_COMPILE=<your_openwrt_path>/staging_dir/toolchain-<suffix_is_variated>/bin/<suffix_is_variated>-linux- -j<THREAD> STAGING_DIR=<your_openwrt_path>/staging_dir
And you would see u-boot.bin file.
4.2 Compile the arm-trusted-firmware
Under path:
./build_dir/target-aarch64_cortex-a53_musl/arm-trusted-firmware-mediatek-mt7988-sdmmc-comb/arm-trusted-firmware-mediatek-<suffix_is_variated>
Run:
make -f Makefile PLAT=mt7988 BOOT_DEVICE=snand BL33=<PATH_TO_UBOOT_BIN> all fip CROSS_COMPILE=<your_openwrt_path>/staging_dir/toolchain-<suffix_is_variated>/bin/<suffix_is_variated>-linux- STAGIN_DIR=<your_openwrt_path>/staging_dir
The artifact is under:
build/mt7988/release/fip.bin
5. Install your custom uboot
5.1 Pre-request
You should flash openwrt(official one or your custom build? doesn’t matter) into NAND.
5.2 Flash it!
- Download you fip.bin
- Choose #7 in the Uboot menu
- Rename it as Uboot wants (in my case
openwrt-mediatek-filogic-bananapi_bpi-r4-snand-bl31-uboot.fip) - Run you TFTP server
You will see this when success:
- Press the Reset button to load the new uboot
However, this method failed
No nvme device detacted
The FDT did work, however
-
Load FDT
-
List it
-
We can see that we did assign the device, status, and pinctrl
fdt print /pinctrl@1001f000
Conclusion
I failed to add nvme support to uBoot and the reason is unknow.
May be
- we can discuss the possible reason for this
- we should find a way to list all device (
pcicommand give me empty output) - we should find a way to make Uboot print detail start log
Any help will be appreciated!
but only last picture…are both from uboot? Not sure if ftd show lists the devicetree uboot uses
