BPI-R2 Pro RK3568 CRYPTO_DEV_ROCKCHIP2 [IPsec] Hardware Accleration

Banana Pi Router design BPI-R2 Pro(RK3568)
1

Hello all,

Got strongswan working on R2 Pro, but the performance is lackluster at best. 78 Mbps UP and DOWN best, saturating one CPU with 100% irq. Better than Mikrotik hAP ax lite [35 Mbps UP and DOWN]

Found a thread on someone is working on CRYPTO_DEV_ROCKCHIP2, [PATCH RFT 4/5] crypto: rockchip: support the new crypto IP for rk3568/rk3588 but I don’t know how to only port this ROCKCHIP2 folder into the source tree and test.

I suspect even if I port the folder over, it will drag bunch of dependencies and create conflicts here and there.

2

Easiest way is searching the patch in rockchip patchwork (state any and also history),export mbox file and apply the patch file with git am

https://patchwork.kernel.org/project/linux-rockchip/patch/[email protected]/

It does not look complex to add,but i guess you will need an additional dts node (compatible rockchip,rk3568-crypto) to trigger the driver.

Ah i see it is a series…and last part is the dts patch

it looks like the series depend on additional patches for rk3288-crypto,

seems to be this: https://patchwork.kernel.org/project/linux-rockchip/patch/[email protected]/

and comments in coverletter there is a report that it does not work…maybe you can contact Corentin to send out a newer patchset

i tried to apply the rk3288 one to 6.1 tree and it fails in many positions…maybe author can send a new one or rebase on 6.1

3

I will contact the author and report back. I might need to read the datasheets and code myself as a last resort. Can’t wait to pick up C, haven’t used since high school.

4

Have you noticed this one?

[PATCH RFT 4/5] crypto: rockchip: support the new crypto IP for rk3568/rk3588 — ARM, OMAP, Xscale Linux Kernel (spinics.net)

5

it seems to be same patchset like above…

6

Then this must be the way to go… Perhapse it will merge better with a newer linux version?

7

The problem is that in any comment it is tested as non-working

https://patchwork.kernel.org/project/linux-rockchip/cover/[email protected]/#25027889

And it seems it needs some patches before i have not found yet…the other series i’ve found is newer

looks like the first patch from the series are already in (moving kconfig), as there is already a rockchip dir and the rk3288 is already there…maybe we just need to skip the first patch(es)

https://github.com/torvalds/linux/tree/master/drivers/crypto/rockchip

but this version was reported as non-working and i did not found newer version and no response yet from author

14

got answer from author that he will look again into these patches…

15

Got answer from author again and he pointed me to his actual code,but have not tested yet

Commits · montjoie/linux · GitHub

have added his patches in my 6.5-r2pro tree, but not yet tested

16

Thanks for all the helping. I will take a look soon. I am having some trouble with the journctl -xe on your base Ubuntu image, continuously running as a StrongSwan server for 20 days now.

17

Had not running the board so long…serial console should work when you reconnect and get additional output…maybe you missed the baud setting (1500000) on reconnect or use wrong serial device? In other thread you’ve wrote that ssh works well then,so board is not freezed.

But as r2pro uses onboard uart adapter,it is possible that this has freezed.

I have booted the kernel with the crypto driver enabled after fixing issue with renamed pmic driver (prevents sdcard access). See no dmesg messages for crypto and also no dma errors…need to know how to check. Maybe i enable the debugfs option then i should see these entries

18

updated code

and tested crypto like this:

root@bpi-r2pro:~# kcapi-rng -b 512 > rng.bin
root@bpi-r2pro:~# cat /sys/kernel/debug/rk3588_crypto/stats
rk3588-crypto fe380000.crypto requests: 596
ecb-aes-rk2 ecb(aes) reqs=131 fallback=2005
        fallback due to length: 352
        fallback due to alignment: 1649
        fallback due to SGs: 0
cbc-aes-rk2 cbc(aes) reqs=154 fallback=2164
        fallback due to length: 378
        fallback due to alignment: 1768
        fallback due to SGs: 6
xts-aes-rk2 xts(aes) reqs=141 fallback=949
        fallback due to length: 0
        fallback due to alignment: 360
        fallback due to SGs: 0
rk2-md5 md5 reqs=13 fallback=717
rk2-sha1 sha1 reqs=35 fallback=661
rk2-sha256 sha256 reqs=27 fallback=620
rk2-sha384 sha384 reqs=32 fallback=662
rk2-sha512 sha512 reqs=35 fallback=702
rk2-sm3 sm3 reqs=28 fallback=662
19

Thanks Frank. I got some time tonight to download the code and re-compiled the kernel.

I am having some trouble with clients on the subnet having 5Mbps download, but 68Mbps upload. the upload is similar to before the crypto modules were applied.

Got a kernel message saying

alg: No test for echainiv(authenc(hmac(sha256),cbc(des3_ede))) (echainiv(authenc(hmac(sha256-ce),cbc(des3_ede-generic))))

20

It is possible that you need to enable some more options…look what i have added last to get into the right page in menuconfig.

21

I started with your config through ./build.sh importconfig

Then added my driver support and nf tables.

The funny thing is that I lost all inbound connection ability as well. I can’t SSH or do an Iperf3 speed test to the BPi anymore. Wondering whether this is nftables related, but tcpdump is also quite slow.

22

If you have no nftables rules applied it should have no impact. Thats strange…you could try using 6.6 without the crypto offloading enabled…i have only made basic test over console…no network test with 6.6 yet

23

I highly doubt the crypto extensions would break the network configuration at large. I am going to learn how the crypto extenion could help with xfrm encryption and decryption.

You did not change any network related kernel options from 6.2 to 6.6?

24

No, not as far as i remember. Config should be similar as on 6.1 except the network-stack…only noticed that mfd driver was renamed in 6.5 from rk808 to rk8xx which breakes sdcard too.

For testing crypto we need to look on debugfs and compare the values to them after running the test command

25

That probably explained when my system wasn’t booting after bringing to 6.5. You know I needed to keep pressing Maskrom, because all I do is writing the new kernel onto the SD card.

I will do a diff between the default importconfig for 6.6 and my working 6.2 kernel (with strongswan, nft, and everything needed for a 70/70 IPsec tunnel). Trying to get better the speed than this. The WAN is 125/125, limitation of the WiFi card/dorm congestion.

26

at least i see 2 traces on bootup that tests failed, that i do not have noticed before

[    1.111914] alg: ahash: rk2-sm3 test failed (wrong result) on test vector "random: psize=4992 ksize=0", cfg="random: may_sleep use_digest sr"
[    1.113720] alg: self-tests for sm3 using rk2-sm3 failed (rc=-22)                                                                            
[    1.113732] ------------[ cut here ]------------                                                                                             
[    1.114701] alg: self-tests for sm3 using rk2-sm3 failed (rc=-22)                                                                            
[    1.114743] WARNING: CPU: 1 PID: 96 at crypto/testmgr.c:5936 alg_test.part.0+0x128/0x3e8                                                     
[    1.116046] Modules linked in:                                                                                                               
[    1.116340] CPU: 1 PID: 96 Comm: cryptomgr_test Not tainted 6.6.0-rc4-bpi-r2pro-r2pro #7                                                     
[    1.117075] Hardware name: Bananapi-R2 Pro (RK3568) DDR4 Board (DT)                                                                          
[    1.117645] pstate: 60400009 (nZCv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)                                                                  
[    1.118278] pc : alg_test.part.0+0x128/0x3e8                                                                                                 
[    1.118679] lr : alg_test.part.0+0x128/0x3e8                                                                                                 
[    1.119081] sp : ffffffc08335bd40                                                                                                            
[    1.119386] x29: ffffffc08335bd40 x28: 0000000000000000 x27: 0000000000000000                                                                
[    1.120046] x26: 00000000ffffffff x25: 0000000000000400 x24: ffffff8001b5b880                                                                
[    1.120705] x23: ffffff800169d880 x22: ffffff8001b5b800 x21: 00000000ffffffea                                                                
[    1.121364] x20: 000000000000038f x19: ffffffc08105fe58 x18: 0000000000000001                                                                
[    1.122022] x17: 0000000064f427b1 x16: 00000000b317d6ec x15: ffffff800169dcc0                                                                
[    1.122681] x14: 0000000000000000 x13: ffffffc081edbd26 x12: ffffffc081edbd22                                                                
[    1.123338] x11: 00000000ffffdfff x10: 00000000ffffdfff x9 : ffffffc080196534                                                                
[    1.123996] x8 : 000000000002ffe8 x7 : c0000000ffffdfff x6 : 0000000000017ff4                                                                
[    1.124652] x5 : 00000000000affa8 x4 : 0000000000000000 x3 : 0000000000000000                                                                
[    1.125309] x2 : ffffffc081c0b5c0 x1 : 0000000000000000 x0 : 0000000000000000                                                                
[    1.125966] Call trace:                                                                                                                      
[    1.126197]  alg_test.part.0+0x128/0x3e8                                                                                                     
[    1.126569]  alg_test+0x28/0x78                                                                                                              
[    1.126871]  cryptomgr_test+0x2c/0x50                                                                                                        
[    1.127217]  kthread+0x124/0x138                                                                                                             
[    1.127529]  ret_from_fork+0x10/0x20                                                                                                         
[    1.127870] ---[ end trace 0000000000000000 ]---                                                                                             
[    1.453153] alg: ahash: rk2-sha256 test failed (wrong result) on test vector "random: psize=8096 ksize=0", cfg="random: may_sleep use_digest"
[    1.454802] alg: self-tests for sha256 using rk2-sha256 failed (rc=-22)                                                                      
[    1.454814] ------------[ cut here ]------------                                                                                             
[    1.455865] alg: self-tests for sha256 using rk2-sha256 failed (rc=-22)                                                                      
[    1.455908] WARNING: CPU: 1 PID: 109 at crypto/testmgr.c:5936 alg_test.part.0+0x128/0x3e8                                                    
[    1.457268] Modules linked in:                                                                                                               
[    1.457564] CPU: 1 PID: 109 Comm: cryptomgr_test Tainted: G        W          6.6.0-rc4-bpi-r2pro-r2pro #7                                   
[    1.458438] Hardware name: Bananapi-R2 Pro (RK3568) DDR4 Board (DT)                                                                          
[    1.459007] pstate: 60400009 (nZCv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)                                                                  
[    1.459640] pc : alg_test.part.0+0x128/0x3e8                                                                                                 
[    1.460043] lr : alg_test.part.0+0x128/0x3e8                                                                                                 
[    1.460444] sp : ffffffc08261bd40                                                                                                            
[    1.460752] x29: ffffffc08261bd40 x28: 0000000000000000 x27: 0000000000000000                                                                
[    1.461413] x26: 00000000ffffffff x25: 0000000000000400 x24: ffffff8001b5b480                                                                
[    1.462072] x23: ffffff800140bb00 x22: ffffff8001b5b400 x21: 00000000ffffffea                                                                
[    1.462730] x20: 000000000000038f x19: ffffffc08105fe58 x18: 0000000000000001                                                                
[    1.463388] x17: 00000000080c04b8 x16: 00000000a50a5975 x15: ffffff800140bf40                                                                
[    1.464047] x14: 0000000000000000 x13: ffffffc081edc4bc x12: ffffffc081edc4b8                                                                
[    1.464704] x11: 00000000ffffdfff x10: 00000000ffffdfff x9 : ffffffc080196534                                                                
[    1.465361] x8 : 000000000002ffe8 x7 : c0000000ffffdfff x6 : 0000000000017ff4                                                                
[    1.466018] x5 : 00000000000affa8 x4 : 0000000000000000 x3 : 0000000000000000                                                                
[    1.466676] x2 : ffffffc081c0b5c0 x1 : 0000000000000000 x0 : 0000000000000000                                                                
[    1.467337] Call trace:                                                                                                                      
[    1.467570]  alg_test.part.0+0x128/0x3e8                                                                                                     
[    1.467943]  alg_test+0x28/0x78                                                                                                              
[    1.468245]  cryptomgr_test+0x2c/0x50                                                                                                        
[    1.468592]  kthread+0x124/0x138                                                                                                             
[    1.468906]  ret_from_fork+0x10/0x20                                                                                                         
[    1.469246] ---[ end trace 0000000000000000 ]---                                                                                             
[    1.704564] alg: skcipher: skipping comparison tests for xts-aes-ce because xts(ecb(aes-generic)) is unavailable                             
[    1.705600] rk3588-crypto fe380000.crypto: Fallback for xts-aes-rk2 is xts-aes-ce                                                            
[    1.736434] alg: skcipher: skipping comparison tests for xts-aes-rk2 because xts(ecb(aes-generic)) is unavailable                            
[    1.846326] rk3588-crypto fe380000.crypto: Fallback for cbc-aes-rk2 is cbc-aes-ce                                                            
[    2.030238] rk3588-crypto fe380000.crypto: Fallback for ecb-aes-rk2 is ecb-aes-ce                                                            
[    2.599763] alg: hash: skipping comparison tests for xcbc-aes-ce because xcbc(aes-generic) is unavailable                                    
[    2.724554] alg: skcipher: skipping comparison tests for essiv-cbc-aes-sha256-ce because essiv(cbc(aes-generic),sha256-generic) is unavailabe
[    2.746481] alg: skcipher: skipping comparison tests for cts-cbc-aes-ce because cts(cbc(aes-generic)) is unavailable                         
[    2.808935] alg: skcipher: skipping comparison tests for xctr-aes-ce because xctr(aes-generic) is unavailable

over wan-port (6.6.0-rc4-bpi-r2pro-r2pro) i get this result:

root@bpi-r2pro:~# iperf3 -c 192.168.0.21 --bidir                                                                                
Connecting to host 192.168.0.21, port 5201
[  5] local 192.168.0.19 port 37764 connected to 192.168.0.21 port 5201                                                                         
[  7] local 192.168.0.19 port 37776 connected to 192.168.0.21 port 5201                                                                         
[ ID][Role] Interval           Transfer     Bitrate         Retr  Cwnd                                                                          
[  5][TX-C]   0.00-1.00   sec  96.8 MBytes   812 Mbits/sec    7    379 KBytes                                                                   
[  7][RX-C]   0.00-1.00   sec   112 MBytes   939 Mbits/sec                                                                                      
[  5][TX-C]   1.00-2.00   sec   112 MBytes   936 Mbits/sec    0    478 KBytes                                                                   
[  7][RX-C]   1.00-2.00   sec   112 MBytes   938 Mbits/sec                                                                                      
[  5][TX-C]   2.00-3.00   sec   112 MBytes   938 Mbits/sec    0    488 KBytes                                                                   
[  7][RX-C]   2.00-3.00   sec   112 MBytes   940 Mbits/sec                                                                                      
[  5][TX-C]   3.00-4.00   sec   112 MBytes   936 Mbits/sec    0    496 KBytes                                                                   
[  7][RX-C]   3.00-4.00   sec   112 MBytes   938 Mbits/sec                                                                                      
[  5][TX-C]   4.00-5.00   sec   111 MBytes   931 Mbits/sec    0    503 KBytes                                                                   
[  7][RX-C]   4.00-5.00   sec   112 MBytes   939 Mbits/sec                                                                                      
[  5][TX-C]   5.00-6.00   sec   111 MBytes   928 Mbits/sec    0    509 KBytes                                                                   
[  7][RX-C]   5.00-6.00   sec   112 MBytes   939 Mbits/sec                                                                                      
[  5][TX-C]   6.00-7.00   sec   111 MBytes   935 Mbits/sec    0    509 KBytes                                                                   
[  7][RX-C]   6.00-7.00   sec   112 MBytes   939 Mbits/sec                                                                                      
[  5][TX-C]   7.00-8.00   sec   111 MBytes   934 Mbits/sec    0    509 KBytes                                                                   
[  7][RX-C]   7.00-8.00   sec   112 MBytes   939 Mbits/sec                                                                                      
[  5][TX-C]   8.00-9.00   sec   111 MBytes   927 Mbits/sec    0    509 KBytes                                                                   
[  7][RX-C]   8.00-9.00   sec   112 MBytes   938 Mbits/sec                                                                                      
[  5][TX-C]   9.00-10.00  sec   112 MBytes   937 Mbits/sec    0    509 KBytes                                                                   
[  7][RX-C]   9.00-10.00  sec   112 MBytes   938 Mbits/sec                                                                                      
- - - - - - - - - - - - - - - - - - - - - - - - -                                                                                               
[ ID][Role] Interval           Transfer     Bitrate         Retr                                                                                
[  5][TX-C]   0.00-10.00  sec  1.07 GBytes   921 Mbits/sec    7             sender                                                              
[  5][TX-C]   0.00-10.04  sec  1.07 GBytes   916 Mbits/sec                  receiver                                                            
[  7][RX-C]   0.00-10.00  sec  1.10 GBytes   941 Mbits/sec    0             sender                                                              
[  7][RX-C]   0.00-10.04  sec  1.09 GBytes   935 Mbits/sec                  receiver                                                            
                                                                                                                                                
iperf Done.