Is it possible to have the crypto extensions working?

Done.

Compiled and tested with importconfig. Works with openssl speed -evp aes-256-cbc -elapsed -engine afalg (no engine specified required after modifying openssl conf).

Still need testing with openvpn/openssh if possible,

Tried to test openvpn:

time openvpn --engine afalg --test-crypto --secret /tmp/secret --verb 0 --tun-mtu 20000 --cipher aes-256-cbc

Got cipher error on 1st run:

Mon Mar 1 19:39:07 2021 cipher_ctx_update: EVP_CipherUpdate() failed

dmesg:

[ 2278.586698] ------------[ cut here ]------------
[ 2278.586737] WARNING: CPU: 2 PID: 4612 at lib/refcount.c:28 refcount_warn_saturate+0x13c/0x174
[ 2278.586742] refcount_t: underflow; use-after-free.
[ 2278.586747] Modules linked in: algif_skcipher af_alg i2c_dev lima gpu_sched spi_mt65xx pwm_mediatek mtk_pmic_keys [last unloaded: cryptodev]
[ 2278.586798] CPU: 2 PID: 4612 Comm: openvpn Tainted: G           O      5.10.18-bpi-r2-main #1
[ 2278.586803] Hardware name: Mediatek Cortex-A7 (Device Tree)
[ 2278.586809] Backtrace:  
[ 2278.586827] [<c0dc9cf8>] (dump_backtrace) from [<c0dca0a4>] (show_stack+0x20/0x24)
[ 2278.586837]  r7:0000001c r6:600d0013 r5:00000000 r4:c14ef938
[ 2278.586850] [<c0dca084>] (show_stack) from [<c0dcdd48>] (dump_stack+0xcc/0xe0)
[ 2278.586862] [<c0dcdc7c>] (dump_stack) from [<c0126840>] (__warn+0xfc/0x114)
[ 2278.586871]  r7:0000001c r6:c057a580 r5:00000009 r4:c1181000
[ 2278.586881] [<c0126744>] (__warn) from [<c0dca7b4>] (warn_slowpath_fmt+0xa8/0xc4)
[ 2278.586890]  r7:0000001c r6:c1181000 r5:c118103c r4:d3640000
[ 2278.586902] [<c0dca710>] (warn_slowpath_fmt) from [<c057a580>] (refcount_warn_saturate+0x13c/0x174)
[ 2278.586913]  r9:c75d61c0 r8:00000000 r7:000000f6 r6:d3640000 r5:0051d3f0 r4:c5b7fa80
[ 2278.586927] [<c057a444>] (refcount_warn_saturate) from [<c037dfa4>] (sys_io_submit+0x6d4/0x8d4)
[ 2278.586939] [<c037d8d0>] (sys_io_submit) from [<c0100060>] (ret_fast_syscall+0x0/0x54)
[ 2278.586945] Exception stack(0xd3641fa8 to 0xd3641ff0)
[ 2278.586955] 1fa0:                   00000000 00000000 b6fc7000 00000001 bed958e4 00000020
[ 2278.586965] 1fc0: 00000000 00000000 bed958e4 000000f6 b6f9a000 bed958e8 00000000 bed958f4
[ 2278.586972] 1fe0: bed958c0 bed958b0 b6f88698 b6c08e30
[ 2278.586982]  r10:000000f6 r9:d3640000 r8:c0100264 r7:000000f6 r6:bed958e4 r5:00000000
[ 2278.586988]  r4:00000000
[ 2278.586994] ---[ end trace 54f7a3c195c24c86 ]---

and kernel panic on 2nd run:

serial output:

r2-gentoo login: af_algaf_alg[ 2357.439515] Internal error: Oops: 5 [#1] SMP ARM                                                                                                                                    
[ 2357.444256] Modules linked in: algif_skcipher afdev lima gpu_sched spi_mt65xx pwm_mediatek mtk_pmic_keys [last unloaded: af_alg]                                                                                 
[ 2357.456905] CPU: 2 PID: 4629 Comm: openvpn Tainted: G        W  O      5.10.18-bpi-r2-main #1                                                                                                                    
[ 2357.465595] Hardware name: Mediatek Cortex-A7 (Device Tree)                                                                                                                                                      
[ 2357.471271] PC is at release_sock+0x3c/0x90                                                                                                                                                                      
[ 2357.475524] LR is at release_sock+0x24/0x90                                                                                                                                                                      
[ 2357.479779] pc : [<c0a7f440>]    lr : [<c0a7f428>]    psr: 600d0013                                                                                                                                              
[ 2357.486148] sp : d54cdd80  ip : d54cdd80  fp : d54cdd94                                                                                                                                                          
[ 2357.491456] r10: c47ad6c0  r9 : d54cc000  r8 : 00000000                                                                                                                                                          
[ 2357.496765] r7 : 00000030  r6 : c5f66c00  r5 : c5f66c70  r4 : c5f66c00                                                                                                                                           
[ 2357.503390] r3 : 00000000  r2 : 00000007  r1 : 00000007  r0 : c5f66c70                                                                                                                                           
[ 2357.510018] Flags: nZCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment none                                                                                                                                    
[ 2357.517268] Control: 10c5387d  Table: 954bc06a  DAC: 00000051                                                                                                                                                    
[ 2357.523116] Process openvpn (pid: 4629, stack limit = 0x777cd319)                                                                                                                                                
[ 2357.529314] Stack: (0xd54cdd80 to 0xd54ce000)                                                                                                                                                                    
[ 2357.533748] dd80: 00000000 c5f65000 d54cddb4 d54cdd98 bf04208c c0a7f410 c81aea80 d54cdf40                                                                                                                        
[ 2357.542059] dda0: 00000010 00000030 d54cddd4 d54cddb8 bf042300 bf04200c d54cdf40 00000010                                                                                                                        
[ 2357.550373] ddc0: c81aea80 00000030 d54cde34 d54cddd8 c0a787c0 bf0422e8 00000000 00000051                                                                                                                        
[ 2357.558685] dde0: d54cde34 d54cddf0 c0a7a40c c0576960 d54cde40 d54cdf48 e06fc5c4 00000000                                                                                                                        
[ 2357.566999] de00: beb028e8 92899668 e06fc5c4 00000000 d54cdf40 d54cc000 c81aea80 00000000                                                                                                                        
[ 2357.575304] de20: 00000000 00000128 d54cdf2c d54cde38 c0a7a4ac c0a78604 00000000 92899668                                                                                                                        
[ 2357.583613] de40: 00000000 00597038 00000010 c15bb6c0 c157dd08 c75d62c0 00000051 beb02998                                                                                                                        
[ 2357.591926] de60: 00000001 00000001 c0378f10 d54cded8 c75d61c0 00000001 d54cc000 beb02998                                                                                                                        
[ 2357.600240] de80: d54cc000 000000f5 d54cdeb4 d54cde98 c037b15c c037ad78 00000001 c75d61c0                                                                                                                        
[ 2357.608554] dea0: 00000001 c037b3d0 d54cdf5c d54cdeb8 c037b3d0 c01a2a9c d54cded8 c036fcf8                                                                                                                        
[ 2357.616868] dec0: 00000000 c2ee7db0 00000000 beb02998 00000000 00000000 00000001 00000008                                                                                                                        
[ 2357.625183] dee0: 00000004 00000000 00000000 d54cdee0 00000000 c033dbd0 d54cdf14 d54cdf08                                                                                                                        
[ 2357.633495] df00: c033dbd0 92899668 d54cc000 beb028f4 00000000 c81aea80 c0100264 d54cc000                                                                                                                        
[ 2357.641806] df20: d54cdf94 d54cdf30 c0a7a928 c0a7a438 00000000 00000000 00000000 fffffff7                                                                                                                        
[ 2357.650116] df40: 00000000 00000000 00000005 00000000 00000010 d54cde44 00000001 c037b2d0                                                                                                                        
[ 2357.658430] df60: c47ad6c0 92899668 00000030 00000000 00000000 92899668 0058d0a0 0059bebc                                                                                                                        
[ 2357.666743] df80: b6f12840 00000128 d54cdfa4 d54cdf98 c0a7a980 c0a7a8d0 00000000 d54cdfa8                                                                                                                        
[ 2357.675053] dfa0: c0100060 c0a7a970 0058d0a0 0059bebc 00000006 beb028f4 00000000 00000000                                                                                                                        
[ 2357.683364] dfc0: 0058d0a0 0059bebc b6f12840 00000128 b6ee3000 beb028e8 00000001 beb028f4                                                                                                                        
[ 2357.691676] dfe0: b6ee3054 beb028a8 b6ed1598 b6b57458 600d0010 00000006 000000000                                                                                                                                
[ 2357.699983] Backtrace:                                                                                                                                                                                           
[ 2357.702512] [<c0a7f404>] (release_sock) from [<bf04208c>] (skcipher_check_key+0x8c/0x90 [algif_skcipher])                                                                                                        
[ 2357.712241]  r5:c5f65000 r4:00000000                                                                                                                                                                             
[ 2357.715889] [<bf042000>] (skcipher_check_key [algif_skcipher]) from [<bf042300>] (skcipher_sendmsg_nokey+0x24/0x54 [algif_skcipher])                                                                             
[ 2357.727991]  r7:00000030 r6:00000010 r5:d54cdf40 r4:c81aea80                                                                                                                                                     
[ 2357.733760] [<bf0422dc>] (skcipher_sendmsg_nokey [algif_skcipher]) from [<c0a787c0>] (____sys_sendmsg+0x1c8/0x260)                                                                                               
[ 2357.744282]  r7:00000030 r6:c81aea80 r5:00000010 r4:d54cdf40                                                                                                                                                     
[ 2357.750053] [<c0a785f8>] (____sys_sendmsg) from [<c0a7a4ac>] (___sys_sendmsg+0x80/0xac)                                                                                                                          
[ 2357.758194]  r10:00000128 r9:00000000 r8:00000000 r7:c81aea80 r6:d54cc000 r5:d54cdf40                                                                                                                            
[ 2357.766147]  r4:00000000                                                                                                                                                                                         
[ 2357.768746] [<c0a7a42c>] (___sys_sendmsg) from [<c0a7a928>] (__sys_sendmsg+0x64/0xa0)                                                                                                                            
[ 2357.776713]  r9:d54cc000 r8:c0100264 r7:c81aea80 r6:00000000 r5:beb028f4 r4:d54cc000                                                                                                                             
[ 2357.784585] [<c0a7a8c4>] (__sys_sendmsg) from [<c0a7a980>] (sys_sendmsg+0x1c/0x20)                                                                                                                               
[ 2357.792284]  r7:00000128 r6:b6f12840 r5:0059bebc r4:0058d0a0                                                                                                                                                     
[ 2357.798050] [<c0a7a964>] (sys_sendmsg) from [<c0100060>] (ret_fast_syscall+0x0/0x54)                                                                                                                             
[ 2357.805921] Exception stack(0xd54cdfa8 to 0xd54cdff0)                                                                                                                                                            
[ 2357.811064] dfa0:                   0058d0a0 0059bebc 00000006 beb028f4 00000000 00000000                                                                                                                        
[ 2357.819378] dfc0: 0058d0a0 0059bebc b6f12840 00000128 b6ee3000 beb028e8 00000001 beb028f4                                                                                                                        
[ 2357.827682] dfe0: b6ee3054 beb028a8 b6ed1598 b6b57458                                                                                                                                                            
[ 2357.832821] Code: 0a000001 e1a00004 ebffffaf e5943020 (e5933048)                                                                                                                                                 
[ 2357.839071] ---[ end trace 54f7a3c195c24c87 ]---                                                                                                                                                                 
[ 2357.843798] Kernel panic - not syncing: Fatal exception in interrupt                                                                                                                                             
[ 2357.843820] CPU0: stopping                                                                                                                                                                                       
[ 2357.843833] CPU: 0 PID: 0 Comm: swapper/0 Tainted: G      D W  O      5.10.18-bpi-r2-main #1                                                                                                                     
[ 2357.843836] Hardware name: Mediatek Cortex-A7 (Device Tree)                                                                                                                                                      
[ 2357.843839] Backtrace:                                                                                                                                                                                           
[ 2357.843862] [<c0dc9cf8>] (dump_backtrace) from [<c0dca0a4>] (show_stack+0x20/0x24)                                                                                                                               
[ 2357.843870]  r7:00000000 r6:60030193 r5:00000000 r4:c14ef938                                                                                                                                                     
[ 2357.843880] [<c0dca084>] (show_stack) from [<c0dcdd48>] (dump_stack+0xcc/0xe0)                                                                                                                                   
[ 2357.843890] [<c0dcdc7c>] (dump_stack) from [<c0110290>] (do_handle_IPI+0x328/0x35c)                                                                                                                              
[ 2357.843897]  r7:00000000 r6:00000000 r5:c1572e40 r4:c15bb640                                                                                                                                                     
[ 2357.843905] [<c010ff68>] (do_handle_IPI) from [<c01102ec>] (ipi_handler+0x28/0x30)                                                                                                                               
[ 2357.843913]  r9:c1401ed0 r8:c2029000 r7:00000000 r6:1e1f2000 r5:c2030e00 r4:00000014                                                                                                                             
[ 2357.843923] [<c01102c4>] (ipi_handler) from [<c01946d4>] (handle_percpu_devid_fasteoi_ipi+0x90/0x160)                                                                                                            
[ 2357.843933] [<c0194644>] (handle_percpu_devid_fasteoi_ipi) from [<c018d76c>] (generic_handle_irq+0x44/0x54)                                                                                                      
[ 2357.843939]  r7:00000000 r6:00000001 r5:00000000 r4:c13a6c24                                                                                                                                                     
[ 2357.843948] [<c018d728>] (generic_handle_irq) from [<c018de7c>] (__handle_domain_irq+0x6c/0xc0)                                                                                                                  
[ 2357.843957] [<c018de10>] (__handle_domain_irq) from [<c010135c>] (gic_handle_irq+0x8c/0xa0)                                                                                                                      
[ 2357.843965]  r9:c1401ed0 r8:e100200c r7:c13a6c30 r6:e1002000 r5:c14ef9f8 r4:c14057cc                                                                                                                             
[ 2357.843973] [<c01012d0>] (gic_handle_irq) from [<c0100b0c>] (__irq_svc+0x6c/0x90)                                                                                                                                
[ 2357.843977] Exception stack(0xc1401ed0 to 0xc1401f18)                                                                                                                                                            
[ 2357.843983] 1ec0:                                     00000000 01779ccc df59a304 c011ee40                                                                                                                        
[ 2357.843991] 1ee0: c1573ba0 00000000 c1404f10 c1404f58 c1572634 c114496c e07fcd00 c1401f2c                                                                                                                        
[ 2357.843997] 1f00: c1401f30 c1401f20 c01096e4 c01096e8 60030013 ffffffff                                                                                                                                          
[ 2357.844005]  r9:c1400000 r8:c1572634 r7:c1401f04 r6:ffffffff r5:60030013 r4:c01096e8                                                                                                                             
[ 2357.844018] [<c01096a0>] (arch_cpu_idle) from [<c0de1148>] (default_idle_call+0x48/0x10c)                                                                                                                        
[ 2357.844027] [<c0de1100>] (default_idle_call) from [<c015f500>] (do_idle+0xe4/0x150)                                                                                                                              
[ 2357.844032]  r5:00000000 r4:c1400000                                                                                                                                                                             
[ 2357.844039] [<c015f41c>] (do_idle) from [<c015f83c>] (cpu_startup_entry+0x28/0x2c)                                                                                                                               
[ 2357.844047]  r9:c1357a54 r8:00000000 r7:c1357a54 r6:c1404ec0 r5:c1400000 r4:000000d8                                                                                                                             
[ 2357.844055] [<c015f814>] (cpu_startup_entry) from [<c0dda448>] (rest_init+0xbc/0xc4)                                                                                                                             
[ 2357.844067] [<c0dda38c>] (rest_init) from [<c1300ad8>] (arch_call_rest_init+0x18/0x1c)                                                                                                                           
[ 2357.844072]  r5:c1400000 r4:c15bb040                                                                                                                                                                             
[ 2357.844081] [<c1300ac0>] (arch_call_rest_init) from [<c1301078>] (start_kernel+0x524/0x55c)                                                                                                                      
[ 2357.844090] [<c1300b54>] (start_kernel) from [<00000000>] (0x0)                                                                                                                                                  
[ 2357.844096] CPU3: stopping                                                                                                                                                                                       
[ 2357.844104] CPU: 3 PID: 4536 Comm: emerge Tainted: G      D W  O      5.10.18-bpi-r2-main #1                                                                                                                     
[ 2357.844107] Hardware name: Mediatek Cortex-A7 (Device Tree)                                                                                                                                                      
[ 2357.844109] Backtrace:                                                                                                                                                                                           
[ 2357.844121] [<c0dc9cf8>] (dump_backtrace) from [<c0dca0a4>] (show_stack+0x20/0x24)                                                                                                                               
[ 2357.844128]  r7:00000000 r6:60030193 r5:00000000 r4:c14ef938                                                                                                                                                     
[ 2357.844136] [<c0dca084>] (show_stack) from [<c0dcdd48>] (dump_stack+0xcc/0xe0)                                                                                                                                   
[ 2357.844145] [<c0dcdc7c>] (dump_stack) from [<c0110290>] (do_handle_IPI+0x328/0x35c)                                                                                                                              
[ 2357.844151]  r7:00000000 r6:00000003 r5:c1572e40 r4:c15bb640                                                                                                                                                     
[ 2357.844159] [<c010ff68>] (do_handle_IPI) from [<c01102ec>] (ipi_handler+0x28/0x30)                                                                                                                               
[ 2357.844167]  r9:ca16dfb0 r8:c2029000 r7:00000000 r6:1e22e000 r5:c2030e00 r4:00000014                                                                                                                             
[ 2357.844175] [<c01102c4>] (ipi_handler) from [<c01946d4>] (handle_percpu_devid_fasteoi_ipi+0x90/0x160)                                                                                                            
[ 2357.844184] [<c0194644>] (handle_percpu_devid_fasteoi_ipi) from [<c018d76c>] (generic_handle_irq+0x44/0x54)                                                                                                      
[ 2357.844190]  r7:00000000 r6:00000001 r5:00000000 r4:c13a6c24                                                                                                                                                     
[ 2357.844200] [<c018d728>] (generic_handle_irq) from [<c018de7c>] (__handle_domain_irq+0x6c/0xc0)                                                                                                                  
[ 2357.844208] [<c018de10>] (__handle_domain_irq) from [<c010135c>] (gic_handle_irq+0x8c/0xa0)                                                                                                                      
[ 2357.844216]  r9:ca16dfb0 r8:e100200c r7:c13a6c30 r6:e1002000 r5:c14ef9f8 r4:c14057cc                                                                                                                             
[ 2357.844223] [<c01012d0>] (gic_handle_irq) from [<c0100eb4>] (__irq_usr+0x54/0x80)                                                                                                                                
[ 2357.844227] Exception stack(0xca16dfb0 to 0xca16dff8)                                                                                                                                                            
[ 2357.844232] dfa0:                                     b6569e60 b660acc4 00000040 0000003f                                                                                                                        
[ 2357.844240] dfc0: 00000030 b660aca0 7e486a30 00517f64 b660aca0 b6f22000 7e486a30 00000040                                                                                                                        
[ 2357.844246] dfe0: b6f30798 beacb800 00001e2c b6cd8180 80030010 ffffffff                                                                                                                                          
[ 2357.844254]  r9:b6f22000 r8:10c5387d r7:10c5387d r6:ffffffff r5:80030010 r4:b6cd8180                                                                                                                             
[ 2357.844259] CPU1: stopping                                                                                                                                                                                       
[ 2357.844266] CPU: 1 PID: 0 Comm: swapper/1 Tainted: G      D W  O      5.10.18-bpi-r2-main #1                                                                                                                     
[ 2357.844269] Hardware name: Mediatek Cortex-A7 (Device Tree)                                                                                                                                                      
[ 2357.844271] Backtrace:                                                                                                                                                                                           
[ 2357.844283] [<c0dc9cf8>] (dump_backtrace) from [<c0dca0a4>] (show_stack+0x20/0x24)                                                                                                                               
[ 2357.844290]  r7:00000000 r6:60030193 r5:00000000 r4:c14ef938                                                                                                                                                     
[ 2357.844298] [<c0dca084>] (show_stack) from [<c0dcdd48>] (dump_stack+0xcc/0xe0)                                                                                                                                   
[ 2357.844306] [<c0dcdc7c>] (dump_stack) from [<c0110290>] (do_handle_IPI+0x328/0x35c)                                                                                                                              
[ 2357.844313]  r7:00000000 r6:00000001 r5:c1572e40 r4:c15bb640                                                                                                                                                     
[ 2357.844320] [<c010ff68>] (do_handle_IPI) from [<c01102ec>] (ipi_handler+0x28/0x30)                                                                                                                               
[ 2357.844328]  r9:c2171f30 r8:c2029000 r7:00000000 r6:1e206000 r5:c2030e00 r4:00000014    357.844376]  r9:c2171f30 r8:e100200c r7:c13a6c30 r6:e1002000 r5:c14ef9f8 r4:c14057cc                                                                                                                             
[ 2357.844383] [<c01012d0>] (gic_handle_irq) from [<c0100b0c>] (__irq_svc+0x6c/0x90)                                                                                                                                
[ 2357.844387] Exception stack(0xc2171f30 to 0xc2171f78)                                                                                                                                                            
[ 2357.844393] 1f20:                                     00000000 00d5c648 df5ae304 c011ee40                                                                                                                        
[ 2357.844400] 1f40: c1573ba0 00000001 c1404f10 c1404f58 c1572634 c114496c 00000000 c2171f8c                                                                                                                        
[ 2357.844407] 1f60: c2171f90 c2171f80 c01096e4 c01096e8 60030013 ffffffff                                                                                                                                          
[ 2357.844414]  r9:c2170000 r8:c1572634 r7:c2171f64 r6:ffffffff r5:60030013 r4:c01096e8                                                                                                                             
[ 2357.844424] [<c01096a0>] (arch_cpu_idle) from [<c0de1148>] (default_idle_call+0x48/0x10c)                                                                                                                        
[ 2357.844433] [<c0de1100>] (default_idle_call) from [<c015f500>] (do_idle+0xe4/0x150)                                                                                                                              
[ 2357.844437]  r5:00000001 r4:c2170000                                                                                                                                                                             
[ 2357.844444] [<c015f41c>] (do_idle) from [<c015f83c>] (cpu_startup_entry+0x28/0x2c)                                                                                                                               
[ 2357.844452]  r9:410fc073 r8:8000406a r7:c15bb650 r6:10c0387d r5:00000001 r4:00000091                                                                                                                             
[ 2357.844460] [<c015f814>] (cpu_startup_entry) from [<c0110bcc>] (secondary_start_kernel+0x170/0x194)                                                                                                              
[ 2357.844468] [<c0110a5c>] (secondary_start_kernel) from [<801017cc>] (0x801017cc)                                                                                                                                 
[ 2357.844472]  r5:00000051 r4:8216806a                                                                                                                                                                             
[ 2358.971480] Rebooting in 3 seconds..357.844337] [<c01102c4>] (ipi_handler) from [<c01946d4>] (handle_percpu_devid_fasteoi_ipi+0x90/0x160)                                                                                                            
[ 2357.844345] [<c0194644>] (handle_percpu_devid_fasteoi_ipi) from [<c018d76c>] (generic_handle_irq+0x44/0x54)                                                                                                      
[ 2357.844351]  r7:00000000 r6:00000001 r5:00000000 r4:c13a6c24                                                                                                                                                     
[ 2357.844360] [<c018d728>] (generic_handle_irq) from [<c018de7c>] (__handle_domain_irq+0x6c/0xc0)                                                                                                                  
[ 2357.844368] [<c018de10>] (__handle_domain_irq) from [<c010135c>] (gic_handle_irq+0x8c/0xa0)                                                                                                                      
[ 2357.844376]  r9:c2171f30 r8:e100200c r7:c13a6c30 r6:e1002000 r5:c14ef9f8 r4:c14057cc                                                                                                                             
[ 2357.844383] [<c01012d0>] (gic_handle_irq) from [<c0100b0c>] (__irq_svc+0x6c/0x90)                                                                                                                                
[ 2357.844387] Exception stack(0xc2171f30 to 0xc2171f78)                                                                                                                                                            
[ 2357.844393] 1f20:                                     00000000 00d5c648 df5ae304 c011ee40                                                                                                                        
[ 2357.844400] 1f40: c1573ba0 00000001 c1404f10 c1404f58 c1572634 c114496c 00000000 c2171f8c                                                                                                                        
[ 2357.844407] 1f60: c2171f90 c2171f80 c01096e4 c01096e8 60030013 ffffffff                                                                                                                                          
[ 2357.844414]  r9:c2170000 r8:c1572634 r7:c2171f64 r6:ffffffff r5:60030013 r4:c01096e8                                                                                                                             
[ 2357.844424] [<c01096a0>] (arch_cpu_idle) from [<c0de1148>] (default_idle_call+0x48/0x10c)                                                                                                                        
[ 2357.844433] [<c0de1100>] (default_idle_call) from [<c015f500>] (do_idle+0xe4/0x150)                                                                                                                              
[ 2357.844437]  r5:00000001 r4:c2170000                                                                                                                                                                             
[ 2357.844444] [<c015f41c>] (do_idle) from [<c015f83c>] (cpu_startup_entry+0x28/0x2c)                                                                                                                               
[ 2357.844452]  r9:410fc073 r8:8000406a r7:c15bb650 r6:10c0387d r5:00000001 r4:00000091                                                                                                                             
[ 2357.844460] [<c015f814>] (cpu_startup_entry) from [<c0110bcc>] (secondary_start_kernel+0x170/0x194)                                                                                                              
[ 2357.844468] [<c0110a5c>] (secondary_start_kernel) from [<801017cc>] (0x801017cc)                                                                                                                                 
[ 2357.844472]  r5:00000051 r4:8216806a                                                                                                                                                                             
[ 2358.971480] Rebooting in 3 seconds..

UPD:

openvpn test success depends on --tun-mtu in works great with --tun-mtu 8192, and fails with --tun-mtu 16384

Some openvpn perfomance tests:

setup:

server:

AF_ALG:

openvpn --engine afalg --dev tun --proto udp --port 11000 --secret ./static.key --tun-mtu 8192 --ifconfig 192.168.222.11 192.168.222.10 --cipher aes-256-cbc

SW:

openvpn --dev tun --proto udp --port 11000 --secret ./static.key --tun-mtu 8192 --ifconfig 192.168.222.11 192.168.222.10 --cipher aes-256-cbc

client:

openvpn --dev tun --proto udp --port 11000 --secret ./static.key --tun-mtu 8192 --ifconfig 192.168.222.10 192.168.222.11 --cipher aes-256-cbc --remote <server_ip>

results:

upload/download

network w/o openvpn - using wifi:

-----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------
Accepted connection from 10.0.1.150, port 59184
[  5] local 10.0.1.2 port 5201 connected to 10.0.1.150 port 59186
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec  10.8 MBytes  90.7 Mbits/sec                  
[  5]   1.00-2.00   sec  13.3 MBytes   112 Mbits/sec                  
[  5]   2.00-3.00   sec  11.7 MBytes  98.4 Mbits/sec                  
[  5]   3.00-4.00   sec  10.8 MBytes  90.2 Mbits/sec                  
[  5]   4.00-5.00   sec  9.66 MBytes  81.1 Mbits/sec                  
[  5]   5.00-6.00   sec  10.6 MBytes  88.8 Mbits/sec                  
[  5]   6.00-7.00   sec  10.4 MBytes  87.6 Mbits/sec                  
[  5]   7.00-8.00   sec  9.96 MBytes  83.5 Mbits/sec                  
[  5]   8.00-9.00   sec  9.92 MBytes  83.2 Mbits/sec                  
[  5]   9.00-10.00  sec  10.2 MBytes  85.8 Mbits/sec                  
[  5]  10.00-10.02  sec   214 KBytes  78.6 Mbits/sec                  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-10.02  sec   108 MBytes  90.1 Mbits/sec                  receiver
-----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------
Accepted connection from 10.0.1.150, port 59196
[  5] local 10.0.1.2 port 5201 connected to 10.0.1.150 port 59198
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  31.2 MBytes   262 Mbits/sec   22    498 KBytes       
[  5]   1.00-2.00   sec  18.8 MBytes   157 Mbits/sec    0    564 KBytes       
[  5]   2.00-3.00   sec  23.8 MBytes   199 Mbits/sec    0    609 KBytes       
[  5]   3.00-4.00   sec  22.5 MBytes   189 Mbits/sec    0    641 KBytes       
[  5]   4.00-5.00   sec  20.0 MBytes   168 Mbits/sec    0    659 KBytes       
[  5]   5.00-6.00   sec  21.2 MBytes   178 Mbits/sec    0    667 KBytes       
[  5]   6.00-7.00   sec  21.2 MBytes   178 Mbits/sec    4    499 KBytes       
[  5]   7.00-8.00   sec  18.8 MBytes   157 Mbits/sec    0    533 KBytes       
[  5]   8.00-9.00   sec  21.2 MBytes   178 Mbits/sec    0    553 KBytes       
[  5]   9.00-10.00  sec  21.2 MBytes   178 Mbits/sec    0    567 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec   220 MBytes   184 Mbits/sec   26             sender  

AF_ALG:

-----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------
Accepted connection from 192.168.222.10, port 52138
[  5] local 192.168.222.11 port 5201 connected to 192.168.222.10 port 52140
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec  2.74 MBytes  23.0 Mbits/sec                  
[  5]   1.00-2.00   sec  2.99 MBytes  25.1 Mbits/sec                  
[  5]   2.00-3.00   sec  3.22 MBytes  27.0 Mbits/sec                  
[  5]   3.00-4.00   sec  3.09 MBytes  25.9 Mbits/sec                  
[  5]   4.00-5.00   sec  2.95 MBytes  24.7 Mbits/sec                  
[  5]   5.00-6.00   sec  3.27 MBytes  27.4 Mbits/sec                  
[  5]   6.00-7.00   sec  3.20 MBytes  26.8 Mbits/sec                  
[  5]   7.00-8.00   sec  3.22 MBytes  27.0 Mbits/sec                  
[  5]   8.00-9.00   sec  3.24 MBytes  27.1 Mbits/sec                  
[  5]   9.00-10.00  sec  3.13 MBytes  26.3 Mbits/sec                  
[  5]  10.00-10.00  sec  5.23 KBytes  14.1 Mbits/sec                  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-10.00  sec  31.0 MBytes  26.0 Mbits/sec                  receiver
-----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------
Accepted connection from 192.168.222.10, port 52152
[  5] local 192.168.222.11 port 5201 connected to 192.168.222.10 port 52154
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  13.8 MBytes   115 Mbits/sec    2   92.8 KBytes       
[  5]   1.00-2.00   sec  2.50 MBytes  21.0 Mbits/sec    4   91.5 KBytes       
[  5]   2.00-3.00   sec  2.50 MBytes  21.0 Mbits/sec    4   79.7 KBytes       
[  5]   3.00-4.00   sec  2.50 MBytes  21.0 Mbits/sec    2   71.9 KBytes       
[  5]   4.00-5.00   sec  2.50 MBytes  21.0 Mbits/sec    0   92.8 KBytes       
[  5]   5.00-6.00   sec  2.50 MBytes  21.0 Mbits/sec    2   82.3 KBytes       
[  5]   6.00-7.00   sec  2.50 MBytes  21.0 Mbits/sec    4   73.2 KBytes       
[  5]   7.00-8.00   sec  2.50 MBytes  21.0 Mbits/sec    0   94.1 KBytes       
[  5]   8.00-9.00   sec  2.50 MBytes  21.0 Mbits/sec    2   84.9 KBytes       
[  5]   9.00-10.00  sec  2.50 MBytes  21.0 Mbits/sec    2   74.5 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  36.2 MBytes  30.4 Mbits/sec   22             sender

SW:

-----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------
Accepted connection from 192.168.222.10, port 52204
[  5] local 192.168.222.11 port 5201 connected to 192.168.222.10 port 52206
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec  4.52 MBytes  37.9 Mbits/sec                  
[  5]   1.00-2.00   sec  5.03 MBytes  42.2 Mbits/sec                  
[  5]   2.00-3.00   sec  6.01 MBytes  50.4 Mbits/sec                  
[  5]   3.00-4.00   sec  5.22 MBytes  43.8 Mbits/sec                  
[  5]   4.00-5.00   sec  5.83 MBytes  48.9 Mbits/sec                  
[  5]   5.00-6.00   sec  4.56 MBytes  38.2 Mbits/sec                  
[  5]   6.00-7.00   sec  4.36 MBytes  36.6 Mbits/sec                  
[  5]   7.00-8.00   sec  3.47 MBytes  29.1 Mbits/sec                  
[  5]   8.00-9.00   sec  4.66 MBytes  39.1 Mbits/sec                  
[  5]   9.00-10.00  sec  4.81 MBytes  40.3 Mbits/sec                  
[  5]  10.00-10.00  sec  24.8 KBytes  46.5 Mbits/sec                  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-10.00  sec  48.5 MBytes  40.7 Mbits/sec                  receiver
-----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------
Accepted connection from 192.168.222.10, port 52212
[  5] local 192.168.222.11 port 5201 connected to 192.168.222.10 port 52214
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  15.0 MBytes   126 Mbits/sec   11   95.4 KBytes       
[  5]   1.00-2.00   sec  3.75 MBytes  31.5 Mbits/sec    1   98.0 KBytes       
[  5]   2.00-3.00   sec  5.00 MBytes  41.9 Mbits/sec    6   67.9 KBytes       
[  5]   3.00-4.00   sec  3.75 MBytes  31.5 Mbits/sec    8   84.9 KBytes       
[  5]   4.00-5.00   sec  3.75 MBytes  31.5 Mbits/sec   15   88.9 KBytes       
[  5]   5.00-6.00   sec  3.75 MBytes  31.5 Mbits/sec    4   92.8 KBytes       
[  5]   6.00-7.00   sec  5.00 MBytes  41.9 Mbits/sec    1   95.4 KBytes       
[  5]   7.00-8.00   sec  3.75 MBytes  31.5 Mbits/sec   20   52.3 KBytes       
[  5]   8.00-9.00   sec  3.75 MBytes  31.5 Mbits/sec   19   47.0 KBytes       
[  5]   9.00-10.00  sec  5.00 MBytes  42.0 Mbits/sec    6   67.9 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.01  sec  52.5 MBytes  44.0 Mbits/sec   91             sender

Looks strange, bun in this case HW accelerated openvpn seems to be slower than software-only. Probably it depends on small buffer size (i might be wrong - i think it depends on mtu size)

I couldnā€™t to force openssh to use afalg engine after recompiling with --with-sandbox=no so for now iā€™m unable to benchmark ssh.

P.S. openvpn is clearly uses aes_mtk - itā€™s visible from/proc/interrupts

UPD2:

after openvpn tuning i finnaly got some nicer results :slight_smile:

added --fragment 0 --mssfix 0 to both client and server configs

HW:

Accepted connection from 192.168.222.10, port 53218
[  5] local 192.168.222.11 port 5201 connected to 192.168.222.10 port 53220
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec  9.33 MBytes  78.3 Mbits/sec                  
[  5]   1.00-2.00   sec  10.6 MBytes  88.7 Mbits/sec                  
[  5]   2.00-3.00   sec  9.66 MBytes  81.1 Mbits/sec                  
[  5]   3.00-4.00   sec  11.9 MBytes  99.5 Mbits/sec                  
[  5]   4.00-5.00   sec  11.0 MBytes  92.6 Mbits/sec                  
[  5]   5.00-6.00   sec  11.4 MBytes  95.7 Mbits/sec                  
[  5]   6.00-7.00   sec  10.8 MBytes  90.2 Mbits/sec                  
[  5]   7.00-8.00   sec  11.1 MBytes  93.0 Mbits/sec                  
[  5]   8.00-9.00   sec  10.9 MBytes  91.1 Mbits/sec                  
[  5]   9.00-10.00  sec  10.8 MBytes  90.7 Mbits/sec                  
[  5]  10.00-10.06  sec   707 KBytes  97.7 Mbits/sec                  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-10.06  sec   108 MBytes  90.1 Mbits/sec                  receiver
-----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------
Accepted connection from 192.168.222.10, port 53222
[  5] local 192.168.222.11 port 5201 connected to 192.168.222.10 port 53224
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  20.0 MBytes   168 Mbits/sec    1    294 KBytes       
[  5]   1.00-2.00   sec  8.75 MBytes  73.4 Mbits/sec    0    413 KBytes       
[  5]   2.00-3.00   sec  10.0 MBytes  83.9 Mbits/sec    0    501 KBytes       
[  5]   3.00-4.00   sec  8.75 MBytes  73.4 Mbits/sec    0    572 KBytes       
[  5]   4.00-5.00   sec  8.75 MBytes  73.4 Mbits/sec    4    350 KBytes       
[  5]   5.00-6.00   sec  10.0 MBytes  83.9 Mbits/sec    0    453 KBytes       
[  5]   6.00-7.00   sec  8.75 MBytes  73.4 Mbits/sec    0    533 KBytes       
[  5]   7.00-8.00   sec  10.0 MBytes  83.9 Mbits/sec   10    318 KBytes       
[  5]   8.00-9.00   sec  8.75 MBytes  73.4 Mbits/sec    0    421 KBytes       
[  5]   9.00-10.00  sec  8.75 MBytes  73.4 Mbits/sec    0    509 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec   102 MBytes  85.9 Mbits/sec   15             sender
-----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------

SW:

Accepted connection from 192.168.222.10, port 53132
[  5] local 192.168.222.11 port 5201 connected to 192.168.222.10 port 53134
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec  8.75 MBytes  73.4 Mbits/sec                  
[  5]   1.00-2.00   sec  8.86 MBytes  74.3 Mbits/sec                  
[  5]   2.00-3.00   sec  8.30 MBytes  69.6 Mbits/sec                  
[  5]   3.00-4.00   sec  9.44 MBytes  79.2 Mbits/sec                  
[  5]   4.00-5.00   sec  8.43 MBytes  70.7 Mbits/sec                  
[  5]   5.00-6.00   sec  8.36 MBytes  70.1 Mbits/sec                  
[  5]   6.00-7.00   sec  8.42 MBytes  70.7 Mbits/sec                  
[  5]   7.00-8.00   sec  8.02 MBytes  67.3 Mbits/sec                  
[  5]   8.00-9.00   sec  9.18 MBytes  77.0 Mbits/sec                  
[  5]   9.00-10.00  sec  8.38 MBytes  70.3 Mbits/sec                  
[  5]  10.00-10.03  sec   358 KBytes  85.7 Mbits/sec                  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-10.03  sec  86.5 MBytes  72.3 Mbits/sec                  receiver
-----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------
Accepted connection from 192.168.222.10, port 53140
[  5] local 192.168.222.11 port 5201 connected to 192.168.222.10 port 53142
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  18.8 MBytes   157 Mbits/sec    4    278 KBytes       
[  5]   1.00-2.00   sec  10.0 MBytes  83.9 Mbits/sec    0    397 KBytes       
[  5]   2.00-3.00   sec  8.75 MBytes  73.4 Mbits/sec    4    270 KBytes       
[  5]   3.00-4.00   sec  10.0 MBytes  83.9 Mbits/sec    0    390 KBytes       
[  5]   4.00-5.00   sec  10.0 MBytes  83.9 Mbits/sec    2    350 KBytes       
[  5]   5.00-6.00   sec  8.75 MBytes  73.4 Mbits/sec    0    445 KBytes       
[  5]   6.00-7.00   sec  8.75 MBytes  73.4 Mbits/sec    9    175 KBytes       
[  5]   7.00-8.00   sec  10.0 MBytes  83.9 Mbits/sec    0    334 KBytes       
[  5]   8.00-9.00   sec  8.75 MBytes  73.4 Mbits/sec   33    231 KBytes       
[  5]   9.00-10.00  sec  8.75 MBytes  73.4 Mbits/sec    4    246 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.01  sec   102 MBytes  85.9 Mbits/sec   56             sender
-----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------

UPD2:

When using --fragment 0 --mssfix 0, the speed is highly depends on --tun-mtu

ā€“tun-mtu 1500:

upload
[  5]   0.00-10.00  sec  35.9 MBytes  30.1 Mbits/sec                  receiver
download
[  5]   0.00-10.00  sec  36.2 MBytes  30.4 Mbits/sec   43             sender

ā€“tun-mtu 16000:

upload
[  5]   0.00-10.11  sec   121 MBytes   101 Mbits/sec                  receiver
download
[  5]   0.00-10.01  sec   144 MBytes   120 Mbits/sec   30             sender

ā€“tun-mtu 16000 - no acceleration:

upload
[  5]   0.00-10.05  sec   112 MBytes  93.4 Mbits/sec                  receiver
download
[  5]   0.00-10.01  sec   126 MBytes   106 Mbits/sec   11             sender

full server/client command:

openvpn --dev tun --proto udp --port 11000 --secret ./static.key --tun-mtu 16000 --ifconfig 192.168.222.11 192.168.222.10 --cipher aes-256-cbc --fragment 0 --mssfix 0

openvpn --dev tun --proto udp --port 11000 --secret ./static.key --tun-mtu 16000 --ifconfig 192.168.222.10 192.168.222.11 --cipher aes-256-cbc --fragment 0 --mssfix 0 --remote 10.0.1.2

UPD3:

testing with --tun-mtu 16384 crashes server with same dmesg as openvpn test above, so --tun-mtu 16000 seems to be working configuration near itā€™s hw limit

setting ā€˜ā€“tun-mtu 32768ā€™ in sw mode didnā€™t gave any speed advantages compairing to --tun-mtu 16000 in sw mode

same test performed using LAN

SW mtu 16000:

vim3 ~ # iperf3 -c 192.168.222.11
Connecting to host 192.168.222.11, port 5201
[  5] local 192.168.222.10 port 47436 connected to 192.168.222.11 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  14.1 MBytes   118 Mbits/sec    0    934 KBytes       
[  5]   1.00-2.00   sec  11.2 MBytes  94.4 Mbits/sec    2    857 KBytes       
[  5]   2.00-3.00   sec  12.5 MBytes   105 Mbits/sec    0    950 KBytes       
[  5]   3.00-4.00   sec  13.8 MBytes   115 Mbits/sec    2    763 KBytes       
[  5]   4.00-5.00   sec  12.5 MBytes   105 Mbits/sec    1    623 KBytes       
[  5]   5.00-6.00   sec  12.5 MBytes   105 Mbits/sec    0    763 KBytes       
[  5]   6.00-7.00   sec  12.5 MBytes   105 Mbits/sec    0    888 KBytes       
[  5]   7.00-8.00   sec  12.5 MBytes   105 Mbits/sec    1    763 KBytes       
[  5]   8.00-9.00   sec  11.2 MBytes  94.4 Mbits/sec    0    888 KBytes       
[  5]   9.00-10.00  sec  12.5 MBytes   105 Mbits/sec    3    716 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec   125 MBytes   105 Mbits/sec    9             sender
[  5]   0.00-10.02  sec   122 MBytes   102 Mbits/sec                  receiver

iperf Done.
vim3 ~ # iperf3 -c 192.168.222.11 -R
Connecting to host 192.168.222.11, port 5201
Reverse mode, remote host 192.168.222.11 is sending
[  5] local 192.168.222.10 port 47440 connected to 192.168.222.11 port 5201
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec  11.1 MBytes  93.1 Mbits/sec                  
[  5]   1.00-2.00   sec  11.5 MBytes  96.7 Mbits/sec                  
[  5]   2.00-3.00   sec  11.6 MBytes  97.3 Mbits/sec                  
[  5]   3.00-4.00   sec  11.6 MBytes  97.0 Mbits/sec                  
[  5]   4.00-5.00   sec  11.6 MBytes  97.6 Mbits/sec                  
[  5]   5.00-6.00   sec  11.7 MBytes  97.7 Mbits/sec                  
[  5]   6.00-7.00   sec  11.6 MBytes  97.6 Mbits/sec                  
[  5]   7.00-8.00   sec  11.7 MBytes  98.1 Mbits/sec                  
[  5]   8.00-9.00   sec  11.7 MBytes  98.1 Mbits/sec                  
[  5]   9.00-10.00  sec  11.6 MBytes  97.6 Mbits/sec                  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.01  sec   126 MBytes   106 Mbits/sec    6             sender
[  5]   0.00-10.00  sec   116 MBytes  97.1 Mbits/sec                  receiver MBytes  98.1 Mbits/sec                  
[  5]   8.00-9.00   sec  11.7 MBytes  98.1 Mbits/sec                  
[  5]   9.00-10.00  sec  11.6 MBytes  97.6 Mbits/sec                  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.01  sec   126 MBytes   106 Mbits/sec    6             sender
[  5]   0.00-10.00  sec   116 MBytes  97.1 Mbits/sec                  receiver

iperf Done.

HW, mtu 16000:

vim3 ~ # iperf3 -c 192.168.222.11
Connecting to host 192.168.222.11, port 5201
[  5] local 192.168.222.10 port 47444 connected to 192.168.222.11 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  17.7 MBytes   148 Mbits/sec    0   1.11 MBytes       
[  5]   1.00-2.00   sec  16.2 MBytes   136 Mbits/sec    0   1.96 MBytes       
[  5]   2.00-3.00   sec  17.5 MBytes   147 Mbits/sec    4   2.48 MBytes       
[  5]   3.00-4.00   sec  17.5 MBytes   147 Mbits/sec    6   1.51 MBytes       
[  5]   4.00-5.00   sec  17.5 MBytes   147 Mbits/sec    0   1.63 MBytes       
[  5]   5.00-6.00   sec  15.0 MBytes   126 Mbits/sec   50   1.25 MBytes       
[  5]   6.00-7.00   sec  16.2 MBytes   136 Mbits/sec    0   1.35 MBytes       
[  5]   7.00-8.00   sec  16.2 MBytes   136 Mbits/sec    0   1.38 MBytes       
[  5]   8.00-9.00   sec  15.0 MBytes   126 Mbits/sec    0   1.48 MBytes       
[  5]   9.00-10.00  sec  15.0 MBytes   126 Mbits/sec    0   1.55 MBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec   164 MBytes   138 Mbits/sec   60             sender
[  5]   0.00-10.00  sec   162 MBytes   136 Mbits/sec                  receiver

iperf Done.
vim3 ~ # iperf3 -c 192.168.222.11 -R
Connecting to host 192.168.222.11, port 5201
Reverse mode, remote host 192.168.222.11 is sending
[  5] local 192.168.222.10 port 47448 connected to 192.168.222.11 port 5201
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec  13.1 MBytes   110 Mbits/sec                  
[  5]   1.00-2.00   sec  13.6 MBytes   114 Mbits/sec                  
[  5]   2.00-3.00   sec  13.6 MBytes   114 Mbits/sec                  
[  5]   3.00-4.00   sec  13.6 MBytes   114 Mbits/sec                  
[  5]   4.00-5.00   sec  13.2 MBytes   110 Mbits/sec                  
[  5]   5.00-6.00   sec  14.0 MBytes   118 Mbits/sec                  
[  5]   6.00-7.00   sec  13.4 MBytes   113 Mbits/sec                  
[  5]   7.00-8.00   sec  13.6 MBytes   114 Mbits/sec                  
[  5]   8.00-9.00   sec  13.7 MBytes   115 Mbits/sec                  
[  5]   9.00-10.00  sec  13.7 MBytes   115 Mbits/sec                  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.01  sec   146 MBytes   123 Mbits/sec    8             sender
[  5]   0.00-10.00  sec   136 MBytes   114 Mbits/sec                  receiver

iperf Done.

In this case HW acceleration definitely gives ~30% boost for upload and ~15% for download.

Raw lan speed 870 to 920 mbits

So for mtu 1500 (or smaller like used in internet,my 1492,or manually reduced against connection issues) there is no benefit,right?

Not exactly: encrypted traffic still goes through LAN/WAN/WiFi with mtu 1492/1500, 16000 is only for virtual tun/tap interface. The only hint i see - the firewall needs to apply mss-fix in case when traffic should be routed to some real interfaces (any home router always do it when uses pppoe/pptp ,and mtu is 1492 for example). It still needs to be tested on some real cases ;).

P.S. Also in case 16000 in not a hw limit, but only some driver bug and it can be increased in some future - it may give some more improvements.

devcrypto testing:

Iā€™ve tried to use cryptodev driver. After compiling kernel module and rebuilding openssl with devcrypto engine, i got:

r2-gentoo ~ # openssl engine -t -c
(devcrypto) /dev/crypto engine
 [AES-128-CBC, AES-192-CBC, AES-256-CBC, AES-128-CTR, AES-192-CTR, AES-256-CTR, AES-128-ECB, AES-192-ECB, AES-256-ECB, MD5, SHA1, SHA224, SHA256, SHA384, SHA512]
     [ available ]
(dynamic) Dynamic engine loading support
     [ unavailable ]
(afalg) AFALG engine support
 [AES-128-CBC, AES-192-CBC, AES-256-CBC]
     [ available ]

quick benchmark showed that if faster than AF_ALG in singlethread and multithread mode:

r2-gentoo ~ # openssl speed -elapsed -evp aes-256-cbc -engine devcrypto
engine "devcrypto" set.
...
The 'numbers' are in 1000s of bytes per second processed.
type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes  16384 bytes
aes-256-cbc        524.01k     2119.77k     8263.25k    30785.19k    83902.46k    86731.43k


r2-gentoo ~ # openssl speed -elapsed -evp aes-256-cbc -engine devcrypto -multi 2
Forked child 0
Forked child 1
...
evp               1037.79k     4108.90k    16176.68k    60337.15k   140787.71k   140170.58k


r2-gentoo ~ # openssl speed -elapsed -evp aes-256-cbc --engine afalg
engine "afalg" set.
...
The 'numbers' are in 1000s of bytes per second processed.
type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes  16384 bytes
aes-256-cbc        486.05k     1948.84k     7123.29k    18607.79k    64984.41k    79779.16k

r2-gentoo ~ # openssl speed -elapsed -evp aes-256-cbc --engine afalg -multi 2
Forked child 0
Forked child 1
engine "afalg" set.
evp                609.21k     2646.38k    10122.07k    34820.10k   119229.10k   142447.96k

but i failed to run openvpn, and self test showed:

openvpn --test-crypto --secret ./static.key --cipher AES-256-CBC --tun-mtu 100 --engine devcrypto  
Tue Mar  2 21:25:04 2021 disabling NCP mode (--ncp-disable) because not in P2MP client or server mode
Tue Mar  2 21:25:04 2021 OpenVPN 2.4.9 armv7a-hardfloat-linux-gnueabi [SSL (OpenSSL)] [LZO] [LZ4] [EPOLL] [MH/PKTINFO] [AEAD] built on Apr 29 2020
Tue Mar  2 21:25:04 2021 library versions: OpenSSL 1.1.1j  16 Feb 2021, LZO 2.10
Tue Mar  2 21:25:04 2021 OpenVPN 2.4.9 armv7a-hardfloat-linux-gnueabi [SSL (OpenSSL)] [LZO] [LZ4] [EPOLL] [MH/PKTINFO] [AEAD] built on Apr 29 2020
Tue Mar  2 21:25:04 2021 Initializing OpenSSL support for engine 'devcrypto'
Tue Mar  2 21:25:04 2021 Entering OpenVPN crypto self-test mode.
Tue Mar  2 21:25:04 2021 TESTING ENCRYPT/DECRYPT of packet length=1
Tue Mar  2 21:25:04 2021 TESTING ENCRYPT/DECRYPT of packet length=2
Tue Mar  2 21:25:04 2021 TESTING ENCRYPT/DECRYPT of packet length=3
Tue Mar  2 21:25:04 2021 TESTING ENCRYPT/DECRYPT of packet length=4
Tue Mar  2 21:25:04 2021 TESTING ENCRYPT/DECRYPT of packet length=5
Tue Mar  2 21:25:04 2021 TESTING ENCRYPT/DECRYPT of packet length=6
Tue Mar  2 21:25:04 2021 TESTING ENCRYPT/DECRYPT of packet length=7
Tue Mar  2 21:25:04 2021 TESTING ENCRYPT/DECRYPT of packet length=8
Tue Mar  2 21:25:04 2021 Authenticate/Decrypt packet error: cipher final failed
Tue Mar  2 21:25:04 2021 SELF TEST FAILED, src.len=8 buf.len=0
Tue Mar  2 21:25:04 2021 Exiting due to fatal error

Taking into account good results with AF_ALG, i think itā€™s a devcrypto issue(?)

BTW maximum working mtu with AF_ALG is 16375

Have you used latest cryptodev?

Yep, i used github version

Maybe file an issue on githubā€¦have nor found solution for this error in openvpn with cryptodev

May be possible but i am not sure the crypto extension is fully working.

Iā€™ve also tried to encrypt hdd, using dm-crypto and hw acceleration. As result i had a fs corruption at random time on writing, when usin mtk-aes, and everything was great when using software only. Iā€™ll provide more details a bit later.

So iā€™ll likely to be agree with @technomoney1 at this time.

(add yes, i realize that EIP97 was 1st of all invented as network security engine, not as hdd encryption :slight_smile: )

I think we need more tests to compare hw and sw results.

@Ryder.Lee, as author of the mtk-aes/hash drivers, may you give some advice how to use/test it property?

Please use upstream insight-secure eip97 driver and you can refer to patch file 501-503 in https://github.com/openwrt/openwrt/tree/master/target/linux/mediatek/patches-5.4

1 Like

Ok, then 1st iā€™ll try these patches.

Thank you.

P.S. @moore, what about 0500-v5.6-crypto-backport-inside-secure.patch? do we need it?

Yes, please also apply this patch which backport up-to-date driver to kernel 5.4.

1 Like

Look fine for me on both 5.4 and 5.10

5.10 doesnā€™t required 0500-v5.6-crypto-backport-inside-secure.patch

Seems to be slower than mtk-aes using both cryptodev and afalg engines.

AF_ALG:

r2-gentoo ~ # time openssl speed -evp aes-128-cbc -elapsed -engine afalg
engine "afalg" set.
You have chosen to measure elapsed time instead of user CPU time.
Doing aes-128-cbc for 3s on 16 size blocks: 28640 aes-128-cbc's in 3.00s
Doing aes-128-cbc for 3s on 64 size blocks: 27269 aes-128-cbc's in 3.00s
Doing aes-128-cbc for 3s on 256 size blocks: 27901 aes-128-cbc's in 3.00s
Doing aes-128-cbc for 3s on 1024 size blocks: 31013 aes-128-cbc's in 3.00s
Doing aes-128-cbc for 3s on 8192 size blocks: 16236 aes-128-cbc's in 3.00s
Doing aes-128-cbc for 3s on 16384 size blocks: 10891 aes-128-cbc's in 3.00s
OpenSSL 1.1.1j  16 Feb 2021
built on: Tue Mar  2 08:26:20 2021 UTC
options:bn(64,32) rc4(char) des(long) aes(partial) idea(int) blowfish(ptr)
compiler: armv7a-hardfloat-linux-gnueabi-gcc -fPIC -pthread -Wa,--noexecstack -Wall -O2 -pipe -march=armv7-a -mfpu=neon-vfpv4 -mfloat-abi=hard -fno-strict-aliasing -Wa,--noexecstack -DOPENSSL_USE_NODELETE -DOPEN
SSL_PIC -DOPENSSL_CPUID_OBJ -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DKECCAK1600_ASM -DAES_ASM -DBSAES_ASM -DGHASH_ASM -DECP_NISTZ256_ASM -DPOLY1305_ASM -DZLIB -DNDEBUG -
DL_ENDIAN  -DOPENSSL_NO_BUF_FREELISTS
The 'numbers' are in 1000s of bytes per second processed.
type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes  16384 bytes
aes-128-cbc        152.75k      581.74k     2380.89k    10585.77k    44335.10k    59479.38k

real    0m18,776s
user    0m0,489s
sys     0m8,253s

devcrypto:

time openssl speed -evp aes-128-cbc -elapsed -engine devcrypto
engine "devcrypto" set.
You have chosen to measure elapsed time instead of user CPU time.
Doing aes-128-cbc for 3s on 16 size blocks: 43778 aes-128-cbc's in 3.00s
Doing aes-128-cbc for 3s on 64 size blocks: 47094 aes-128-cbc's in 3.00s
Doing aes-128-cbc for 3s on 256 size blocks: 48750 aes-128-cbc's in 3.00s
Doing aes-128-cbc for 3s on 1024 size blocks: 44854 aes-128-cbc's in 3.00s
Doing aes-128-cbc for 3s on 8192 size blocks: 25572 aes-128-cbc's in 3.00s
Doing aes-128-cbc for 3s on 16384 size blocks: 19328 aes-128-cbc's in 3.00s
OpenSSL 1.1.1j  16 Feb 2021
built on: Tue Mar  2 08:26:20 2021 UTC
options:bn(64,32) rc4(char) des(long) aes(partial) idea(int) blowfish(ptr)
compiler: armv7a-hardfloat-linux-gnueabi-gcc -fPIC -pthread -Wa,--noexecstack -Wall -O2 -pipe -march=armv7-a -mfpu=neon-vfpv4 -mfloat-abi=hard -fno-strict-aliasing -Wa,--noexecstack -DOPENSSL_USE_NODELETE -DOPEN
SSL_PIC -DOPENSSL_CPUID_OBJ -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DKECCAK1600_ASM -DAES_ASM -DBSAES_ASM -DGHASH_ASM -DECP_NISTZ256_ASM -DPOLY1305_ASM -DZLIB -DNDEBUG -
DL_ENDIAN  -DOPENSSL_NO_BUF_FREELISTS
The 'numbers' are in 1000s of bytes per second processed.
type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes  16384 bytes
aes-128-cbc        233.48k     1004.67k     4160.00k    15310.17k    69828.61k   105556.65k

real    0m18,181s
user    0m0,191s
sys     0m4,961s

But openvpn tests passed (iā€™ve testsd with mtu up-to 65000)

openvpn with mtu 16000, same as aflag/mtk-aes:

server: openvpn --engine devcrypto --dev tun --proto udp --port 11000 --secret ./static.key --tun-mtu 16000 --ifconfig 192.168.222.11 192.168.222.10 --cipher aes-256-cbc --fragment 0 --mssfix 0

vim3 ~ # iperf3 -c 192.168.222.11
Connecting to host 192.168.222.11, port 5201
[  5] local 192.168.222.10 port 38452 connected to 192.168.222.11 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  9.16 MBytes  76.8 Mbits/sec    0    732 KBytes       
[  5]   1.00-2.00   sec  8.37 MBytes  70.2 Mbits/sec    0   1.11 MBytes       
[  5]   2.00-3.00   sec  8.75 MBytes  73.4 Mbits/sec    0   1.52 MBytes       
[  5]   3.00-4.00   sec  7.50 MBytes  62.9 Mbits/sec    0   1.93 MBytes       
[  5]   4.00-5.00   sec  8.75 MBytes  73.4 Mbits/sec    0   2.33 MBytes       
[  5]   5.00-6.00   sec  7.50 MBytes  62.9 Mbits/sec    0   2.75 MBytes       
[  5]   6.00-7.00   sec  10.0 MBytes  83.9 Mbits/sec    0   3.21 MBytes       
[  5]   7.00-8.00   sec  10.0 MBytes  83.9 Mbits/sec   23   2.48 MBytes       
[  5]   8.00-9.00   sec  6.25 MBytes  52.4 Mbits/sec  100   1.83 MBytes       
[  5]   9.00-10.00  sec  8.75 MBytes  73.4 Mbits/sec    0   1.99 MBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  85.0 MBytes  71.3 Mbits/sec  123             sender
[  5]   0.00-10.00  sec  84.1 MBytes  70.5 Mbits/sec                  receiver

iperf Done.
vim3 ~ # iperf3 -c 192.168.222.11 -R
Connecting to host 192.168.222.11, port 5201
Reverse mode, remote host 192.168.222.11 is sending
[  5] local 192.168.222.10 port 38456 connected to 192.168.222.11 port 5201
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec  6.21 MBytes  52.1 Mbits/sec                  
[  5]   1.00-2.00   sec  6.13 MBytes  51.4 Mbits/sec                  
[  5]   2.00-3.00   sec  5.79 MBytes  48.6 Mbits/sec                  
[  5]   3.00-4.00   sec  6.54 MBytes  54.9 Mbits/sec                  
[  5]   4.00-5.00   sec  5.99 MBytes  50.3 Mbits/sec                  
[  5]   5.00-6.00   sec  6.42 MBytes  53.8 Mbits/sec                  
[  5]   6.00-7.00   sec  6.51 MBytes  54.6 Mbits/sec                  
[  5]   7.00-8.00   sec  6.46 MBytes  54.2 Mbits/sec                  
[  5]   8.00-9.00   sec  6.51 MBytes  54.6 Mbits/sec                  
[  5]   9.00-10.00  sec  6.48 MBytes  54.3 Mbits/sec                  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  73.8 MBytes  61.8 Mbits/sec    6             sender
[  5]   0.00-10.00  sec  63.0 MBytes  52.9 Mbits/sec                  receiver

iperf Done.

Speed is about twice lower on same mtu - 16000

But now driver allows me to start server with --tun-mtu 65000 - itā€™s near itā€™s maximum

and now speeds are higher tnan afalg --tun-mtu 16000 :

vim3 ~ # iperf3 -c 192.168.222.11
Connecting to host 192.168.222.11, port 5201
[  5] local 192.168.222.10 port 38468 connected to 192.168.222.11 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  18.7 MBytes   157 Mbits/sec    0   2.35 MBytes       
[  5]   1.00-2.00   sec  18.8 MBytes   157 Mbits/sec   23   2.17 MBytes       
[  5]   2.00-3.00   sec  21.2 MBytes   178 Mbits/sec    0   2.54 MBytes       
[  5]   3.00-4.00   sec  21.2 MBytes   178 Mbits/sec    0   2.60 MBytes       
[  5]   4.00-5.00   sec  20.0 MBytes   168 Mbits/sec    0   2.79 MBytes       
[  5]   5.00-6.00   sec  21.2 MBytes   178 Mbits/sec    0   3.04 MBytes       
[  5]   6.00-7.00   sec  21.2 MBytes   178 Mbits/sec   36   1.61 MBytes       
[  5]   7.00-8.00   sec  20.0 MBytes   168 Mbits/sec    0   1.98 MBytes       
[  5]   8.00-9.00   sec  21.2 MBytes   178 Mbits/sec    0   2.29 MBytes       
[  5]   9.00-10.00  sec  20.0 MBytes   168 Mbits/sec    0   2.54 MBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec   204 MBytes   171 Mbits/sec   59             sender
[  5]   0.00-10.00  sec   203 MBytes   171 Mbits/sec                  receiver

iperf Done.
vim3 ~ # iperf3 -c 192.168.222.11 -R
Connecting to host 192.168.222.11, port 5201
Reverse mode, remote host 192.168.222.11 is sending
[  5] local 192.168.222.10 port 38472 connected to 192.168.222.11 port 5201
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec  10.5 MBytes  88.4 Mbits/sec                  
[  5]   1.00-2.00   sec  16.0 MBytes   135 Mbits/sec                  
[  5]   2.00-3.00   sec  16.2 MBytes   136 Mbits/sec                  
[  5]   3.00-4.00   sec  16.8 MBytes   141 Mbits/sec                  
[  5]   4.00-5.00   sec  17.2 MBytes   144 Mbits/sec                  
[  5]   5.00-6.00   sec  16.5 MBytes   138 Mbits/sec                  
[  5]   6.00-7.00   sec  16.5 MBytes   139 Mbits/sec                  
[  5]   7.00-8.00   sec  15.9 MBytes   133 Mbits/sec                  
[  5]   8.00-9.00   sec  15.7 MBytes   132 Mbits/sec                  
[  5]   9.00-10.00  sec  15.5 MBytes   130 Mbits/sec                  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec   168 MBytes   140 Mbits/sec    0             sender
[  5]   0.00-10.00  sec   157 MBytes   132 Mbits/sec                  receiver

iperf Done.

P.S. .config should be chanded after patches:

CONFIG_CRYPTO_DEV_MEDIATEK=y > CONFIG_CRYPTO_DEV_SAFEXCEL=y

P.P.S All tests was performed on 5.10