Is it possible to have the crypto extensions working?

Banana Pi BPI-R4/BPI-R3 Mini/BPI-R3/BPI-R64/BPI-R2
42

https://forums.openvpn.net/viewtopic.php?p=75326&sid=5e93d78cbcc1a93769f848fa8be0e971#p75326

if this is true openvpn should use cryptodev in openssl by default…you can make a test and again without cryptodev loaded

44

Hello,

How can I cross compile openssl 1.1.1 with cryptodev support on Debian Buster ? I think there were some changes from version 1.1.1.and up. I was trying to follow the steps highlighted here http://trac.gateworks.com/wiki/linux/crypto#BuildingOpenSSLversions1.1.1andlaterwithcryptodevsupport but at some point it fails because of some missing libraries

make[1]: Entering directory '/home/sysadmin/openssl-1.1.1/openssl-1.1.1c'
sed -i '/^udeb: libssl/s/libcrypto1.1-udeb/libssl1.1-udeb/' debian/libssl1.1/DEBIAN/shlibs
dh_shlibdeps -a -L libssl1.1
dpkg-shlibdeps: error: cannot find library libdl.so.2 needed by debian/openssl/usr/bin/openssl (ELF format: 'elf32-littlearm' abi: '0101002800000000'; RPATH: '')
dpkg-shlibdeps: error: cannot find library libpthread.so.0 needed by debian/openssl/usr/bin/openssl (ELF format: 'elf32-littlearm' abi: '0101002800000000'; RPATH: '')
dpkg-shlibdeps: error: cannot find library libc.so.6 needed by debian/openssl/usr/bin/openssl (ELF format: 'elf32-littlearm' abi: '0101002800000000'; RPATH: '')
dpkg-shlibdeps: error: cannot find library ld-linux-armhf.so.3 needed by debian/openssl/usr/bin/openssl (ELF format: 'elf32-littlearm' abi: '0101002800000000'; RPATH: '')
dpkg-shlibdeps: error: cannot continue due to the errors listed above
Note: libraries are not searched in other binary packages that do not have any shlibs or symbols file.
To help dpkg-shlibdeps find private libraries, you might need to use -l.
dh_shlibdeps: dpkg-shlibdeps -Tdebian/openssl.substvars -Sdebian/libssl1.1 debian/openssl/usr/bin/openssl returned exit code 2
dh_shlibdeps: Aborting due to earlier error
make[1]: *** [debian/rules:150: override_dh_shlibdeps] Error 25
make[1]: Leaving directory '/home/sysadmin/openssl-1.1.1/openssl-1.1.1c'
make: *** [debian/rules:49: binary] Error 2
dpkg-buildpackage: error: debian/rules binary subprocess returned exit status 2

All the files which the compiler complains missing are available. Do I need to set some PATH or environment variable for the compiler to be able access them ?

Update: I found some advice on a forum to edit the debian/rules file and add the LD_LIBRARY_PATH there and then compilation completed wit success.

root@bpi-r2:~# openvpn --show-engines

OpenSSL Crypto Engines

/dev/crypto engine [devcrypto]

Dynamic engine loading support [dynamic]

45

btw. i started to add openssl building some time ago to my repo, just take a look on openssl.sh

https://github.com/frank-w/BPI-R2-4.14/tree/4.19-main/utils

i expect building on r2 is much slower and not everybody wants compile-tools on a router :wink:

you could also look for cryptodev to use hardware-acceleration in openssl

i don’t know if the scripts were complete, but maybe you can help here if they aren’t working

46

It seems that after loading cryptodev openssl module SSH connections using OpenSSH 7.9p on Debian 10 do not work anymore.

I saw there is a bug opened on Debian bug tracker for this issue: https://www.mail-archive.com/[email protected]/msg1686025.html

As a workaround I have switched to Dropbear which does not seem to be affected by this bug.

48

Hi,

I just tried to use your script to build openssl with cryptodev and it compiles ok but the checkinstall command when trying to build he deb package fails at the step of building the file list and generates a very small deb file 724 bytes which contains only an openssl config file.

49

Can checkinstall be done for different arch? Imho you need to do make install and some kind of fakeroot/dpkg-buildpackage

btw. Where did you try cryptodev? imho it needs a dts-node i have added only in 4.14

50

i have tried in 4.19-main branch. this is the output from the script

======================== Installation successful ==========================

Copying files to the temporary directory...OK

Stripping ELF binaries and libraries...OK

Compressing man pages...OK

Building file list... FAILED!

Building Debian package...OK

NOTE: The package will not be installed

Erasing temporary files...OK

Writing backup package...OK
OK

Deleting temp dir...OK


**********************************************************************

 Done. The new package has been saved to

 /home/sysadmin/bpir2-4.19/utils/openssl/openssl_20190829-1_armhf.deb
 You can install it in your system anytime using:

      dpkg -i openssl_20190829-1_armhf.deb

**********************************************************************
51

Mhm…package is names armhf…but error occours in building file list…i don’t know checkinstall very well…maybe there is any debug-mode to get a clearer message?

52

@frank-w I have copied the utils folder from 4.14-main repository to 5.4-main branch and I tried to compile cryptodev (I saw you updated it to version 1.9) but it fails. When I ran ./build.sh cryptodev it throws the below compilation error

Building modules, stage 2.
  MODPOST 1 modules
ERROR: "crypto_givcipher_type" [/home/sysadmin/bpi-r2_5.4/utils/cryptodev/cryptodev-linux/cryptodev.ko] undefined!
ERROR: "sys_close" [/home/sysadmin/bpi-r2_5.4/utils/cryptodev/cryptodev-linux/cryptodev.ko] undefined!
make[2]: *** [scripts/Makefile.modpost:94: __modpost] Error 1
make[1]: *** [Makefile:1609: modules] Error 2
make[1]: Leaving directory '/home/sysadmin/bpi-r2_5.4'
make: *** [Makefile:27: build] Error 2
53

Look where crypto_givcipher_type and sys_close are defined and make sure file with implementation is linked by makefile.

54

UPDATE: I have removed version 1.9 and pulled the latest master from github and it works fine. It appears 1.9 release does not work with 5.x kernels and it was fixed after 1.10 release. (https://github.com/cryptodev-linux/cryptodev-linux/commit/f971e0cd4a0ebe59fb2e8e17240399bf6901b09b)

1 Like
55

i have added cryptodev 1.10 + the Patch you’ve sorted out to my 5.4-main tree

thanks for checking

56

5.4.27-bpi-r2-main

I could not get cryptodev working. I spent nearly two whole days trying to figure out how to enable it for OpenSSL.

It turns out, that devcryptoeng is not enabled by default and the defines HAVE_CRYPTODEV and USE_CRYPTODEV_DIGESTS don’t change a thing about that. You need to add one more argument - enable-devcryptoeng.

Working commands for me were

$ export CROSS=arm-linux-gnueabihf
$ export CC=${CROSS}-gcc
$ export LD=${CROSS}-ld
$ export AS=${CROSS}-as
$ export AR=${CROSS}-ar
$ export DEB_HOST_ARCH=armhf
$ export DEB_BUILD_OPTIONS=nocheck

$ git clone https://github.com/cryptodev-linux/cryptodev-linux.git
$ apt-get source openssl
# You must match your version
$ cd openssl-1.1.1.d
$ sed -i -e "s/CONFARGS  =/CONFARGS = enable-devcryptoeng -DHAVE_CRYPTODEV -DUSE_CRYPTODEV_DIGESTS -I..\/..\/cryptodev-linux/" debian/rules
# I also had to disable shlibdeps by commenting `dh_shlibdeps` at the end of the `debian/rules`
$dpkg-buildpackage -us -uc -aarmhf -b

You need to have cryptodev already compiled. Then just install created deb packages.

Hope this helps. If you found anything wrong, let me know and I will edit this post.

1 Like
57

Can you post a patch for cryptodev i included in my kernelrepo (5.4-main)?

58

The soonest I will have time to do it is maybe next weekend.

Anyway… I had problem with SSH. I couldn’t establish a connection. devcrypto was obviously on fault.

I found this article where it says where is the problem…

Disabling digests

Please, don’t enable digests unless you know what you’re doing. They are usually slower than software, >except for large (> 10k) blocks. Some applications–openssh, for example–will not work with /dev/crypto >digests. This is a limitation of how the engine works. Openssh will save a partial digest, and then fork, >duplicating that context, and working with successive copies of it, which is useful for HMAC, where the >hash of the key remains constant. In the kernel, however, those contexts are still linked to the same >session, so when one process calls another update, or closes that digest context, the kernel session is >changed/closed for all of the instances, and you’ll get a libcrypto failure. For well-behaved applications >using large update blocks, you may enable digests. Use a separate copy of the openssl.cnf >configuration file, and set OPENSSL_CONF=_path_to_file in the environment before running it (add it >to the respective file in /etc/init.d/). Again, benchmarking the actual application you’re using is the best >way to gauge the impact of hardware crypto.

I tried to disable using DIGESTS in OpenSSL through configuration, but it seems that it ignores default_algorithms for cryptodev, even with USE_SOFTDRIVERS=1 I also removed defines for compilation, so that only enable-devcryptoeng was left, but it didn’t help. At least I found out, that cryptodev can be enabled just with enable-devcryptoeng and no other defines are needed.

Anyway… that lead me to the only possible solution left - removing it from source. Source file where you can disable it is in openssl/crypto/engines

After I compiled it, I got this (missing digests)

root@claudius:~# openssl engine -c
(devcrypto) /dev/crypto engine
 [DES-CBC, DES-EDE3-CBC, AES-128-CBC, AES-192-CBC, AES-256-CBC, AES-128-CTR, AES-192-CTR, AES-256-CTR, AES-128-ECB, AES-192-ECB, AES-256-ECB]
(dynamic) Dynamic engine loading support

I am planning to disable all ciphers except AES-CBC. I need to use cryptodev only for VPN and more ciphers would just add more interrupts to core0.

If somebody could figure out how to disable cryptodev for DIGESTS without making modifications to source, that would be great. Here is a “documentation” https://www.openssl.org/docs/man1.1.1/man5/config.html#Engine-Configuration-Module

I couldn’t find any usage of engine config, except for some sample

1 Like
59

I had also that problem with SSH when I enabled devcrypto for OpenSSL. Initially I did not know why I cannot login and then I found some page that stated you cannot accelerate OpenSSH connection because latest version of OpenSSH enforce usage of seccomp sandbox which forbids some syscalls required to use AF_ALG. I think the same applied to devcrypto engine also.

Refer to Debian bug #931271.

I have recompiled openssh from source and disabled seccomp sandbox to allow me to connect remotely to the box

#Edit debian/rules and add it to common build options:

confflags += --with-sandbox=no

1 Like
60

Hi everyone!

I’ve tested my R2 with AF_ALG acceleration - work good for me. All i needed to do is to add following kernel config options:

CONFIG_CRYPTO_USER=m
CONFIG_CRYPTO_USER_API=m              # added automatically after one options below was added
CONFIG_CRYPTO_USER_API_HASH=m         # probably usable, but was not used this time
CONFIG_CRYPTO_USER_API_SKCIPHER=m # this one was really used
CONFIG_CRYPTO_USER_API_RNG=m      # - theese two are kikely useless for R2
CONFIG_CRYPTO_USER_API_AEAD=m    # /

After recompiling kernel and reboot, ssl conf needed to be changed:

cat /etc/ssl/openssl.cnf:

openssl_conf = openssl_def

[openssl_def]
engines = openssl_engines

[openssl_engines]
afalg = afalg_engine

[afalg_engine]
init=1

openssl engine list:

r2-gentoo ~ # openssl engine -t -c -v
(dynamic) Dynamic engine loading support
     [ unavailable ]
     SO_PATH, NO_VCHECK, ID, LIST_ADD, DIR_LOAD, DIR_ADD, LOAD
(afalg) AFALG engine support
 [AES-128-CBC, AES-192-CBC, AES-256-CBC]
     [ available ]

and finnaly perfomance test:

no acceleration:

bpi-r2-gentoo ~ # time openssl speed -evp aes-128-cbc -elapsed
You have chosen to measure elapsed time instead of user CPU time.
Doing aes-128-cbc for 3s on 16 size blocks: 4551769 aes-128-cbc's in 3.00s
Doing aes-128-cbc for 3s on 64 size blocks: 1420522 aes-128-cbc's in 3.00s
Doing aes-128-cbc for 3s on 256 size blocks: 357557 aes-128-cbc's in 3.00s
Doing aes-128-cbc for 3s on 1024 size blocks: 79175 aes-128-cbc's in 3.00s
Doing aes-128-cbc for 3s on 8192 size blocks: 9870 aes-128-cbc's in 3.00s
Doing aes-128-cbc for 3s on 16384 size blocks: 4496 aes-128-cbc's in 3.00s
OpenSSL 1.1.1j  16 Feb 2021
built on: Fri Feb 19 22:05:09 2021 UTC
options:bn(64,32) rc4(char) des(long) aes(partial) idea(int) blowfish(ptr)
compiler: armv7a-hardfloat-linux-gnueabi-gcc -fPIC -pthread -Wa,--noexecstack -Wall -O2 -pipe -march=armv7-a -mfpu=neon-vfpv4 -mfloat-abi=hard -fno-strict-aliasing -Wa,--noexecstack -DOPENSSL_USE_NODELETE -DOPEN
SSL_PIC -DOPENSSL_CPUID_OBJ -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DKECCAK1600_ASM -DAES_ASM -DBSAES_ASM -DGHASH_ASM -DECP_NISTZ256_ASM -DPOLY1305_ASM -DZLIB -DNDEBUG -
DL_ENDIAN  -DOPENSSL_NO_BUF_FREELISTS
The 'numbers' are in 1000s of bytes per second processed.
type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes  16384 bytes
aes-128-cbc      24276.10k    30304.47k    30511.53k    27025.07k    26951.68k    24554.15k

real    0m18,025s
user    0m15,880s
sys     0m0,060s

AF_ALG:

r2-gentoo ~ # time openssl speed -evp aes-128-cbc -elapsed
You have chosen to measure elapsed time instead of user CPU time.
Doing aes-128-cbc for 3s on 16 size blocks: 63195 aes-128-cbc's in 3.00s
Doing aes-128-cbc for 3s on 64 size blocks: 62880 aes-128-cbc's in 3.00s
Doing aes-128-cbc for 3s on 256 size blocks: 61864 aes-128-cbc's in 3.00s
Doing aes-128-cbc for 3s on 1024 size blocks: 59010 aes-128-cbc's in 3.00s
Doing aes-128-cbc for 3s on 8192 size blocks: 29029 aes-128-cbc's in 3.00s
Doing aes-128-cbc for 3s on 16384 size blocks: 16145 aes-128-cbc's in 3.00s
OpenSSL 1.1.1j  16 Feb 2021
built on: Mon Feb 22 11:56:18 2021 UTC
options:bn(64,32) rc4(char) des(long) aes(partial) idea(int) blowfish(ptr)
compiler: armv7a-hardfloat-linux-gnueabi-gcc -fPIC -pthread -Wa,--noexecstack -Wall -O2 -pipe -march=armv7-a -mfpu=neon-vfpv4 -mfloat-abi=hard -fno-strict-aliasing -Wa,--noexecstack -DOPENSSL_USE_NODELETE -DOPEN
SSL_PIC -DOPENSSL_CPUID_OBJ -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DKECCAK1600_ASM -DAES_ASM -DBSAES_ASM -DGHASH_ASM -DECP_NISTZ256_ASM -DPOLY1305_ASM -DZLIB -DNDEBUG -
DL_ENDIAN  -DOPENSSL_NO_BUF_FREELISTS
The 'numbers' are in 1000s of bytes per second processed.
type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes  16384 bytes
aes-128-cbc        337.04k     1341.44k     5279.06k    20142.08k    79268.52k    88173.23k

real    0m18,543s
user    0m0,824s
sys     0m12,075s

user 0m15,880s - non-accelerated vs user 0m0,824s - accelerated

~2.94 times faster on 8192 bytes blocks and ~3.6 times faster on 16384 bytes blocks (single run)

similar difference whet testing aes-192-cbc and aes-256-cbc

r2-gentoo ~ # cat /proc/interrupts | grep aes
 51:     339897          0          0          0  MT_SYSIRQ  82 Level     mtk-aes
 52:          0          0          0          0  MT_SYSIRQ  83 Level     mtk-aes

As for me - a good way to accelerate out of box: on mainline kernel w/o additional modules and mainline openssl w/o patches/additional engines.

Openvpn/openssh(sandbox=no required) are not yet tested,

Also testing on standard @frank-w’s kernel config is probably needed.

P.S. tested on 5.9.0-rc8 kernel

61

so you need no recompile of openssl and no change in dts (for using eip97 engine imho now using different compatible/driver), right?

you can create a pull request (for LTS main branch), then i add the config-options

63

Yes, at least for my kernel config. Only openssl configuration is neeed to be changed.

Sure, what is the LTS-main branch, 5.4 or 5.10? I’ll can do for both, but 5.4 is low-priority.

I’ll appreciate for any additional tests by community.

64

5.10 is enough…can cherry-pick it to 5.4.