https://forums.openvpn.net/viewtopic.php?p=75326&sid=5e93d78cbcc1a93769f848fa8be0e971#p75326
if this is true openvpn should use cryptodev in openssl by default…you can make a test and again without cryptodev loaded
Hello,
How can I cross compile openssl 1.1.1 with cryptodev support on Debian Buster ? I think there were some changes from version 1.1.1.and up. I was trying to follow the steps highlighted here http://trac.gateworks.com/wiki/linux/crypto#BuildingOpenSSLversions1.1.1andlaterwithcryptodevsupport but at some point it fails because of some missing libraries
make[1]: Entering directory '/home/sysadmin/openssl-1.1.1/openssl-1.1.1c' sed -i '/^udeb: libssl/s/libcrypto1.1-udeb/libssl1.1-udeb/' debian/libssl1.1/DEBIAN/shlibs dh_shlibdeps -a -L libssl1.1 dpkg-shlibdeps: error: cannot find library libdl.so.2 needed by debian/openssl/usr/bin/openssl (ELF format: 'elf32-littlearm' abi: '0101002800000000'; RPATH: '') dpkg-shlibdeps: error: cannot find library libpthread.so.0 needed by debian/openssl/usr/bin/openssl (ELF format: 'elf32-littlearm' abi: '0101002800000000'; RPATH: '') dpkg-shlibdeps: error: cannot find library libc.so.6 needed by debian/openssl/usr/bin/openssl (ELF format: 'elf32-littlearm' abi: '0101002800000000'; RPATH: '') dpkg-shlibdeps: error: cannot find library ld-linux-armhf.so.3 needed by debian/openssl/usr/bin/openssl (ELF format: 'elf32-littlearm' abi: '0101002800000000'; RPATH: '') dpkg-shlibdeps: error: cannot continue due to the errors listed above Note: libraries are not searched in other binary packages that do not have any shlibs or symbols file. To help dpkg-shlibdeps find private libraries, you might need to use -l. dh_shlibdeps: dpkg-shlibdeps -Tdebian/openssl.substvars -Sdebian/libssl1.1 debian/openssl/usr/bin/openssl returned exit code 2 dh_shlibdeps: Aborting due to earlier error make[1]: *** [debian/rules:150: override_dh_shlibdeps] Error 25 make[1]: Leaving directory '/home/sysadmin/openssl-1.1.1/openssl-1.1.1c' make: *** [debian/rules:49: binary] Error 2 dpkg-buildpackage: error: debian/rules binary subprocess returned exit status 2
All the files which the compiler complains missing are available. Do I need to set some PATH or environment variable for the compiler to be able access them ?
Update: I found some advice on a forum to edit the debian/rules file and add the LD_LIBRARY_PATH there and then compilation completed wit success.
root@bpi-r2:~# openvpn --show-engines
OpenSSL Crypto Engines
/dev/crypto engine [devcrypto]
Dynamic engine loading support [dynamic]
btw. i started to add openssl building some time ago to my repo, just take a look on openssl.sh
https://github.com/frank-w/BPI-R2-4.14/tree/4.19-main/utils
i expect building on r2 is much slower and not everybody wants compile-tools on a router 
you could also look for cryptodev to use hardware-acceleration in openssl
i don’t know if the scripts were complete, but maybe you can help here if they aren’t working
It seems that after loading cryptodev openssl module SSH connections using OpenSSH 7.9p on Debian 10 do not work anymore.
I saw there is a bug opened on Debian bug tracker for this issue: https://www.mail-archive.com/[email protected]/msg1686025.html
As a workaround I have switched to Dropbear which does not seem to be affected by this bug.
Hi,
I just tried to use your script to build openssl with cryptodev and it compiles ok but the checkinstall command when trying to build he deb package fails at the step of building the file list and generates a very small deb file 724 bytes which contains only an openssl config file.
Can checkinstall be done for different arch? Imho you need to do make install and some kind of fakeroot/dpkg-buildpackage
btw. Where did you try cryptodev? imho it needs a dts-node i have added only in 4.14
i have tried in 4.19-main branch. this is the output from the script
======================== Installation successful ==========================
Copying files to the temporary directory...OK
Stripping ELF binaries and libraries...OK
Compressing man pages...OK
Building file list... FAILED!
Building Debian package...OK
NOTE: The package will not be installed
Erasing temporary files...OK
Writing backup package...OK
OK
Deleting temp dir...OK
**********************************************************************
Done. The new package has been saved to
/home/sysadmin/bpir2-4.19/utils/openssl/openssl_20190829-1_armhf.deb
You can install it in your system anytime using:
dpkg -i openssl_20190829-1_armhf.deb
**********************************************************************Mhm…package is names armhf…but error occours in building file list…i don’t know checkinstall very well…maybe there is any debug-mode to get a clearer message?
@frank-w I have copied the utils folder from 4.14-main repository to 5.4-main branch and I tried to compile cryptodev (I saw you updated it to version 1.9) but it fails. When I ran ./build.sh cryptodev it throws the below compilation error
Building modules, stage 2.
MODPOST 1 modules
ERROR: "crypto_givcipher_type" [/home/sysadmin/bpi-r2_5.4/utils/cryptodev/cryptodev-linux/cryptodev.ko] undefined!
ERROR: "sys_close" [/home/sysadmin/bpi-r2_5.4/utils/cryptodev/cryptodev-linux/cryptodev.ko] undefined!
make[2]: *** [scripts/Makefile.modpost:94: __modpost] Error 1
make[1]: *** [Makefile:1609: modules] Error 2
make[1]: Leaving directory '/home/sysadmin/bpi-r2_5.4'
make: *** [Makefile:27: build] Error 2Look where crypto_givcipher_type and sys_close are defined and make sure file with implementation is linked by makefile.
UPDATE: I have removed version 1.9 and pulled the latest master from github and it works fine. It appears 1.9 release does not work with 5.x kernels and it was fixed after 1.10 release. (https://github.com/cryptodev-linux/cryptodev-linux/commit/f971e0cd4a0ebe59fb2e8e17240399bf6901b09b)
1 Like
i have added cryptodev 1.10 + the Patch you’ve sorted out to my 5.4-main tree
thanks for checking
5.4.27-bpi-r2-main
I could not get cryptodev working. I spent nearly two whole days trying to figure out how to enable it for OpenSSL.
It turns out, that devcryptoeng is not enabled by default and the defines HAVE_CRYPTODEV and USE_CRYPTODEV_DIGESTS don’t change a thing about that. You need to add one more argument - enable-devcryptoeng.
Working commands for me were
$ export CROSS=arm-linux-gnueabihf
$ export CC=${CROSS}-gcc
$ export LD=${CROSS}-ld
$ export AS=${CROSS}-as
$ export AR=${CROSS}-ar
$ export DEB_HOST_ARCH=armhf
$ export DEB_BUILD_OPTIONS=nocheck
$ git clone https://github.com/cryptodev-linux/cryptodev-linux.git
$ apt-get source openssl
# You must match your version
$ cd openssl-1.1.1.d
$ sed -i -e "s/CONFARGS =/CONFARGS = enable-devcryptoeng -DHAVE_CRYPTODEV -DUSE_CRYPTODEV_DIGESTS -I..\/..\/cryptodev-linux/" debian/rules
# I also had to disable shlibdeps by commenting `dh_shlibdeps` at the end of the `debian/rules`
$dpkg-buildpackage -us -uc -aarmhf -b
You need to have cryptodev already compiled. Then just install created deb packages.
Hope this helps. If you found anything wrong, let me know and I will edit this post.
1 Like
Can you post a patch for cryptodev i included in my kernelrepo (5.4-main)?
The soonest I will have time to do it is maybe next weekend.
Anyway… I had problem with SSH. I couldn’t establish a connection. devcrypto was obviously on fault.
I found this article where it says where is the problem…
Disabling digests
Please, don’t enable digests unless you know what you’re doing. They are usually slower than software, >except for large (> 10k) blocks. Some applications–openssh, for example–will not work with /dev/crypto >digests. This is a limitation of how the engine works. Openssh will save a partial digest, and then fork, >duplicating that context, and working with successive copies of it, which is useful for HMAC, where the >hash of the key remains constant. In the kernel, however, those contexts are still linked to the same >session, so when one process calls another update, or closes that digest context, the kernel session is >changed/closed for all of the instances, and you’ll get a libcrypto failure. For well-behaved applications >using large update blocks, you may enable digests. Use a separate copy of the
openssl.cnf>configuration file, and setOPENSSL_CONF=_path_to_filein the environment before running it (add it >to the respective file in /etc/init.d/). Again, benchmarking the actual application you’re using is the best >way to gauge the impact of hardware crypto.
I tried to disable using DIGESTS in OpenSSL through configuration, but it seems that it ignores default_algorithms for cryptodev, even with USE_SOFTDRIVERS=1 I also removed defines for compilation, so that only enable-devcryptoeng was left, but it didn’t help. At least I found out, that cryptodev can be enabled just with enable-devcryptoeng and no other defines are needed.
Anyway… that lead me to the only possible solution left - removing it from source. Source file where you can disable it is in openssl/crypto/engines
After I compiled it, I got this (missing digests)
root@claudius:~# openssl engine -c
(devcrypto) /dev/crypto engine
[DES-CBC, DES-EDE3-CBC, AES-128-CBC, AES-192-CBC, AES-256-CBC, AES-128-CTR, AES-192-CTR, AES-256-CTR, AES-128-ECB, AES-192-ECB, AES-256-ECB]
(dynamic) Dynamic engine loading support
I am planning to disable all ciphers except AES-CBC. I need to use cryptodev only for VPN and more ciphers would just add more interrupts to core0.
If somebody could figure out how to disable cryptodev for DIGESTS without making modifications to source, that would be great. Here is a “documentation” https://www.openssl.org/docs/man1.1.1/man5/config.html#Engine-Configuration-Module
I couldn’t find any usage of engine config, except for some sample
1 Like
I had also that problem with SSH when I enabled devcrypto for OpenSSL. Initially I did not know why I cannot login and then I found some page that stated you cannot accelerate OpenSSH connection because latest version of OpenSSH enforce usage of seccomp sandbox which forbids some syscalls required to use AF_ALG. I think the same applied to devcrypto engine also.
Refer to Debian bug #931271.
I have recompiled openssh from source and disabled seccomp sandbox to allow me to connect remotely to the box
#Edit debian/rules and add it to common build options:
confflags += --with-sandbox=no
1 Like
Hi everyone!
I’ve tested my R2 with AF_ALG acceleration - work good for me. All i needed to do is to add following kernel config options:
CONFIG_CRYPTO_USER=m
CONFIG_CRYPTO_USER_API=m # added automatically after one options below was added
CONFIG_CRYPTO_USER_API_HASH=m # probably usable, but was not used this time
CONFIG_CRYPTO_USER_API_SKCIPHER=m # this one was really used
CONFIG_CRYPTO_USER_API_RNG=m # - theese two are kikely useless for R2
CONFIG_CRYPTO_USER_API_AEAD=m # /
After recompiling kernel and reboot, ssl conf needed to be changed:
cat /etc/ssl/openssl.cnf:
openssl_conf = openssl_def
[openssl_def]
engines = openssl_engines
[openssl_engines]
afalg = afalg_engine
[afalg_engine]
init=1
openssl engine list:
r2-gentoo ~ # openssl engine -t -c -v
(dynamic) Dynamic engine loading support
[ unavailable ]
SO_PATH, NO_VCHECK, ID, LIST_ADD, DIR_LOAD, DIR_ADD, LOAD
(afalg) AFALG engine support
[AES-128-CBC, AES-192-CBC, AES-256-CBC]
[ available ]
and finnaly perfomance test:
no acceleration:
bpi-r2-gentoo ~ # time openssl speed -evp aes-128-cbc -elapsed
You have chosen to measure elapsed time instead of user CPU time.
Doing aes-128-cbc for 3s on 16 size blocks: 4551769 aes-128-cbc's in 3.00s
Doing aes-128-cbc for 3s on 64 size blocks: 1420522 aes-128-cbc's in 3.00s
Doing aes-128-cbc for 3s on 256 size blocks: 357557 aes-128-cbc's in 3.00s
Doing aes-128-cbc for 3s on 1024 size blocks: 79175 aes-128-cbc's in 3.00s
Doing aes-128-cbc for 3s on 8192 size blocks: 9870 aes-128-cbc's in 3.00s
Doing aes-128-cbc for 3s on 16384 size blocks: 4496 aes-128-cbc's in 3.00s
OpenSSL 1.1.1j 16 Feb 2021
built on: Fri Feb 19 22:05:09 2021 UTC
options:bn(64,32) rc4(char) des(long) aes(partial) idea(int) blowfish(ptr)
compiler: armv7a-hardfloat-linux-gnueabi-gcc -fPIC -pthread -Wa,--noexecstack -Wall -O2 -pipe -march=armv7-a -mfpu=neon-vfpv4 -mfloat-abi=hard -fno-strict-aliasing -Wa,--noexecstack -DOPENSSL_USE_NODELETE -DOPEN
SSL_PIC -DOPENSSL_CPUID_OBJ -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DKECCAK1600_ASM -DAES_ASM -DBSAES_ASM -DGHASH_ASM -DECP_NISTZ256_ASM -DPOLY1305_ASM -DZLIB -DNDEBUG -
DL_ENDIAN -DOPENSSL_NO_BUF_FREELISTS
The 'numbers' are in 1000s of bytes per second processed.
type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes 16384 bytes
aes-128-cbc 24276.10k 30304.47k 30511.53k 27025.07k 26951.68k 24554.15k
real 0m18,025s
user 0m15,880s
sys 0m0,060s
AF_ALG:
r2-gentoo ~ # time openssl speed -evp aes-128-cbc -elapsed
You have chosen to measure elapsed time instead of user CPU time.
Doing aes-128-cbc for 3s on 16 size blocks: 63195 aes-128-cbc's in 3.00s
Doing aes-128-cbc for 3s on 64 size blocks: 62880 aes-128-cbc's in 3.00s
Doing aes-128-cbc for 3s on 256 size blocks: 61864 aes-128-cbc's in 3.00s
Doing aes-128-cbc for 3s on 1024 size blocks: 59010 aes-128-cbc's in 3.00s
Doing aes-128-cbc for 3s on 8192 size blocks: 29029 aes-128-cbc's in 3.00s
Doing aes-128-cbc for 3s on 16384 size blocks: 16145 aes-128-cbc's in 3.00s
OpenSSL 1.1.1j 16 Feb 2021
built on: Mon Feb 22 11:56:18 2021 UTC
options:bn(64,32) rc4(char) des(long) aes(partial) idea(int) blowfish(ptr)
compiler: armv7a-hardfloat-linux-gnueabi-gcc -fPIC -pthread -Wa,--noexecstack -Wall -O2 -pipe -march=armv7-a -mfpu=neon-vfpv4 -mfloat-abi=hard -fno-strict-aliasing -Wa,--noexecstack -DOPENSSL_USE_NODELETE -DOPEN
SSL_PIC -DOPENSSL_CPUID_OBJ -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DKECCAK1600_ASM -DAES_ASM -DBSAES_ASM -DGHASH_ASM -DECP_NISTZ256_ASM -DPOLY1305_ASM -DZLIB -DNDEBUG -
DL_ENDIAN -DOPENSSL_NO_BUF_FREELISTS
The 'numbers' are in 1000s of bytes per second processed.
type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes 16384 bytes
aes-128-cbc 337.04k 1341.44k 5279.06k 20142.08k 79268.52k 88173.23k
real 0m18,543s
user 0m0,824s
sys 0m12,075s
user 0m15,880s - non-accelerated vs user 0m0,824s - accelerated
~2.94 times faster on 8192 bytes blocks and ~3.6 times faster on 16384 bytes blocks (single run)
similar difference whet testing aes-192-cbc and aes-256-cbc
r2-gentoo ~ # cat /proc/interrupts | grep aes
51: 339897 0 0 0 MT_SYSIRQ 82 Level mtk-aes
52: 0 0 0 0 MT_SYSIRQ 83 Level mtk-aes
As for me - a good way to accelerate out of box: on mainline kernel w/o additional modules and mainline openssl w/o patches/additional engines.
Openvpn/openssh(sandbox=no required) are not yet tested,
Also testing on standard @frank-w’s kernel config is probably needed.
P.S. tested on 5.9.0-rc8 kernel
so you need no recompile of openssl and no change in dts (for using eip97 engine imho now using different compatible/driver), right?
you can create a pull request (for LTS main branch), then i add the config-options
Yes, at least for my kernel config. Only openssl configuration is neeed to be changed.
Sure, what is the LTS-main branch, 5.4 or 5.10? I’ll can do for both, but 5.4 is low-priority.
I’ll appreciate for any additional tests by community.
5.10 is enough…can cherry-pick it to 5.4.