Compiling project: linking fails on Ubuntu 16.04, LimeSuite

Banana Pi Router design BPI-R4(MT7988)
1

Hello everyone,

I am trying to compile LimeSuite (https://github.com/myriadrf/LimeSuite https://wiki.myriadrf.org/Lime_Suite) for BananaPI-R2.

I am using Ubuntu Mate 16.04 2017-11-27 (BPI-R2 new image : ubuntu 16.04 V1.2.1 BT and WIFI AP mode are working fine 2017-11-27)

On Ubuntu 16.04 desktop (x86-64) this project compiles without any problems, while here on BananaPI i get these linking errors:

/usr/bin/ld: CMakeFiles/basicRX.dir/basicRX.cpp.o: undefined reference to symbol '__printf_chk@@GLIBC_2.4'
//lib/arm-linux-gnueabihf/libc.so.6: error adding symbols: DSO missing from command line
collect2: error: ld returned 1 exit status
src/examples/CMakeFiles/basicRX.dir/build.make:97: recipe for target 'bin/basicRX' failed
make[2]: *** [bin/basicRX] Error 1
CMakeFiles/Makefile2:261: recipe for target 'src/examples/CMakeFiles/basicRX.dir/all' failed
make[1]: *** [src/examples/CMakeFiles/basicRX.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
/usr/bin/ld: CMakeFiles/gpio_example.dir/gpio_example.cpp.o: undefined reference to symbol 'exit@@GLIBC_2.4'
//lib/arm-linux-gnueabihf/libc.so.6: error adding symbols: DSO missing from command line
collect2: error: ld returned 1 exit status
/usr/bin/ld: CMakeFiles/boardEmulator.dir/boardEmulator.cpp.o: undefined reference to symbol '__printf_chk@@GLIBC_2.4'
//lib/arm-linux-gnueabihf/libc.so.6: error adding symbols: DSO missing from command line
src/examples/CMakeFiles/gpio_example.dir/build.make:97: recipe for target 'bin/gpio_example' failed
make[2]: *** [bin/gpio_example] Error 1
collect2: error: ld returned 1 exit status
CMakeFiles/Makefile2:335: recipe for target 'src/examples/CMakeFiles/gpio_example.dir/all' failed
src/CMakeFiles/boardEmulator.dir/build.make:97: recipe for target 'src/boardEmulator' failed
make[1]: *** [src/examples/CMakeFiles/gpio_example.dir/all] Error 2
make[2]: *** [src/boardEmulator] Error 1
CMakeFiles/Makefile2:164: recipe for target 'src/CMakeFiles/boardEmulator.dir/all' failed
make[1]: *** [src/CMakeFiles/boardEmulator.dir/all] Error 2
/usr/bin/ld: CMakeFiles/dualRXTX.dir/dualRXTX.cpp.o: undefined reference to symbol 'exit@@GLIBC_2.4'
//lib/arm-linux-gnueabihf/libc.so.6: error adding symbols: DSO missing from command line
collect2: error: ld returned 1 exit status
src/examples/CMakeFiles/dualRXTX.dir/build.make:97: recipe for target 'bin/dualRXTX' failed
make[2]: *** [bin/dualRXTX] Error 1
CMakeFiles/Makefile2:298: recipe for target 'src/examples/CMakeFiles/dualRXTX.dir/all' failed
make[1]: *** [src/examples/CMakeFiles/dualRXTX.dir/all] Error 2
Makefile:127: recipe for target 'all' failed
make: *** [all] Error

I really do not understand how can there be problems finding exit() and __printf_chk() symbols, as far as i understand libc is always linked by default, and both exit and printf are in the standard libc library. I’m not sure how to tackle this issue. Using cmake-gui and adding -lc to force linking libc in the cmake configuration (CMAKE_MODULE_LINKER_FLAGS, CMAKE_SHARED_LINKER_FLAGS, CMAKE_STATIC_LINKER_FLAGS) did not help.

Someone had a similar problem while compiling WiringPi (How to: banana pi BPI-M3 install BPI-WiringPi) however, no solution (other than moving to WiringPi2) is mentioned.

Other search results do not seem useful.

Relevant software versions on the bananapi: 1. gcc version: gcc (Ubuntu/Linaro 5.4.0-6ubuntu1~16.04.4) 5.4.0 20160609 2. ld version: GNU ld (GNU Binutils for Ubuntu) 2.26.1 3. libc version: ldd (Ubuntu GLIBC 2.23-0ubuntu9) 2.23 4. /lib/arm-linux-gnueabihf/libc.so.6 version:GNU C Library (Ubuntu GLIBC 2.23-0ubuntu9) stable release version 2.23

Any help or pointers in the right direction would be greatly appreciated

EDIT: Apparently cmake defaults to using /usr/local/lib as the library path for gcc, while it should be using /lib/arm-linux-gnueabihf. /usr/local/lib/libc.so is actually the musl library, manually linking the boardEmulator.cpp.o file while using -L/lib/arm-linux-gnueabihf fixes all the libc problems, but i can’t simply delete libc.so from there. Is it possible to set GLIBC to be default, instead of Musl libc?

EDIT2: PROBLEM HAS BEEN FIXED I have created a patch for bananapi-r2. Apparently CMAKE finds libusb in /usr/local/lib by default, and then passes the argument -L/usr/local/lib to the linker, while the correct lib location is /lib/arm-linux-gnueabihf.

For compiling on the bananapi-R2, please clone the bananapi-r2-cmake-patch branch, like so:

git clone -b bananapi-r2-cmake-patch https://github.com/myriadrf/LimeSuite.git

and when running cmake, pass -Dbananapi-r2=YES as an argument BEFORE the source folder path:

(builddir)$ cmake -Dbananapi-r2=YES ../

All the other build steps are the same as explained on the myriadrf/LimeSuite wiki. http://wiki.myriadrf.org/Lime_Suite

2

Why do you using crosscompiler when ccompiling on target-system?

3

I am not crosscompiling, cmake autodetects and uses the default tools located at /usr/bin/gcc, /usr/bin/g++, /usr/bin/ld on the bananapi. I compiled on x86 just for testing, but it was a native build, not crosscompilation

4

arm-linux-gnueabihf is a crosscompiler…

5

well… i don’t know? i have no idea why it uses those libraries at //lib/arm-linux-gnueabihf/libc.so.6. It should use /usr/local/lib/libc.so by default then, should it not? Sorry, i am not overly experienced in this

6

i’m not an expert too, but i know this is wrong if you get those crosscompiler-messages directly on the bpi…

i understand you right that you compile on the bpi-r2?

7

Yes, i am compiling directly on the bpi-r2, however i really dont know why it is linking against the crosscompiler libs, and as this happens by default i just assumed that was the correct behavior.

I think that the libs should be in /usr/local/lib, but using make VERBOSE=1 shows that /usr/local/lib is correctly passed as the library path, though i dont get why it tries to link against libs in /lib/arm-linux-gnueabihf

8

Maybe the build-script does not detect the host for check if crosscompile is needed…it will simply make always a crosscompile…you have to check where CROSS_COMPILE environment var is set and disable it…if you don’t find that you can search for the crosscompiler which is used…

grep -Rn 'gnueabihf'
9

isnt /lib/arm-linux-gnueabihf for multiarch support or something?

What i dont understand is why is there a musl libc.so in /usr/local/lib, when on an x86-64 machine there is no musl libc by default, only glibc in /lib/x86-64-linux-gnueabi? ld seems to have a problem with the musl libc.so.

So i just found out how to check all the libs and dirs used by ld (here: https://stackoverflow.com/questions/23026126/how-to-remove-an-entry-from-ld-so-cache and https://stackoverflow.com/questions/9922949/how-to-print-the-ldlinker-search-path

and the result is

/usr/lib/mali:
/lib/arm-linux-gnueabihf:
/usr/lib/arm-linux-gnueabihf:
/usr/lib/arm-linux-gnueabihf/mesa-egl:
/usr/lib/arm-linux-gnueabihf/mesa:
/usr/local/lib:
/opt/vc/lib:
/lib:
/usr/lib:
/usr/lib/arm-linux-gnueabihf/vfp: (hwcap: 0x0000000000000040)
/usr/lib/arm-linux-gnueabihf/vfp/neon: (hwcap: 0x0000000000001040)

so can i assume that ld by default looks at /lib/arm-linux-gnueabihf? I dont think it is an okay solution to delete the multiarch support from /etc/ld.so.conf.d/. Makes me think that ubuntu on bananapi-r2 is somehow incorrectly pachaged, as there isnt such a problem on rpi v2, at least as far as i know.

edit: Looking at cmake’s generated link.txt file for the boardEmulator.cpp.o is this:

/usr/bin/c++    -fvisibility-inlines-hidden -O3 -DNDEBUG   
CMakeFiles/boardEmulator.dir/boardEmulator.cpp.o  -o boardEmulator 
-L/usr/local/lib -rdynamic libLimeSuite.so.18.01.0 -pthread -lsqlite3
-lusb-1.0 -Wl,-rpath,/home/pi/Desktop/LimeSuite/builddir/src:/usr/local/lib

I wrote a simple helloworld using cstdlib:

#include <cstdio>
#include <cstdlib>
int main(void){
   printf("\nhello\n");
   exit(0);
}

compiling and linking with -L/usr/local/lib results in similar errors:

pi@bpi-iot-ros-ai:~/Desktop$ c++ -o test test.cpp -L/usr/local/lib
/usr/lib/gcc/arm-linux-gnueabihf/5/../../../arm-linux-gnueabihf/crt1.o: In function `_start':
(.text+0x1c): undefined reference to `__libc_start_main'
/usr/lib/gcc/arm-linux-gnueabihf/5/../../../arm-linux-gnueabihf/crt1.o: In function `_start':
(.text+0x20): undefined reference to `abort'
/usr/lib/gcc/arm-linux-gnueabihf/5/../../../arm-linux-gnueabihf/crt1.o: In function `_start':
(.text+0x24): undefined reference to `__libc_csu_fini'
/usr/lib/gcc/arm-linux-gnueabihf/5/../../../arm-linux-gnueabihf/crt1.o: In function `_start':
(.text+0x2c): undefined reference to `__libc_csu_init'
/tmp/ccWVCtFg.o: In function `main':
test.cpp:(.text+0xc): undefined reference to `puts'
test.cpp:(.text+0x12): undefined reference to `exit'
/tmp/ccWVCtFg.o:(.ARM.exidx+0x0): undefined reference to `__aeabi_unwind_cpp_pr0'
collect2: error: ld returned 1 exit status

while not passing -L/usr/local/lib, or passing -L/lib/arm-linux-gnueabihf produces a working executable:

pi@bpi-iot-ros-ai:~/Desktop$ c++ -o test test.cpp 
pi@bpi-iot-ros-ai:~/Desktop$ ./test

hello
pi@bpi-iot-ros-ai:~/Desktop$ c++ -o test test.cpp -L/lib/arm-linux-gnueabihf
pi@bpi-iot-ros-ai:~/Desktop$ ./test

hello
10

I also tried it，I could compile the project successfully on my PC（Ubuntu，GCC Version is 5.4.0），but I failed to cross compile the project on R2，I compared the compile tools version，they are same，only different thing is platform （X86 and ARM），I am also trying to find the way to compile LIMESUITE on R2……

11

Problem has been fixed, I have created a patch for bananapi-r2. Apparently CMAKE finds libusb in /usr/local/lib by default, and then passes the argument -L/usr/local/lib to the linker, while the correct lib location is /lib/arm-linux-gnueabihf.

For compiling on the bananapi-R2, please clone the bananapi-r2-cmake-patch branch, like so:

git clone -b bananapi-r2-cmake-patch https://github.com/myriadrf/LimeSuite.git

and when running cmake, pass -Dbananapi-r2=YES as an argument BEFORE the source folder path:

(builddir)$ cmake -Dbananapi-r2=YES ../

All the other build steps are the same as explained on the myriadrf/LimeSuite wiki. http://wiki.myriadrf.org/Lime_Suite

1 Like
12

Hi Toby，Can you share what have you done in your branch? I’m very interested in it. thank you.

13

yes, of course, here is the patch commit

It just forces cmake to look for libusb in /usr/lib/arm-linux-gnueabihf when building for bananapi-r2

1 Like
14

Cool, I built it successfully, Thanks Toby.

1 Like