Ubuntu 20.04 LTS and Version 100.11 "Illegal instruction" sample Qt apps crashing

[OSINT_ENGINEER]

We just setup a new Ubuntu 20.04 LTS machine using Qt 5.15 for app development. The Qt Quick and widget apps work like expected. When we create a simple Qt Quick or widget app using ArcGIS Runtime for Qt Version 100.11 the app crashes immediately with "illegal instruction (core dump)". We set the LD_LIBRARY_PATH to the Qt and ArcGIS Runtime libs. Maybe the runtime core libs are trying to load any missing library.

gdb output:

(gdb) run
Starting program: /home/gisfromscratch/Projects/build-QuickMapApp-Desktop_Qt_5_15_2_GCC_64bit-Debug/QuickMapApp
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".

Program received signal SIGILL, Illegal instruction.
0x00007ffff49b9b2c in ?? ()
from /home/gisfromscratch/arcgis/runtime_sdk/qt100.11/sdk/linux/x64/lib/libruntimecore.so
(gdb) where
#0 0x00007ffff49b9b2c in ?? ()
from /home/gisfromscratch/arcgis/runtime_sdk/qt100.11/sdk/linux/x64/lib/libruntimecore.so
#1 0x00007ffff7fe0b8a in ?? () from /lib64/ld-linux-x86-64.so.2
#2 0x00007ffff7fe0c91 in ?? () from /lib64/ld-linux-x86-64.so.2
#3 0x00007ffff7fd013a in ?? () from /lib64/ld-linux-x86-64.so.2
#4 0x0000000000000001 in ?? ()
#5 0x00007fffffffe157 in ?? ()
#6 0x0000000000000000 in ?? ()
(gdb)

Any help to narrow this down would be fine. System infos are attached.

[JamesBallard1]

Hi @OSINT_ENGINEER ,

We have tested on Ubuntu 20.04 LTS and not seen this issue before. If you setup the environment and runtime ldd -r on libruntimecore.so, are there any unresolved symbols?

Another thing to check would be to see if there is a secondary install of Qt installed on the system that's getting picked up first in the LD_LIBRARY_PATH vs your build path. We've see strange things like this where Qt 5.13 (for example) is installed on the system but we're building with Qt 5.15 and Qt 5.13 is loaded at runtime.

In your systeminfo.txt file, it shows Qt 5.12.8 libraries, and your shadow build directory is displaying Qt 5.15.2. I would start there and see if a different version of Qt is getting picked up at runtime.

[OSINT_ENGINEER]

I just was lazy and did a 'apt list *libqt5* to find all the qt5 related libraries which were deployed by Ubuntu.

We are running the apps from the terminal and ensure that the LD_LIBRARY_PATH is targetting the Qt 5.15 and the ArcGIS Runtime for Qt directory.

export LD_LIBRARY_PATH=~/Qt/5.15.2/gcc_64/lib:~/arcgis/runtime_sdk/qt100.11/sdk/linux/x64/lib

We attached the ldd -r output and attached the gdb 'info sharedlibrary' output. The gdb shows, that at least the correct Qt and runtime libraries are loaded during execution. But when we set a breakpoint at main using 'break main', the breakpoint is not reached. As far as we can judge it, the libruntimecore.so try to load some library or is missing something on this OS causing the SIGILL.

For the simple Qt Quick App without using ArcGIS Runtime we had to install the necessary OpenGL dev libraries, too.

sudo apt-get install libxcb-randr0-dev libxcb-xtest0-dev libxcb-xinerama0-dev libxcb-shape0-dev libxcb-xkb-dev

Is it possible to get some debug symbols for the libruntimecore.so?

[JamesBallard1]

@OSINT_ENGINEER 

Is it possible to get some debug symbols for the libruntimecore.so?

---

Yes, but that request has to go through Esri support. We are preparing that soon on the team so it should be ready shortly.

I will do some tests on Ubuntu 20.04 to double check our previous testing efforts. In the meantime when you run your tests can you set LD_DEBUG to bindings? You will see every library that is loaded.

LD_DEBUG=bindings (on the command line)

 If anything is not found, I think the LD_DEBUG output might help spot the problem. Also, you'd be able to spot if anything is being loaded from an older version of Qt or anything else that seems off.

[JamesBallard1]

@OSINT_ENGINEER 

I ran some tests on a fresh Ubuntu 20.04 install on our end. In addition to the packages you mentioned I also added mesa-common-dev for libGL (I couldn't build our apps without that). Once all those were in place, I was able to build and run both the Qt Widgets and Qt Quick samples with no crashes.

I am using Runtime 100.11 and Qt 5.15.2 downloaded from the Qt Company's online installer.

[OSINT_ENGINEER]

We were seeing similar issues in the past when one of our core C++ library was compiled using some CPU instructions like avx/avx2 which were not supported on older CPUs. The desktop machine I am using for tests is having a "long-serving" motherboard.

I just downgraded to the ArcGIS Runtime for Qt Version 100.1 (Version 100.10, 100.9 failed too) and voila the Qt Quick Map App appears. Is it possible to get the libruntimecore.so compiled without these CPU optimizations? I know the gcc is somewhat special, I used the ArcGIS Runtime 100.9 with Win10 and VS C++ on this machine before and it worked like a charm.

Similar issues are known for tensorflow running on linux, too.

Tensorflow Illegal instruction on older CPUs 

I have attached the cpu-infos,

[JamesBallard1]

@OSINT_ENGINEER 

Are you able to share your CPU details? AVX aside, it doesn't support SSE3 or SSSE3 which were introduced in 2004 and 2006.

I do believe our library will contain both SSE3 and SSSE3 instructions so you'd need a more modern CPU to support those. I will look into AVX and see if we have this requirement now for 100.11.

---

I used the ArcGIS Runtime 100.9 with Win10 and VS C++ on this machine before and it worked like a charm.

That is using the visual studio compilers which may have different cpu instruction sets enabled.

[OSINT_ENGINEER]

Sure lscpu prints:

Architecture:                    x86_64
CPU op-mode(s):                  32-bit, 64-bit
Byte Order:                      Little Endian
Address sizes:                   48 bits physical, 48 bits virtual
CPU(s):                          4
On-line CPU(s) list:             0-3
Thread(s) per core:              1
Core(s) per socket:              4
Socket(s):                       1
NUMA node(s):                    1
Vendor ID:                       AuthenticAMD
CPU family:                      16
Model:                           5
Model name:                      AMD Athlon(tm) II X4 640 Processor
Stepping:                        3
CPU MHz:                         3000.000
CPU max MHz:                     3000,0000
CPU min MHz:                     800,0000
BogoMIPS:                        6000.00
Virtualization:                  AMD-V
L1d cache:                       256 KiB
L1i cache:                       256 KiB
L2 cache:                        2 MiB
NUMA node0 CPU(s):               0-3
Vulnerability Itlb multihit:     Not affected
Vulnerability L1tf:              Not affected
Vulnerability Mds:               Not affected
Vulnerability Meltdown:          Not affected
Vulnerability Spec store bypass: Not affected
Vulnerability Spectre v1:        Mitigation; usercopy/swapgs barriers and __user
                                  pointer sanitization
Vulnerability Spectre v2:        Mitigation; Full AMD retpoline, STIBP disabled,
                                  RSB filling
Vulnerability Srbds:             Not affected
Vulnerability Tsx async abort:   Not affected
Flags:                           fpu vme de pse tsc msr pae mce cx8 apic sep mtr
                                 r pge mca cmov pat pse36 clflush mmx fxsr sse s
                                 se2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtsc
                                 p lm 3dnowext 3dnow constant_tsc rep_good nopl 
                                 nonstop_tsc cpuid extd_apicid pni monitor cx16 
                                 popcnt lahf_lm cmp_legacy svm extapic cr8_legac
                                 y abm sse4a misalignsse 3dnowprefetch osvw ibs 
                                 skinit wdt hw_pstate vmmcall npt lbrv svm_lock 
                                 nrip_save

[OSINT_ENGINEER]

We have to stay with ArcGIS Runtime 100.4 Version when addressing this outmoded working station.