Native libraries (libzen) do not load (x86_64)

Support for Synology NAS, QNAP NAS and other Embedded Linux systems
Post Reply
Pator
Power User
Posts: 15
Joined: 04 Oct 2016, 07:29

Native libraries (libzen) do not load (x86_64)

Post by Pator »

Code: Select all

System: QNAP 453A x86_64 GNU/Linux Celeron CPU  N3150  
filebot_4.7.2_noarch.ipk + latest jar

Code: Select all

FileBot 4.7.2 (r4334)
JNA Native: 4.0.1
MediaInfo: java.lang.UnsatisfiedLinkError: Unable to load library 'zen': Native library (linux-x86/libzen.so) not found in resource path ([file:/share/CACHEDEV1_DATA/.qpkg/Entware-ng/share/filebot/FileBot.jar])
Apache Commons VFS: [zip, ftps, rar, ftp, sftp]
Chromaprint: java.io.IOException: Cannot run program "fpcalc": error=2, No such file or directory
Extended Attributes: OK
Groovy Engine: 2.4.7
JRE: Java(TM) SE Embedded Runtime Environment 1.8.0_65 (headless)
JVM: 32-bit Java HotSpot(TM) Embedded Server VM
CPU/MEM: 4 Core / 860 MB Max Memory / 16 MB Used Memory
OS: Linux (i386)
Package: IPKG
Data: /opt/share/filebot/data/admin
uname: Linux NASFB0BD7 3.19.8 #1 SMP Thu Sep 1 07:54:20 CST 2016 x86_64 GNU/Linux
Done ヾ(@⌒ー⌒@)ノ

Things I tried:
1. Downloaded the native libraries libzen.so and libmediainfo.so for linux an placed them in the current dir, in a dir defined by the LD_LIBRARY_PATH, in the APP_ROOT (/opt/share/filebot/), in APP_ROOT/linux-x86/ always get the same UnsatisfiedLinkError. Wondering if those libs were wrong for my system (tried both 32 and 64 bits) downloaded fpcalc (again both versions). 64bit. They were.

Code: Select all

./fpcalc.linux-amd64: error while loading shared libraries: libstdc++.so.6: wrong ELF class: ELFCLASS32
./fpcalc.linux-i686: /lib/libc.so.6: version `GLIBC_2.7' not found (required by ./fpcalc.linux-i686)
# opkg info glib2
Package: glib2
Version: 2.46.2-1
2. So decided to compile my own libraries on the QNAP. Downloaded from
http://mediaarea.net/download/binary/li ... rce.tar.xz
Very straight forward. Compiled and installed under --prefix=/opt/. Same error as above. with a copy of both .so next to filebot.jar same error. as if these libraries are never found

Code: Select all

/opt/share/filebot/FileBot.jar:     Java archive data (JAR)
/opt/share/filebot/libmediainfo.so: ELF 64-bit LSB shared object, x86-64, version 1 (GNU/Linux), dynamically linked, not stripped
/opt/share/filebot/libzen.so:       ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped
But it is actually found, if I run a strace I see:

Code: Select all

[pid 16878] stat64("/usr/local/lib/libzen.so", 0xf6c0723c) = -1 ENOENT (No such file or directory)
[pid 16878] lstat64("/lib64/libzen.so", 0xf6c04b1c) = -1 ENOENT (No such file or directory)
[pid 16878] stat64("/lib64/libzen.so", 0xf6c0723c) = -1 ENOENT (No such file or directory)
[pid 16878] open("/share/CACHEDEV1_DATA/.qpkg/JRE/jre/bin/../lib/i386/jli/libzen.so", O_RDONLY) = -1 ENOENT (No such file or directory)
[pid 16878] open("/share/CACHEDEV1_DATA/.qpkg/JRE/jre/bin/../lib/i386/libzen.so", O_RDONLY) = -1 ENOENT (No such file or directory)
[pid 16878] open("/opt/share/filebot/libzen.so", O_RDONLY) = 18
[pid 16878] open("/lib/libzen.so", O_RDONLY) = -1 ENOENT (No such file or directory)
[pid 16878] open("/usr/lib/libzen.so", O_RDONLY) = -1 ENOENT (No such file or directory)
[pid 16878] write(1, "MediaInfo: java.lang.Unsatisfied"..., 211) = 211
in fact it seams not to like it

Code: Select all

[pid 16878] open("/opt/share/filebot/libzen.so", O_RDONLY) = 18                                                               
[pid 16878] read(18, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\200\6\1\0\0\0\0\0"..., 512) = 512                        
[pid 16878] close(18)                   = 0                                                                                   
[pid 16878] open("/etc/ld.so.cache", O_RDONLY) = 18                                                                           
[pid 16878] fstat64(18, {st_mode=S_IFREG|0644, st_size=31149, ...}) = 0                                                       
[pid 16878] mmap2(NULL, 31149, PROT_READ, MAP_PRIVATE, 18, 0) = 0xf3a06000                                                    
[pid 16878] close(18)                   = 0                                                                                   
[pid 16878] open("/lib/libzen.so", O_RDONLY) = -1 ENOENT (No such file or directory)                                          
[pid 16878] open("/usr/lib/libzen.so", O_RDONLY) = -1 ENOENT (No such file or directory)                                      
[pid 16878] munmap(0xf3a06000, 31149)   = 0             


do I need to compile a 32bit version?








2.
Last edited by Pator on 05 Oct 2016, 07:41, edited 1 time in total.
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Native libraries (libzen) do not loead (x86_64)

Post by rednoah »

Pator wrote:do I need to compile a 32bit version?
Since you're using 32-bit Java you'll need 32-bit native libraries. But why are you using 32-bit Java? 64-bit libraries fail because you're using 32-bit Java. 32-bit libraries fail because you're using a 64-bit OS without the 32-bit support libraries.


You're using a 64-bit OS so you should be using 64-bit Java and 64-bit native libraries.


From the FAQ:
FAQ wrote:Q: Why is MediaInfo not working? Why is 7zip not working?
If you're using 64-bit Java you must install 64-bit FileBot, if you're using 32-bit Java you must install 32-bit FileBot. If you have installed 64-bit Windows and 64-bit FileBot and 32-bit Java, the java process will be in 32-bit compatibility mode and the native libraries won't work. So uninstall 32-bit Java and then only install the latest 64-bit Java and latest 64-bit FileBot.
PS: How do you trace filesystem access!? I had no idea you could do that out-of-the-box! :lol:
:idea: Please read the FAQ and How to Request Help.
Pator
Power User
Posts: 15
Joined: 04 Oct 2016, 07:29

Re: Native libraries (libzen) do not loead (x86_64)

Post by Pator »

Java embedded is only 32bit.
http://www.oracle.com/technetwork/java/ ... 09751.html

I will need to see how to compile the libs for 32bit, or try a full JRE 64bit
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Native libraries (libzen) do not loead (x86_64)

Post by rednoah »

Why do you need Java Embedded? If you have less than 16 MB memory FileBot won't run anyway.

QNAP 453A should have 512 MB memory so I see no reason why wouldn't just use the standard 64-bit JDK:
http://www.oracle.com/technetwork/java/ ... index.html

This should make it easy, though I haven't bothered with making a QNAP package yet:
https://github.com/rednoah/java-install ... on-via-ssh
:idea: Please read the FAQ and How to Request Help.
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Native libraries (libzen) do not loead (x86_64)

Post by rednoah »

Pator wrote:

Code: Select all

[pid 16878] open("/opt/share/filebot/libzen.so", O_RDONLY) = 18
How do you get output like that? Can you do that with standard Linux tools?
:idea: Please read the FAQ and How to Request Help.
Pator
Power User
Posts: 15
Joined: 04 Oct 2016, 07:29

Re: Native libraries (libzen) do not loead (x86_64)

Post by Pator »

rednoah wrote:
Pator wrote:

Code: Select all

[pid 16878] open("/opt/share/filebot/libzen.so", O_RDONLY) = 18
How do you get output like that? Can you do that with standard Linux tools?
Hello Noah,

yes, you can see all the system calls on linux with strace (system trace)

Code: Select all

strace <your_command>
In this case I needed -f to trace also child processed (or it would trace only the bash wrapper)
You can also attach it to an already running process. Very handy!
It comes standard with most linux distributions. Not in my QNAP, but I use entware-ng (optware replacement), a package manager for many embedded systems, and I installed it from there (together with gcc, make, etc)

Back to your proposal, I downloaded the JRE x64. Unfortunately this the QNAP OS (QTS4.2) has a partial x64 libs and java failed to load.

Code: Select all

# ./jre1.8.0_101/bin/java --version
./jre1.8.0_101/bin/java: error while loading shared libraries: libdl.so.2: wrong ELF class: ELFCLASS32
My options
1. Java32 (part of QTS4.2) + 32bit libzen
2. find out if I can install (or compile) a full x64 environment
3. I discovered today that QTS 4.3 alpha/preview is out, The main new feature is a full x64 environment!
http://community.benchmarkemail.com/use ... -3-0-now--

1 is ugly, 2 is probably too much work, 3 is risky but if it works, optimal.

I will go for 3 and will post my results here
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Native libraries (libzen) do not loead (x86_64)

Post by rednoah »

1.
You ARE running 64-bit Linux:

Code: Select all

Linux NASFB0BD7 3.19.8 #1 SMP Thu Sep 1 07:54:20 CST 2016 x86_64 GNU/Linux
The mere fact that your OS is running tells us that you have all the necessary 64-bit native libraries. ;)


2.
If a 64-bit java process is trying to load 32-bit system libraries then that is indeed very strange. Bad LD_LIBRARY_PATH would be one of the usual suspects.

@see http://www.tldp.org/HOWTO/Program-Libra ... aries.html
:idea: Please read the FAQ and How to Request Help.
Pator
Power User
Posts: 15
Joined: 04 Oct 2016, 07:29

Re: Native libraries (libzen) do not load (x86_64)

Post by Pator »

Hello Noah,

QTS4.2 on 64 systems have in fact a mix of 32 and 64 libraries, with many important 64bit libraries missing. I did not research that too deep but you could see in the forums ppl creating x64libs packages to cope with the lack of them.

The good news is the prealpha QTS4.3 ended up being pretty stable and much faster
All apps from entware-ng have no problems
There could be some problems with official QNAP apps still
ie: the QNAP x64 beta version of plex has currently problems playing some formats. (I use kodi, and has 0 problems)
but:

1. JRE updated to x64 on its own.
2. Now your native fpcalc.linux-amd64 runs!
3. Left my compiled libzen & medialib (presume your native 64 would load too)

The only problem I had was that filebot was not finding fpcalc (not in ./ and not in the same dir as filebot.jar)
But I checked with strace where filebot was looking for and moved it to one of those dirs (I chose /opt/bin/)

Code: Select all

FileBot 4.7.2 (r4334)
JNA Native: 4.0.1
MediaInfo: MediaInfoLib - v0.7.89
Apache Commons VFS: [zip, ftps, rar, ftp, sftp]
Chromaprint: fpcalc version 1.1.0 (fpcalc)
Extended Attributes: OK
Groovy Engine: 2.4.7
JRE: Java(TM) SE Runtime Environment 1.8.0_101 (headless)
JVM: 64-bit Java HotSpot(TM) 64-Bit Server VM
CPU/MEM: 4 Core / 860 MB Max Memory / 20 MB Used Memory
OS: Linux (amd64)
Package: IPKG
Data: /opt/share/filebot/data/admin
uname: Linux NASFB0BD7 4.2.8 #1 SMP Fri Aug 26 01:09:54 CST 2016 x86_64 GNU/Linux
Done ヾ(@⌒ー⌒@)ノ
:) Thanks

And for those with similar problems in the future, upgrade to QTS4.3 when it becomes available
viceversa
Posts: 6
Joined: 05 Oct 2016, 19:34

Re: Native libraries (libzen) do not load (x86_64)

Post by viceversa »

Thank you very much for getting to the bottom of this issue!!

Along with upgrading to the very alpha QTS 4.3, did you then install the full JRE 64 bit or did 4.3 come with that? I'm trying to note what else is needed after upgrading to 4.3 to get all filebot features working via entware on the QNAP.
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Native libraries (libzen) do not load (x86_64)

Post by rednoah »

I recommend installing the latest Oracle JDK:
https://github.com/rednoah/java-install ... on-via-ssh
:idea: Please read the FAQ and How to Request Help.
viceversa
Posts: 6
Joined: 05 Oct 2016, 19:34

Re: Native libraries (libzen) do not load (x86_64)

Post by viceversa »

I thought I would add on to this thread to attempt to keep this as a more central help spot for others with focus to QNAP devices....

Well, I updated my QNAP today to 4.3 which is now a full 64 bit environment so I thought I'd give a crack at getting mediainfo to work. My QNAP is a TS-451+ with Intel processor and 64 bit kernel and use entware-ng for package management (like the OP of this thread). I gathered the compiled libraries from https://github.com/filebot/filebot/tree ... inux-amd64

and placed them into the needed directory:

APP_ROOT="/opt/share/filebot"
LD_LIBRARY_PATH: /opt/share/filebot

I also needed to place fpcalc over in "/opt/bin/" and filebot seems to pick it up just fine. libzen also appears to be picked up by filebot (as its not complaining about it missing when I run "filebot -script fn:sysinfo").

What my current problem is i'm running into an exception when trying to load libmediainfo.so

Code: Select all

[/opt/share/filebot/bin] # filebot -script fn:sysinfo
FileBot 4.7.2 (r4178)
JNA Native: 4.0.1
MediaInfo: net.filebot.mediainfo.MediaInfoException: Unable to load amd64 (64-bit) native library libmediainfo.so: Unable to load library 'mediainfo': Native library (linux-x86-64/libmediainfo.so) not found in resource path ([file:/share/CACHEDEV1_DATA/.qpkg/Entware-ng/share/filebot/FileBot.jar])
Apache Commons VFS: [zip, rar]
Chromaprint: 1.1.0
Extended Attributes: OK
Script Bundle: 2016-11-07 (r458)
Groovy: 2.4.7
JRE: Java(TM) SE Runtime Environment 1.8.0_101 (headless)
JVM: 64-bit Java HotSpot(TM) 64-Bit Server VM
CPU/MEM: 4 Core / 3 GB Max Memory / 16 MB Used Memory
OS: Linux (amd64)
Package: IPKG
Data: /share/CACHEDEV1_DATA/.qpkg/Entware-ng/share/filebot/data/admin
uname: Linux qfano 4.2.8 #1 SMP Sun Dec 11 01:13:24 CST 2016 x86_64 GNU/Linux
Following suit from op, running a strace -f filebot -script fn:sysinfo yeilds this:

Code: Select all

[pid 18062] open("/share/CACHEDEV1_DATA/.qpkg/JRE/jre/bin/../lib/amd64/jli/libmediainfo.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
[pid 18062] open("/share/CACHEDEV1_DATA/.qpkg/JRE/jre/bin/../lib/amd64/libmediainfo.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
[pid 18062] open("/opt/share/filebot/libmediainfo.so", O_RDONLY|O_CLOEXEC) = 17
[pid 18062] lstat("/lib/libmediainfo.so", 0x7f2afa00a670) = -1 ENOENT (No such file or directory)
[pid 18062] stat("/lib64/libmediainfo.so", 0x7f2afa00d3c0) = -1 ENOENT (No such file or directory)
[pid 18062] lstat("/usr/lib/libmediainfo.so", 0x7f2afa00a670) = -1 ENOENT (No such file or directory)
[pid 18062] stat("/usr/lib/libmediainfo.so", 0x7f2afa00d3c0) = -1 ENOENT (No such file or directory)
[pid 18062] lstat("/lib/libmediainfo.so", 0x7f2afa00a670) = -1 ENOENT (No such file or directory)
[pid 18062] stat("/lib/libmediainfo.so", 0x7f2afa00d3c0) = -1 ENOENT (No such file or directory)
[pid 18062] lstat("/usr/local/lib/libmediainfo.so", 0x7f2afa00a670) = -1 ENOENT (No such file or directory)
[pid 18062] stat("/usr/local/lib/libmediainfo.so", 0x7f2afa00d3c0) = -1 ENOENT (No such file or directory)
[pid 18062] open("/share/CACHEDEV1_DATA/.qpkg/JRE/jre/bin/../lib/amd64/jli/libmediainfo.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
[pid 18062] open("/share/CACHEDEV1_DATA/.qpkg/JRE/jre/bin/../lib/amd64/libmediainfo.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
[pid 18062] open("/opt/share/filebot/libmediainfo.so", O_RDONLY|O_CLOEXEC) = 17
[pid 18062] write(1, "MediaInfo: net.filebot.mediainfo"..., 298MediaInfo: net.filebot.mediainfo.MediaInfoException: Unable to load amd64 (64-bit) native library libmediainfo.so: Unable to load library 'mediainfo': Native library (linux-x86-64/libmediainfo.so) not found in resource path ([file:/share/CACHEDEV1_DATA/.qpkg/Entware-ng/share/filebot/FileBot.jar])
Any help on the cause for the exception is greatly appreciated!
viceversa
Posts: 6
Joined: 05 Oct 2016, 19:34

Re: Native libraries (libzen) do not load (x86_64)

Post by viceversa »

I would like to try and compile these libraries myself and am having difficulty figuring out how to do that. Can anyone give me some tips on how to do so using entware-ng and I am willing to give that a try.
WeaponX
Posts: 1
Joined: 23 Dec 2016, 16:43

Re: Native libraries (libzen) do not load (x86_64)

Post by WeaponX »

I downloaded the older version 0.7.69 and everything ran fine for me after that. Hope that helps.
viceversa
Posts: 6
Joined: 05 Oct 2016, 19:34

Re: Native libraries (libzen) do not load (x86_64)

Post by viceversa »

WeaponX wrote:I downloaded the older version 0.7.69 and everything ran fine for me after that. Hope that helps.
reverted to libmediainfo 0.7.69 and its working now! Thank you very much for the suggestion!

For anyone else, these older versions can be found here: https://github.com/filebot/filebot/comm ... d503e6c691
Post Reply