Filebot hangs when matching on Ubuntu

Support for Ubuntu and other Desktop Linux distributions
Post Reply
Kepesk
Posts: 1
Joined: 12 Nov 2018, 18:59

Filebot hangs when matching on Ubuntu

Post by Kepesk »

I have run into an intermittent freeze when matching titles. It happens maybe a third of the time, and is repeatable if I retry a specific search. I've tried anything I could find with on the forums with no effect.

There are no relevant errors in ~/.filebot/error.log

What I've tried
Updating Filebot
Reinstalling Filebot
Reinstalling Java
Switching databases (this made it hang on different searches, but it still hangs a third of the time)

Here's my filebot -script fn:sysinfo

Code: Select all

FileBot 4.8.2 (r5789)
JNA Native: 5.2.2
MediaInfo: net.filebot.mediainfo.MediaInfoException: Unable to load amd64 (64-bit) native library libmediainfo.so: Unable to load library 'zen': Native library (linux-x86-64/libzen.so) not found in resource path (/usr/share/filebot/jar/filebot.jar)
7-Zip-JBinding: 9.20
Chromaprint: 1.4.2
Extended Attributes: OK
Unicode Filesystem: OK
GVFS: PlatformGVFS [/run/user/1000/gvfs]
Script Bundle: 2018-10-05 (r536)
Groovy: 2.5.1
JRE: Java(TM) SE Runtime Environment 10.0.2
JVM: 64-bit Java HotSpot(TM) 64-Bit Server VM
CPU/MEM: 4 Core / 1 GB Max Memory / 36 MB Used Memory
OS: Linux (amd64)
HW: Linux mordalfus 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9 14:43:09 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
DATA: /home/jacob/.filebot
Package: DEB
License: FileBot License P6157096 (Valid-Until: 2019-11-12)
Done \u30fe(\uff20\u2312\u30fc\u2312\uff20)\u30ce
Could someone point me in the right direction?

Thanks!
User avatar
rednoah
The Source
Posts: 22985
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Filebot hangs when matching on Ubuntu

Post by rednoah »

Sorry, no idea. Probably a problem specific to Linux / Gnome / KDE / etc on your distribution. I'll have a look. What exactly are you using? (screenshots are very helpful here)

Forcing Nimbus Look-and-Feel might work around the issue, whatever it may be:
viewtopic.php?f=9&t=5747

Code: Select all

export JAVA_OPTS="-Dswing.systemlaf=javax.swing.plaf.nimbus.NimbusLookAndFeel"


:idea: If you're an advanced user, then you can use VisualVM to create a heap / thread dump so we can see exactly where each application thread is stuck.
:idea: Please read the FAQ and How to Request Help.
goofy183
Posts: 6
Joined: 24 Nov 2018, 02:53

Re: Filebot hangs when matching on Ubuntu

Post by goofy183 »

I'm seeing similar behavior.

Code: Select all

$ filebot -script fn:sysinfo
FileBot 4.8.2 (r5789)
JNA Native: 5.2.2
MediaInfo: net.filebot.mediainfo.MediaInfoException: Unable to load amd64 (64-bit) native library libmediainfo.so: Unable to load library 'mediainfo': Native library (linux-x86-64/libmediainfo.so) not found in resource path (/usr/share/filebot/jar/filebot.jar)
7-Zip-JBinding: 9.20
Chromaprint: 1.4.2
Extended Attributes: OK
Unicode Filesystem: OK
GVFS: PlatformGVFS [/run/user/1000/gvfs]
Script Bundle: 2018-10-05 (r536)
Groovy: 2.5.1
JRE: Java(TM) SE Runtime Environment 10.0.2
JVM: 64-bit Java HotSpot(TM) 64-Bit Server VM
CPU/MEM: 8 Core / 3 GB Max Memory / 55 MB Used Memory
OS: Linux (amd64)
HW: Linux goofy183-vm 4.15.0-36-generic #39~16.04.1-Ubuntu SMP Tue Sep 25 08:59:23 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
DATA: /home/goofy183/.filebot
Package: DEB
License: FileBot License P4416814 (Valid-Until: 2019-09-01)
Done ヾ(@⌒ー⌒@)ノ
I grabbed a thread dump: https://pastebin.com/6iA2j4zL

The main thread is stuck "- waiting to lock <0x00000006c57d0470> (a net.filebot.ui.MainFrame)" which is currently owned by "pool-2-thread-2". That pool thread appears to be waiting in some swing/AWT event queue at net.filebot.ui.rename.EpisodeListMatcher.selectSearchResult(EpisodeListMatcher.java:283)

This isn't technically a true deadlock but it is effectively a deadlock. I have no idea what the code is actually doing but my very naive guess it that "EpisodeListMatcher.selectSearchResult" should be releasing its lock on net.filebot.ui.MainFrame before waiting in the AWT event queue?

Let me know if you need more information (or would like this in a new thread)
Last edited by goofy183 on 24 Nov 2018, 03:52, edited 1 time in total.
goofy183
Posts: 6
Joined: 24 Nov 2018, 02:53

Re: Filebot hangs when matching on Ubuntu

Post by goofy183 »

I just tried the NimbusLookAndFeel and while it doesn't fix the lock it does render the dialog before it locks up. It appears to happen with the following flow:

1. Select folder of files
2. Select Presets > Plex (this is my custom Plex/TVDB preset)
3. Get "Failed to identify some of the following files:" dialog with an option to select from multiple shows
4. Select the best match and click Select button
5. Get another "Failed to identify some of the following files:" dialog with just one of the original files listed

That last dialog is frozen and at this point I have to force-kill the JVM.

If I remove the one file that is listed in step 5 from the "Original Files" list then everything works.
User avatar
rednoah
The Source
Posts: 22985
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Filebot hangs when matching on Ubuntu

Post by rednoah »

Thanks for the detailed logs. This particular issue has already been fixed, but you'll have to wait for the next release (hopefully ready by the end of this year) for the fix.

:idea: PM me if you're interested in early beta packages, because I'm looking for testers on various Linux distros anyway to give the new packages a spin.
:idea: Please read the FAQ and How to Request Help.
Wallentini
Posts: 3
Joined: 01 Mar 2019, 16:35

Re: Filebot hangs when matching on Ubuntu

Post by Wallentini »

Hi, new to the forum, fan of the FileBot function :)

Have encountered the issue of FileBot freezing my server, FileBot 4.8.2 headless on Ubuntu 18.04.
Have since updated to the available beta release 4.8.5 (r6064), I can confirm that it too contains the bug.
I'm using Java SE Runtime Environment 11.0.2

Looking forward to additional releases and can be of assistance with testing on Ubuntu.

Regards,
User avatar
rednoah
The Source
Posts: 22985
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Filebot hangs when matching on Ubuntu

Post by rednoah »

Please take a screenshot so we can see where it freezes. And if possible, a stack dump would be extremely helpful as well so we can see which thread is locked up where.
:idea: Please read the FAQ and How to Request Help.
Wallentini
Posts: 3
Joined: 01 Mar 2019, 16:35

Re: Filebot hangs when matching on Ubuntu

Post by Wallentini »

Thank you for your response :)

I must start by admitting that I might have jumped the gun on this one. I first started to experience the issue after upgrading from FileBot 4.7.9 to 4.8.2. Found this thread which error description matched my situation. Upgraded to the mentioned beta version 4.8.5 (r6064) and the situation persisted, hence I have not confirmed it to be the very same issue as presented by goofy183.

The information that I can provide at this point is the following:
From looking in to it a bit more closely, especially the dmesg content, it does look to be related to the FileBot copying action and the cifsd daemon.

Information about my setup:
[vm_mediaserver]
Presenting a share called "mediaserver".
[vm_plex]
Accessing the share "mediaserver" over cifs while hosting an instance of Transmission and FileBot.


FileBot-script

Code: Select all

#!/bin/sh -xu

# DATE: 2019-02-09
# VER:  1.0.4

# Input Parameters
ARG_PATH="$TR_TORRENT_DIR/$TR_TORRENT_NAME"
ARG_NAME="$TR_TORRENT_NAME"
ARG_LABEL="N/A"

# Configuration
CONFIG_OUTPUT="/media/mediaserver"
CONFIG_INPUT="/media/mediaserver/downloads/complete"
CONFIG_LOG="/var/log/filebot.log"
CONFIG_EXCLUDE="/etc/transmission-daemon/filebot.exclude"

# Definitions
DEF_PLEX="127.0.0.1:TOKEN-HAS-BEEN-REMOVED"
DEF_SERIES="/media/mediaserver/serier/{ny}/{n} - Season {s}/{n} - {s00e00} - {t}/{n.upperInitial().space('.')}.{s00e00}{'.'+t.upperInitial().space('.')}{'.'+source}{'.'+vf}{'.'+vc}{'-'+group}"
DEF_MOVIES="/media/mediaserver/filmer/{ny}/{n.upperInitial().space('.')}.{y}{'.'+source}{'.'+vf}{'.'+vc}{'-'+group}"

/usr/bin/filebot -extract -script fn:amc --output "$CONFIG_OUTPUT" --action copy --conflict skip -non-strict --log all --log-file "$CONFIG_LOG" "$CONFIG_INPUT" --def excludeList="$CONFIG_EXCLUDE" --def plex="$DEF_PLEX" --def seriesFormat="$DEF_SERIES" movieFormat="$DEF_MOVIES"

Strace of FileBot-script running
Note: The run is resulting in the error state of the vm and me killing the process.

Code: Select all

execve("/bin/sh", ["sh", "/etc/transmission-daemon/filebot"...], 0x7ffe747bb9f8 /* 30 vars */) = 0
brk(NULL)                               = 0x560ba098e000
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
access("/etc/ld.so.preload", R_OK)      = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=35451, ...}) = 0
mmap(NULL, 35451, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7fafcade1000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\260\34\2\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=2030544, ...}) = 0
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fafcaddf000
mmap(NULL, 4131552, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fafca7d2000
mprotect(0x7fafca9b9000, 2097152, PROT_NONE) = 0
mmap(0x7fafcabb9000, 24576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1e7000) = 0x7fafcabb9000
mmap(0x7fafcabbf000, 15072, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7fafcabbf000
close(3)                                = 0
arch_prctl(ARCH_SET_FS, 0x7fafcade0540) = 0
mprotect(0x7fafcabb9000, 16384, PROT_READ) = 0
mprotect(0x560b9fcd0000, 8192, PROT_READ) = 0
mprotect(0x7fafcadea000, 4096, PROT_READ) = 0
munmap(0x7fafcade1000, 35451)           = 0
getuid()                                = 1000
getgid()                                = 1000
getpid()                                = 9560
rt_sigaction(SIGCHLD, {sa_handler=0x560b9fac7200, sa_mask=~[RTMIN RT_1], sa_flags=SA_RESTORER, sa_restorer=0x7fafca810f20}, NULL, 8) = 0
geteuid()                               = 1000
brk(NULL)                               = 0x560ba098e000
brk(0x560ba09af000)                     = 0x560ba09af000
getppid()                               = 9558
stat("/home/plex", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
stat(".", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
openat(AT_FDCWD, "/etc/transmission-daemon/filebot-transmission-postprocess_bkp.sh", O_RDONLY) = 3
fcntl(3, F_DUPFD, 10)                   = 10
close(3)                                = 0
fcntl(10, F_SETFD, FD_CLOEXEC)          = 0
geteuid()                               = 1000
getegid()                               = 1000
rt_sigaction(SIGINT, NULL, {sa_handler=SIG_DFL, sa_mask=[], sa_flags=0}, 8) = 0
rt_sigaction(SIGINT, {sa_handler=0x560b9fac7200, sa_mask=~[RTMIN RT_1], sa_flags=SA_RESTORER, sa_restorer=0x7fafca810f20}, NULL, 8) = 0
rt_sigaction(SIGQUIT, NULL, {sa_handler=SIG_DFL, sa_mask=[], sa_flags=0}, 8) = 0
rt_sigaction(SIGQUIT, {sa_handler=SIG_DFL, sa_mask=~[RTMIN RT_1], sa_flags=SA_RESTORER, sa_restorer=0x7fafca810f20}, NULL, 8) = 0
rt_sigaction(SIGTERM, NULL, {sa_handler=SIG_DFL, sa_mask=[], sa_flags=0}, 8) = 0
rt_sigaction(SIGTERM, {sa_handler=SIG_DFL, sa_mask=~[RTMIN RT_1], sa_flags=SA_RESTORER, sa_restorer=0x7fafca810f20}, NULL, 8) = 0
read(10, "#!/bin/sh -xu\n\n# DATE: 2019-02-0"..., 8192) = 1001
clone(child_stack=NULL, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x7fafcade0810) = 9561
wait4(-1, 0x7fff7e80292c, 0, NULL)      = ? ERESTARTSYS (To be restarted if SA_RESTART is set)
--- SIGINT {si_signo=SIGINT, si_code=SI_KERNEL} ---
rt_sigreturn({mask=[]})                 = -1 EINTR (Interrupted system call)
wait4(-1, 0x7fff7e80292c, 0, NULL)      = ? ERESTARTSYS (To be restarted if SA_RESTART is set)
--- SIGINT {si_signo=SIGINT, si_code=SI_KERNEL} ---
rt_sigreturn({mask=[]})                 = -1 EINTR (Interrupted system call)
wait4(-1, [{WIFSIGNALED(s) && WTERMSIG(s) == SIGINT}], 0, NULL) = 9561
--- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_KILLED, si_pid=9561, si_uid=1000, si_status=SIGINT, si_utime=0, si_stime=0} ---
rt_sigreturn({mask=[]})                 = 9561
rt_sigprocmask(SIG_SETMASK, [], [], 8)  = 0
rt_sigaction(SIGINT, {sa_handler=SIG_DFL, sa_mask=[INT], sa_flags=SA_RESTORER|SA_RESTART, sa_restorer=0x7fafca810f20}, {sa_handler=0x560b9fac7200, sa_mask=~[KILL STOP RTMIN RT_1], sa_flags=SA_RESTORER, sa_restorer=0x7fafca810f20}, 8) = 0
rt_sigprocmask(SIG_BLOCK, ~[RTMIN RT_1], [], 8) = 0
getpid()                                = 9560
gettid()                                = 9560
tgkill(9560, 9560, SIGINT)              = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
--- SIGINT {si_signo=SIGINT, si_code=SI_TKILL, si_pid=9560, si_uid=1000} ---
+++ killed by SIGINT +++

Dmesg of the vm in the error state

Code: Select all

[83539.697281] general protection fault: 0000 [#1] SMP PTI
[83539.697344] Modules linked in: cmac arc4 md4 nls_utf8 cifs ccm fscache joydev input_leds serio_raw intel_rapl_perf vboxvideo(CE) snd_intel8x0 snd_ac97_codec ttm ac97_bus drm_kms_helper vboxguest snd_pcm snd_timer drm snd mac_hid soundcore fb_sys_fops syscopyarea sysfillrect sysimgblt sch_fq_codel ib_iser rdma_cm iw_cm ib_cm ib_core iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi ip_tables x_tables autofs4 btrfs zstd_compress raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear hid_generic usbhid hid crct10dif_pclmul crc32_pclmul ghash_clmulni_intel pcbc aesni_intel aes_x86_64 crypto_simd glue_helper cryptd psmouse ahci libahci i2c_piix4 e1000 pata_acpi video
[83539.697975] CPU: 1 PID: 1379 Comm: cifsd Tainted: G         C  E    4.15.0-45-generic #48-Ubuntu
[83539.698046] Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
[83539.698140] RIP: 0010:prefetch_freepointer+0x15/0x30
[83539.698185] RSP: 0018:ffffad4c820e7da0 EFLAGS: 00010282
[83539.698232] RAX: 0000000000000000 RBX: e7752587f8e78be9 RCX: 00000000000b7609
[83539.698291] RDX: 00000000000b7608 RSI: e7752587f8e78be9 RDI: ffff9634bd8ad800
[83539.698349] RBP: ffffad4c820e7da0 R08: ffff9634bfcab450 R09: 0000000000000000
[83539.698408] R10: 00000000000002b7 R11: 00000000000002a1 R12: 0000000001011200
[83539.698467] R13: ffff9634bd8ad800 R14: ffff9634b6081180 R15: ffff9634bd8ad800
[83539.698527] FS:  0000000000000000(0000) GS:ffff9634bfc80000(0000) knlGS:0000000000000000
[83539.698593] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[83539.698642] CR2: 00007fd07c0afc10 CR3: 000000000bc0a003 CR4: 00000000000606e0
[83539.698707] Call Trace:
[83539.698742]  kmem_cache_alloc+0xa2/0x1b0
[83539.698794]  ? mempool_alloc_slab+0x15/0x20
[83539.698850]  ? wait_woken+0x80/0x80
[83539.698887]  mempool_alloc_slab+0x15/0x20
[83539.698927]  mempool_alloc+0x71/0x190
[83539.698995]  cifs_small_buf_get+0x1a/0x30 [cifs]
[83539.699058]  cifs_demultiplex_thread+0x5c7/0xb30 [cifs]
[83539.699117]  ? __schedule+0x299/0x8a0
[83539.699158]  kthread+0x121/0x140
[83539.699210]  ? cifs_handle_standard+0x190/0x190 [cifs]
[83539.699259]  ? kthread_create_worker_on_cpu+0x70/0x70
[83539.699307]  ? do_syscall_64+0x73/0x130
[83539.700880]  ? SyS_exit_group+0x14/0x20
[83539.702325]  ret_from_fork+0x35/0x40
[83539.703753] Code: eb bb 49 8b 74 24 60 48 c7 c7 c0 cf 0e 95 e8 13 b9 ea ff eb 90 90 0f 1f 44 00 00 55 48 85 f6 48 89 e5 74 14 48 63 47 20 48 01 c6 <48> 33 36 48 33 b7 40 01 00 00 0f 18 0e 5d c3 66 90 66 2e 0f 1f 
[83539.706977] RIP: prefetch_freepointer+0x15/0x30 RSP: ffffad4c820e7da0
[83539.708232] ---[ end trace fece81a60a85c75b ]---

I hope that this information if of use to you, please advice if I can provide any additional information.
User avatar
rednoah
The Source
Posts: 22985
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Filebot hangs when matching on Ubuntu

Post by rednoah »

I see, I had assumed you're talking about the GUI freezing.

What does the filebot log say? What is the last line that is printed before it stops printing more console output?

The CLI can't freeze in that sense, as far as FileBot is concerned, it is however very possible that underlying OS calls just take very long or forever. Especially if the log / console output stops at a filesystem operation, then probably nothing we can do. Not something that can be fixed by making code changes to FileBot. If you're using VirtualBox, maybe tweaking some settings might resolve the issue though.
:idea: Please read the FAQ and How to Request Help.
Wallentini
Posts: 3
Joined: 01 Mar 2019, 16:35

Re: Filebot hangs when matching on Ubuntu

Post by Wallentini »

My apologies, for not more clearly pointing out that I'm running headless, in a thread focused on GUI, as well as jumping the gun without prior confirmation.

Just as you suspected the issue at hand is not due to FileBot but my underlying system.
Further digging following the information gathered while writing my last post indicated a kernel issue and I have now implemented the the presented workaround and confirmed normal function.
Completely off topic in this thread, but the fix was downgrading the smb version in my mount.

Previous fstab entry (vers=3.0):

Code: Select all

//192.168.0.20/mediaserver      /media/mediaserver      cifs    vers=3.0,uid=1000,gid=1000,credentials=/home/plex/.smbcredentials,iocharset=utf8,sec=ntlmssp,file_mode=0771,dir_mode=0771        0       0

Updated and working fstab entry (vers=1.0):

Code: Select all

//192.168.0.20/mediaserver      /media/mediaserver      cifs    vers=1.0,uid=1000,gid=1000,credentials=/home/plex/.smbcredentials,iocharset=utf8,sec=ntlmssp,file_mode=0771,dir_mode=0771        0       0

Thank you so much for your attention on my issue, and my apologies for taking up your time.
I will keep my eyes open for new releases and will hopefully be of more help then.

Regards,
User avatar
rednoah
The Source
Posts: 22985
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Filebot hangs when matching on Ubuntu

Post by rednoah »

No worries. Symptoms are misleading more often than not. Glad we found a fix for this one quickly. Could have been a tricky one.
:idea: Please read the FAQ and How to Request Help.
Post Reply