High CPU, Permission denied: /proc

[deLuXe83]

Hello,
I execute my Filebot Script on my Synology Diskstation.
After processing all files, there is A LOT of "permission denied" entries, which leads to a massive CPU load.
I think this is new, since I upgrade today to version 4.9.0

here is the log example:

Code: Select all

Ignore hidden: /volume1/download/RSScrawler/Homeland.S08E07.Fker.Shot.Me.720p.AMZN.WEB-DL.DDP5.1.h264-NTb/.DS_Store
Ignore system path: /volume1/download/RSScrawler/Homeland.S08E07.Fker.Shot.Me.720p.AMZN.WEB-DL.DDP5.1.h264-NTb/@eaDir
No files selected for processing
Done ¯\_(ツ)_/¯
Permission denied: /etc/postgresql
Permission denied: /etc/space
Permission denied: /etc/sssd
Permission denied: /etc/sudoers.d
Permission denied: /etc.defaults/postgresql
Permission denied: /etc.defaults/sssd
Permission denied: /etc.defaults/sudoers.d
Permission denied: /lost+found
Permission denied: /proc/1/fd
Permission denied: /proc/1/fdinfo
Permission denied: /proc/1/ns
Permission denied: /proc/1/task/1/fd
Permission denied: /proc/1/task/1/fdinfo
Permission denied: /proc/1/task/1/ns
Permission denied: /proc/2/fd
Permission denied: /proc/2/fdinfo
Permission denied: /proc/2/ns
Permission denied: /proc/2/task/2/fd
Permission denied: /proc/2/task/2/fdinfo
Permission denied: /proc/2/task/2/ns
Permission denied: /proc/3/fd
Permission denied: /proc/3/fdinfo
Permission denied: /proc/3/ns
Permission denied: /proc/3/task/3/fd
Permission denied: /proc/3/task/3/fdinfo
Permission denied: /proc/3/task/3/ns
Permission denied: /proc/5/fd
Permission denied: /proc/5/fdinfo
Permission denied: /proc/5/ns
Permission denied: /proc/5/task/5/fd
Permission denied: /proc/5/task/5/fdinfo
Permission denied: /proc/5/task/5/ns
Permission denied: /proc/7/fd
Permission denied: /proc/7/fdinfo
Permission denied: /proc/7/ns
Permission denied: /proc/7/task/7/fd
Permission denied: /proc/7/task/7/fdinfo
Permission denied: /proc/7/task/7/ns
Permission denied: /proc/8/fd
Permission denied: /proc/8/fdinfo
Permission denied: /proc/8/ns
Permission denied: /proc/8/task/8/fd
Permission denied: /proc/8/task/8/fdinfo
Permission denied: /proc/8/task/8/ns
Permission denied: /proc/9/fd
Permission denied: /proc/9/fdinfo
Permission denied: /proc/9/ns
Permission denied: /proc/9/task/9/fd
Permission denied: /proc/9/task/9/fdinfo
Permission denied: /proc/9/task/9/ns
Permission denied: /proc/10/fd
Permission denied: /proc/10/fdinfo
Permission denied: /proc/10/ns
Permission denied: /proc/10/task/10/fd
Permission denied: /proc/10/task/10/fdinfo
Permission denied: /proc/10/task/10/ns
Permission denied: /proc/11/fd
Permission denied: /proc/11/fdinfo
Permission denied: /proc/11/ns
Permission denied: /proc/11/task/11/fd
Permission denied: /proc/11/task/11/fdinfo
Permission denied: /proc/11/task/11/ns
Permission denied: /proc/12/fd
Permission denied: /proc/12/fdinfo
Permission denied: /proc/12/ns
Permission denied: /proc/12/task/12/fd
Permission denied: /proc/12/task/12/fdinfo
Permission denied: /proc/12/task/12/ns
Permission denied: /proc/13/fd
Permission denied: /proc/13/fdinfo
Permission denied: /proc/13/ns
Permission denied: /proc/13/task/13/fd
Permission denied: /proc/13/task/13/fdinfo
Permission denied: /proc/13/task/13/ns
Permission denied: /proc/15/fd
Permission denied: /proc/15/fdinfo
Permission denied: /proc/15/ns
Permission denied: /proc/15/task/15/fd
Permission denied: /proc/15/task/15/fdinfo
Permission denied: /proc/15/task/15/ns
Permission denied: /proc/16/fd
Permission denied: /proc/16/fdinfo
Permission denied: /proc/16/ns
Permission denied: /proc/16/task/16/fd
Permission denied: /proc/16/task/16/fdinfo
Permission denied: /proc/16/task/16/ns
Permission denied: /proc/17/fd
Permission denied: /proc/17/fdinfo
Permission denied: /proc/17/ns
Permission denied: /proc/17/task/17/fd
Permission denied: /proc/17/task/17/fdinfo

and this goes on... until i break with ctrl+c


any suggestions?
thank you very much!!

Code: Select all

[rednoah]

Please paste the filebot command you're executing.


Looks like you're accidentally passing in your entire file system / for processing. Don't do that.


Probably a user error unrelated to a specific version of FileBot.



EDIT:

Though I seem to remember that FileBot does now follow symlinks, as per request, so if you've got an accidental symlink to / somewhere in your media folder, that could explain the issue.

[deLuXe83]

Hi rednoah, thanks for the quick reply

here is the command:

Code: Select all

bash /volume1/share/filebot.sh

and content of filebot.sh:

Code: Select all

LOG="/volume1/share/filebot.log"
EXC="/volume1/share/filebot.txt"
SERIEN="/volume1/video/Serien/{n}/{s}/{n}.{s00e00} - {t} ({y}) {vf}"
FILME="/volume1/video/Filme/{n} ({y}) {source}({vf}@{vc}) [{rating}]/{n} ({y}) {source}({vf}@{vc}) [{rating}]"
/var/packages/filebot/target/filebot.sh -script 'fn:amc' /volume1/download/RSScrawler/ --output /volume2/share2/serien --action move -non-strict --conflict auto --lang en --def 'ut_label=tv' 'music=y' 'skipExtract=y' 'seriesFormat={n}/{'\''Season '\''+s}/{n} {s00e00} {t}' 'excludeList=.excludes' --log all --log-file $LOG 

/var/packages/filebot/target/filebot.sh -script fn:cleaner "$1" --def root=y

thx

[deLuXe83]

regarding your edit:
i don't see any symlinks, but i' not sure, if they are not visible....
the only path, that should be crawled in my example, is the /volume1/download/RSScrawler/

[rednoah]

1.
What does this command say?

Code: Select all

find /volume1/download/RSScrawler -type l -ls

https://stackoverflow.com/a/21438684/1514467


2.
Replace fn:amc with fn:sysenv to double-check that you're passing arguments correctly, and not accidentally passing / as additional input argument somewhere down the line.

[deLuXe83]

1.
What does this command say?

Code: Select all

find /volume1/download/RSScrawler -type l -ls

https://stackoverflow.com/a/21438684/1514467

Code: Select all

@ds416play:/volume1/download/RSScrawler$ find /volume1/download/RSScrawler -type l -ls
@ds416play:/volume1/download/RSScrawler$ 

so, no content ?!

2.
Replace fn:amc with fn:sysenv to double-check that you're passing arguments correctly, and not accidentally passing / as additional input argument somewhere down the line.

this says

Code: Select all

@ds416play:/volume1/download/RSScrawler$ bash /volume1/share/filebot.sh
# Environment Variables #
PATH: /sbin:/bin:/usr/sbin:/usr/bin:/usr/syno/sbin:/usr/syno/bin:/usr/local/sbin:/usr/local/bin:/var/packages/Java8/target/j2sdk-image/bin:/var/packages/Java8/target/j2sdk-image/jre/bin:/usr/local/chromaprint/bin:/var/packages/filebot/target/lib/x86_64
SHELL: /bin/sh
PAGER: more
TERMINFO: /usr/share/terminfo
JAVA_HOME: /var/packages/Java8/target/j2sdk-image/jre
SSH_TTY: /dev/pts/14
SSH_CLIENT: 192.168.2.108 61413 22
OLDPWD: /volume1/download/RSScrawler
TERM: xterm-256color
USER: PGDATA: /var/services/pgsql
CLASSPATH: .:/var/packages/Java8/target/j2sdk-image/jre/lib
LANG: en_US.UTF-8
SSH_CONNECTION: 192.168.2.108 61413 192.168.2.75 22
MAIL: /var/mail/mkunkel
LC_ALL: en_US.UTF-8
LOGNAME: mkunkel
PWD: /volume1/download/RSScrawler
HOME: /var/services/homes/mkunkel
SHLVL: 3
_: /usr/local/bin/java


# Java System Properties #
java.specification.version: 14
sun.jnu.encoding: UTF-8
java.class.path: /var/packages/filebot/target/jar/filebot.jar
java.vm.vendor: Oracle Corporation
grape.root: /var/packages/filebot/target/data/mkunkel/grape
useExtendedFileAttributes: true
jna.library.path: /usr/local/mediainfo/lib:/usr/local/chromaprint/lib:/var/packages/filebot/target/lib/x86_64
sun.arch.data.model: 64
net.filebot.archive.extractor: SevenZipNativeBindings
java.vendor.url: https://java.oracle.com/
jna.nounpack: true
user.timezone: Europe/Amsterdam
java.vm.specification.version: 14
os.name: Linux
sun.net.client.defaultReadTimeout: 60000
jna.boot.library.path: /usr/local/mediainfo/lib:/usr/local/chromaprint/lib:/var/packages/filebot/target/lib/x86_64
sun.java.launcher: SUN_STANDARD
user.country: US
sun.boot.library.path: /volume1/@appstore/java-installer/jdk-14/lib
sun.java.command: /var/packages/filebot/target/jar/filebot.jar -script fn:sysenv /volume1/download/RSScrawler/ --output /volume2/share2/serien --action move -non-strict --conflict auto --lang en --def ut_label=tv music=y skipExtract=y seriesFormat={n}/{'Season '+s}/{n} {s00e00} {t} excludeList=.excludes --log all --log-file /volume1/share/filebot.log
jdk.debug: release
sun.cpu.endian: little
user.home: /var/packages/filebot/target/data/mkunkel
user.language: en
java.util.prefs.PreferencesFactory: net.filebot.util.prefs.FilePreferencesFactory
java.specification.vendor: Oracle Corporation
java.version.date: 2020-03-17
java.home: /volume1/@appstore/java-installer/jdk-14
file.separator: /
java.vm.compressedOopsMode: 32-bit
line.separator: 

unixfs: false
useCreationDate: false
java.vm.specification.vendor: Oracle Corporation
java.specification.name: Java Platform API Specification
java.awt.headless: true
application.cache: /var/packages/filebot/target/data/mkunkel/cache
http.agent: FileBot/4.9.0
sun.management.compiler: HotSpot 64-Bit Tiered Compilers
java.runtime.version: 14+36-1461
user.name: mkunkel
sun.net.client.defaultConnectTimeout: 10000
net.filebot.license: /var/packages/filebot/target/data/.license
java.net.useSystemProxies: true
path.separator: :
jna.nosys: true
os.version: 3.10.105
java.runtime.name: OpenJDK Runtime Environment
file.encoding: UTF-8
java.vm.name: OpenJDK 64-Bit Server VM
java.vendor.url.bug: https://bugreport.java.com/bugreport/
net.filebot.media.parser: libmediainfo
java.io.tmpdir: /var/packages/filebot/target/data/mkunkel/tmp
net.filebot.media.ffprobe: /volume1/@appstore/MediaServer/bin/ffprobe
swing.crossplatformlaf: javax.swing.plaf.nimbus.NimbusLookAndFeel
java.version: 14
user.dir: /volume1/download/RSScrawler
net.filebot.util.prefs.file: /var/packages/filebot/target/data/mkunkel/prefs.properties
os.arch: amd64
java.vm.specification.name: Java Virtual Machine Specification
application.deployment: spk
java.library.path: /usr/local/mediainfo/lib:/usr/local/chromaprint/lib:/var/packages/filebot/target/lib/x86_64
java.vm.info: mixed mode, sharing
java.vendor: Oracle Corporation
java.vm.version: 14+36-1461
application.dir: /var/packages/filebot/target/data/mkunkel
sun.io.unicode.encoding: UnicodeLittle
org.apache.commons.logging.Log: org.apache.commons.logging.impl.NoOpLog
java.class.version: 58.0


# Arguments #
args[0] = -script
args[1] = fn:sysenv
args[2] = /volume1/download/RSScrawler/
args[3] = --output
args[4] = /volume2/share2/serien
args[5] = --action
args[6] = move
args[7] = -non-strict
args[8] = --conflict
args[9] = auto
args[10] = --lang
args[11] = en
args[12] = --def
args[13] = ut_label=tv
args[14] = music=y
args[15] = skipExtract=y
args[16] = seriesFormat={n}/{'Season '+s}/{n} {s00e00} {t}
args[17] = excludeList=.excludes
args[18] = --log
args[19] = all
args[20] = --log-file
args[21] = /volume1/share/filebot.log


Done ヾ(＠⌒ー⌒＠)ノ
Delete /volume1/download/RSScrawler/Neuer Ordner
Delete /volume1/download/RSScrawler/@eaDir
Delete /volume1/download/RSScrawler
Done ヾ(＠⌒ー⌒＠)ノ
@ds416play:/volume1/download/RSScrawler$ 

[rednoah]

The next step would be to create a new share, add a sample file, see if it works, and then do systematic trial and error on different folders / subfolders until you narrow it down enough to find the culprit.


Note that Done ¯\_(ツ)_/¯ indicates the end of FileBot logging. So the Permission denied logs are probably caused by your script, commands called after filebot, or commands called before filebot.


EDIT:

If $1 has the value of / then you will accidentally delete all your files:

Code: Select all

/var/packages/filebot/target/filebot.sh -script fn:cleaner "$1" --def root=y

[deLuXe83]

yes, thats true.
but there is nothing (by hand) executed after the filbot bash.
so the error has to be inside this filebot.sh file ??

[deLuXe83]

okay thank you.
where can i quickliy find "new versions" or at least manual for both the scripts fn:amc and fn:cleaner ?


EDIT:
because the error is gone, if i out-comment the last line with the cleaner script....

[rednoah]

because the error is gone, if i out-comment the last line with the cleaner script....

Presumably, $1 accidentally ends up being / for some reason, unrelated to filebot version of script revision... Well, looks like you got lucky, if the permission issues wouldn't have slowed down the script, you would have ended up deleting your entire file system.


The amc script manual is quite thorough. The cleaner script is simple enough to not require extensive documentation.

[deLuXe83]

perfect - thank you very much!!!