AMC Script creating SYMLINK's with illegal characters

Any questions? Need some help?
Post Reply
punisherag
Posts: 7
Joined: 19 Dec 2020, 15:34

AMC Script creating SYMLINK's with illegal characters

Post by punisherag »

I'm using the deluge-amc.sh script and some of the SYMLINK files contain illegal characters. There are some files that contain – instead of -
I can't figure out how to run the script to only output standard characters.

Code: Select all

Run script [fn:amc] at [Tue Dec 15 05:48:29 CET 2020]
Parameter: artwork = false
Parameter: ut_kind = multi
Parameter: ut_dir = /Outbox/Auto-Added/The.Third.Day.S01E01.REPACK.720p.WEB.x265-MiNX[eztv.io].mkv
Parameter: ut_title = The.Third.Day.S01E01.REPACK.720p.WEB.x265-MiNX[eztv.io].mkv
Parameter: subtitles = en
Parameter: extractFolder = /files/_extracted
Parameter: music = y
Parameter: skipExtract = y
Input: /Outbox/Auto-Added/The.Third.Day.S01E01.REPACK.720p.WEB.x265-MiNX[eztv.io].mkv
Group: {Series=the third day} => [The.Third.Day.S01E01.REPACK.720p.WEB.x265-MiNX[eztv.io].mkv]
Rename episodes using [TheTVDB] with [Airdate Order]
Auto-detected query: [The Third Day]
Fetching episode data for [The Third Day]
Fetching episode data for [The Third: The Girl with the Blue Eye]
Fetching episode data for [The Third Man]
Fetching episode data for [The Third Eye]
Fetching episode data for [The Day]
[SYMLINK] from [/Outbox/Auto-Added/The.Third.Day.S01E01.REPACK.720p.WEB.x265-MiNX[eztv.io].mkv] to [/media/TV Shows/The Third Day/Season 01/The Third Day - S01E01 - Friday – The Father.mkv]
Processed 1 files
punisherag
Posts: 7
Joined: 19 Dec 2020, 15:34

Re: AMC Script creating SYMLINK's with illegal characters

Post by punisherag »

Here's the script I'm using. (I added a space between $ and HOME since it was creating an error when I tried to post it on the forums.)

Code: Select all

export LANG=en_US.UTF-8
export LANGUAGE=en_US.UTF-8
export LC_CTYPE="en_US.UTF-8"
TORRENT_PATH="$3/$2"
TORRENT_NAME="$2"
TORRENT_LABEL="N/A"
# Subtitle language
SUBLANG=en
SKIP_EXTRACT=y
MUSIC=y
"$ HOME"/bin/filebot -script fn:amc \
    --output "$ HOME/media" \
    -non-strict --encoding utf8 --log all --log-file "$ HOME"/scripts/amc-deluge.log --action symlink --conflict override \
    --def artwork=false ut_kind=multi "ut_dir=$TORRENT_PATH" "ut_title=$TORRENT_NAME" subtitles=$SUBLANG \
        extractFolder="$ HOME/files/_extracted" music=$MUSIC skipExtract=$SKIP_EXTRACT &
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: AMC Script creating SYMLINK's with illegal characters

Post by rednoah »

On Linux, there is no such thing as illegal characters. If the path was invalid, the file system would prevent you from creating that path in the first place.

However, Linux notably does not enforce a specific file system character encoding. So you need to make sure that each application is configured to use utf-8 so that each application encodes and decodes paths the same way. The LANG and LC_* environment variable are typically responsible for ensuring just that.


EDIT:

Also, FileBot will write log files with UTF-8 encoding. So when reading log files, make sure you're text file reader is configured with UTF-8 as well.
:idea: Please read the FAQ and How to Request Help.
punisherag
Posts: 7
Joined: 19 Dec 2020, 15:34

Re: AMC Script creating SYMLINK's with illegal characters

Post by punisherag »

Thanks for replying, and yes, I understand that they are technically not "illegal" characters. Thanks for correcting me, however, I'm not sure you've answered my actual question. Is there a way that I can control the character set that's used to create SYMLINKs while using the AMC script. I know that it's possible to perform character substitution when running filebot from CLI or the standalone application, but is there a way to perform character substitution while using the AMC script. And I apologize in advance if I'm not using the proper terminology.
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: AMC Script creating SYMLINK's with illegal characters

Post by rednoah »

1.
LANG=en_US.UTF-8 controls the file system encoding, so it's probably already working.

How did you copy & paste this line? By running cat *.log on your Linux device? By opening the file with Notepad from a remote Windows machine?

Code: Select all

[SYMLINK] from [/Outbox/Auto-Added/The.Third.Day.S01E01.REPACK.720p.WEB.x265-MiNX[eztv.io].mkv] to [/media/TV Shows/The Third Day/Season 01/The Third Day - S01E01 - Friday – The Father.mkv]
:idea: – would suggest that you're incorrectly using Windows-1252 to decode a UTF-8 encoded text file.



2.
You can of course use a format such as {plex.path.ascii()} to ensure ASCII file paths if this is something you need or want for some reason.
:idea: Please read the FAQ and How to Request Help.
punisherag
Posts: 7
Joined: 19 Dec 2020, 15:34

Re: AMC Script creating SYMLINK's with illegal characters

Post by punisherag »

I was using WinSCP to view the log using the internal editor. When I use cat to view the log, it shows the dash correctly.

It sounds like plex.path.ascii would solve my problem. I'm not really sure how I would implement that though. In the AMC documentation page I see the example:

Code: Select all

--def movieFormat="M:/{plex}" seriesFormat="S:/{plex}" animeFormat="T:/{plex}" musicFormat="N:/{plex}"
I'm thinking I would just need to modify my script to include something like this:

Code: Select all

--def movieFormat="$_HOME/media/{plex.path.ascii()}" seriesFormat="$_HOME/media/{plex.path.ascii()}" animeFormat="$_HOME/media/{plex.path.ascii()}" musicFormat="$_HOME/media/{plex.path.ascii()}"
Does that look right?
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: AMC Script creating SYMLINK's with illegal characters

Post by rednoah »

punisherag wrote: 21 Dec 2020, 22:27 I was using WinSCP to view the log using the internal editor
The solution is to configure WinSCP to open files with UTF-8 encoding. If you need further help, please include screenshots so that we can see what you see, or ask the WinSCP developers directly:
https://winscp.net/forum/viewforum.php?f=2


:!: Files are likely renamed correctly all along. You're merely reading the the log file with the wrong text encoding. This is a generic WinSCP end user issue, unrelated to FileBot or file system encoding.
:idea: Please read the FAQ and How to Request Help.
punisherag
Posts: 7
Joined: 19 Dec 2020, 15:34

Re: AMC Script creating SYMLINK's with illegal characters

Post by punisherag »

Thanks again for your response, however, I was just using WinSCP as part of my diagnostic. I actually really want to get rid of any non-ascii characters in the SYMLINKs that are created. I have a script running on a Windows computer that is transferring the files after they are processed by filebot. So what do you think about the proposed switch in my script above? Will it result in SYMLINK files that only contain ascii characters?
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: AMC Script creating SYMLINK's with illegal characters

Post by rednoah »

punisherag wrote: 22 Dec 2020, 14:48 I actually really want to get rid of any non-ascii characters in the SYMLINKs that are created. I have a script running on a Windows computer that is transferring the files after they are processed by filebot.
Windows has no problem with Unicode file paths, so why not keep the Unicode file paths as they are? In any case, naming is up to you via your custom format. If your custom format yields ASCII-only file paths, then you will get ASCII-only file paths. {plex.path.ascii()} should work but you are encouraged to run your own set of tests in the Format Editor GUI on your Windows Desktop before pasting the format into your bash script.
:idea: Please read the FAQ and How to Request Help.
punisherag
Posts: 7
Joined: 19 Dec 2020, 15:34

Re: AMC Script creating SYMLINK's with illegal characters

Post by punisherag »

Ok, can you tell me if the syntax looks correct?
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: AMC Script creating SYMLINK's with illegal characters

Post by rednoah »

Looks good at first glance. Log in SSH, do a test run and send me a screenshot of the result. Based on that I can tell you if the syntax is correct or not. That's kinda how these things work. :lol:
:idea: Please read the FAQ and How to Request Help.
punisherag
Posts: 7
Joined: 19 Dec 2020, 15:34

Re: AMC Script creating SYMLINK's with illegal characters

Post by punisherag »

Yes, that did the trick

Code: Select all

punisherag:~$ /punisherag/bin/filebot -script fn:amc --output "/punisherag/media" -non-strict --encoding utf8 --log all --log-file "/punisherag"/scripts/amc-deluge.log --action symlink --conflict override --def artwork=false ut_kind=multi "ut_dir=/punisherag/Outbox/Auto-Added/His.Dark.Materials.S02E07.Esahaettr.720p.HDTV.x264-KETTLE[eztv.re].mkv" "ut_title=His.Dark.Materials.S02E07.Esahaettr.720p.HDTV.x264-KETTLE[eztv.re].mkv" subtitles=en extractFolder="/punisherag/files/_extracted" music=y skipExtract=y --def movieFormat="/punisherag/media/{plex.path.ascii()}" seriesFormat="/punisherag/media/{plex.path.ascii()}" musicFormat="/punisherag/media/{plex.path.ascii()}"
Run script [fn:amc] at [Tue Dec 22 17:01:42 CET 2020]
Parameter: artwork = false
Parameter: ut_kind = multi
Parameter: ut_dir = /punisherag/Outbox/Auto-Added/His.Dark.Materials.S02E07.Esahaettr.720p.HDTV.x264-KETTLE[eztv.re].mkv
Parameter: ut_title = His.Dark.Materials.S02E07.Esahaettr.720p.HDTV.x264-KETTLE[eztv.re].mkv
Parameter: subtitles = en
Parameter: extractFolder = /punisherag/files/_extracted
Parameter: music = y
Parameter: skipExtract = y
Parameter: movieFormat = /punisherag/media/{plex.path.ascii()}
Parameter: seriesFormat = /punisherag/media/{plex.path.ascii()}
Parameter: musicFormat = /punisherag/media/{plex.path.ascii()}
Input: /punisherag/Outbox/Auto-Added/His.Dark.Materials.S02E07.Esahaettr.720p.HDTV.x264-KETTLE[eztv.re].mkv
xattr: [His.Dark.Materials.S02E07.Esahaettr.720p.HDTV.x264-KETTLE[eztv.re].mkv] => [His Dark Materials - 2x07 - Æsahættr]
Group: {Series=his dark materials} => [His.Dark.Materials.S02E07.Esahaettr.720p.HDTV.x264-KETTLE[eztv.re].mkv]
Get [English] subtitles for 1 files
Looking up subtitles by hash via OpenSubtitles
No matching subtitles found: /punisherag/Outbox/Auto-Added/His.Dark.Materials.S02E07.Esahaettr.720p.HDTV.x264-KETTLE[eztv.re].mkv
Rename episodes using [TheTVDB] with [Airdate Order]
Auto-detected query: [His Dark Materials]
Fetching episode data for [His Dark Materials]
[SYMLINK] from [/punisherag/Outbox/Auto-Added/His.Dark.Materials.S02E07.Esahaettr.720p.HDTV.x264-KETTLE[eztv.re].mkv] to [/punisherag/media/TV Shows/His Dark Materials/Season 02/His Dark Materials - S02E07 - AEsahaettr.mkv]
Processed 1 files
This will make my script happy again. Thanks for your help.
Post Reply