Request a little help from a GUI format guru!

Any questions? Need some help?
DavidRTurner
Power User
Posts: 85
Joined: 01 Feb 2014, 16:59

Request a little help from a GUI format guru!

Post by DavidRTurner »

Big thanks to rednoah for creating this tool; and to (specifically) Ithiel & DevXen, whose 'ultimate' formatting posts have led so many forward.

I'm not yet ready to run scripts on my files; too many exceptions to any given set of rules, to let it automatically run.
So I'm quite happy using the GUI & revising format code.

I've found some useful Groovy programming references, but a few things still elude me, and I'm hoping some kind soul can look at my code & suggest some improvements.
I have streamlined it as best I can (so far), and I'm sure there are still efficiency hacks that can be found - for now, I'm content in understanding the code as it is, but could use some assistance fixing a few still-unknowns.

Below the examples is my code (with documentation), line-broken for readability.
I've tweaked a few workarounds to some of these issues just below, but if anyone can suggest fixes, much appreciated.

I know red's a busy guy; anyone else willing to review this & take a crack at (any of) these?
Once I've got it figured out (or, once someone has helped me determine the best method/s), I want to document all of this for reference in future.

What I'm still struggling with (examples):

Formatting
1) Au --> AU will capitalize the 2-letter phrase "Au". I want this for references to Australia (i.e. when a show name has a UK version & an AU version. So I want this working for instances of " (Au)" or " Au" but not " Audition".
I'm having trouble finding the code for a LITERAL interpretation of round brackets - so that " (Au)" is seen as "space-openbracket-A-u-closebracket".
I can find that \b refers to a space before the next characters, which it would in the case of a "showname (country)", but I'm still seeing instances where it doesn't quite work.

2) AM/PM, A.M./P.M. --> I'd like these to show the periods if they're part of the name, but no periods if they're not.
I've tried various replace functions, but I keep getting instances of "A Madman" coming back as "A.M.dman" or "aM" or something else that's not right.

3) Roman Numerals
The code covers anything in RN except '500' ('D') - when included, any names/titles with "Did" in them, gets uppercased, since both D & I are in the phrase.
Any thoughts on expanding the brief one-liner? Or would I be better off simply using direct replaceAll instances to "DI", "DV", "DX", "DC", etc... ??

4) multiple replaceAll's exist with strings along with 'not-sure-what-to-call-these' (.replaceAll(/[?|]/, "!") ) to cover various scenarios. I'd like to make these better, perhaps using a TV Replacements.csv file - note that Scottish-type names ('McTaggart') are using a straight "replace Mct with McT" format; I'd like to set all letters up for this, without repeating it 26 times - perhaps a code for {n.replaceAll ('Mc*'), ('Mc')+('*').uppercase} or similar...

Filename replacing
5) When there are crossover episodes between 2 shows, I like to keep a copy of the 'real show' with 'this show'.
I rename the copy as 'this show s00e00a' - 'real show s00e00...'
Ex: Without a Trace S06E06 aired just after CSI S08E06, so in the CSI folder, it's
CSI S08E06a- Without a Trace S06E06 - Where & Why - 2007-11-08 [Action, Adventure, Crime, Drama]
I know this could be done with a {fn} expression, but I'm stumped & starting this bit from scratch.
Of course, the file would need to be named CSI S08E06a manually first, so that it could have a function performed on it. How can I I.D. the 'a' (or 'b' or 'c') that I placed in the episode number, and pull the inner show name WAT from it? Or is this just TOO picky... :) There may only be a dozen instances of this in my collection, but if it CAN be done, I want to try & figure out HOW to...
Ideally, I want to replace the 'a' file as a 'b' file, and add an 'a' to the actual show filename:
In this case, CSI S08E06a becomes CSI S08E06b - WAT... and CSI S08E06 becomes CSI S08E06a
I like to keep the files as close to 'in order' as I can, alphanumerically.
Anyone done anything similar?

Missing fields
6) I've brought this up in another post, but I'll try here fresh...
Some (a very few - like a half-dozen) shows out of 6,000 - get returned by TVRage without a genre - not a 'blank' result, but a 'missing field'.
My code replaces a blank result with the phrase 'GENRE', but on these few, it is coming back empty.
Crude Britannia- The Story of North Sea Oil (2009, NETWORK) S01E01 - Episode 1 (2009-06-18) [] is an example show.
My format brings up the AIRDATE, but no NETWORK or GENRE. Only the GENRE is 'missing' from the resulting data.
The blank NETWORK field, correctly gets replaced with the phrase 'NETWORK', but genre is blank (the last bit of the code, which results in a '[]' but should be '[GENRE]'.
I have just recently figured out that TVRage data comes back with some blank genres, even when there is data on the show profile; don't know if it's an issue with their dbase, or API, or something else.

CSVs
7) I use a TV Network.csv file to determine country of origin
I have seen references (usually to script/CLI efforts) in the forums to using a TV Show.csv to keep a list of shows standard when renaming... that is; when the filename is TBBT, to use showname 'The Big Bang Theory', etc.
I've got most filenames set now; so I'd like to have the {fn} check against the CSV & never try to match another similarly-named show.
I've used MediaIndex to create a detailed file of names/specs which can provide the base CSV to check against.
I'm not quite sure how to do this one. ??

8) Also for CSVs, I'd like to set a TV Genres.csv file to check - essentially, "if Comedy is one of the genres returned, use Comedy" as opposed to doing a long if/then statement as it's set for right now, which is inefficient & clunky, as the genres are not sorted alphabetically when returned.


Code: Select all

{
//
Section 1
1) Set location for sorted files to go (Z:\Premiers\SORTED) - I prefer the Z: drive to \\Drobo5N\DRT-Media as it is shorter for the GUI, and Z: should always exist if I'm managing the files.
2) Check genres - if any of the genres are X, then use Y - this selects all Comedy as a primary sort, then Sci Fi, etc...
3) If no specific genre found, use '_NO_GENRE'.
NOTE: folder path slash must be right up against the brackets, else a blank space will appear.  This may be correctable, but I haven't found a solution yet, to allow the next field to be on a separate line.
4a) If a series start year (y) exists, then use it to sort into decades (i.e. 1987 becomes 1980s).
4b) If no series start year exists, try the airdate year - this particular formatting is for my Premiers collection, so all episodes are S01E01 anyway.
4c) If neither year exists, set field to 'YEAR' so it can be sorted separately, for later review.
NOTE: folder path slash must be right up against the brackets, else a blank space will appear.  This may be correctable, but I haven't found a solution yet, to allow the next field to be on a separate line.
5a) Series name (n) with the typical Uppercase first letter, lower rest of word; A/An/The to the back of the title, and a few punctuation corrections (characters Windows doesn't like in filenames).
5b) Set numerical references (1st, 2nd, 3rd...) to lowercase to avoid 1St, 2Nd, 3Rd...
5c) Set Roman Numerals to uppercase.
5d) Set common uppercase words.
5e) Set common lowercase words.
5f) Set a few unique 'phrases' that might otherwise be considered a word if part of another word.
5g) STILL ISSUES WITH some - US, AU, etc. - there should be a more defined formatting option I haven't discovered yet...
6a) TVRage brings back a lot of 'empty' genres & networks, despite them having data.  FB is treating some of the blanks as 'blank' and some as 'missing', thus I cannot figure a way to cover both with one function.
6b) When there is NO GENRE RESULT, try full line with Z:\...
6c) When this fails (seemingly depends on the db in use) where genre & network are blank/missing, try dropping the any-genres from inside the first curly-bracket and the curly-bracket 'GENRES' just before the folder slash at the end of the operation.
6d) Still working on this.........
}

{
//
Section 1a
-add a check to replace the showname (YEAR) with blank, so that the Y or AIRDATE will pick it up, avoiding a duplicate, i.e. Abcde (2011) (2011, ABC, #12345)...
}

Z:\Premiers\SORTED\{if(genres.contains
('Comedy')) {'Comedy'} else if (genres.contains
('Science Fiction')) {'Sci Fi'} else if (genres.contains
('Science-Fiction')) {'Sci Fi'} else if (genres.contains
('Fantasy')) {'Fantasy'} else if (genres.contains
('Action')) {'Action'} else if (genres.contains
('Thriller')) {'Thriller'}  else if (genres.contains
('Documentary')) {'Documentary'} else if (genres.contains
('Reality')) {'Reality'} else if (genres.contains
('News')) {'Documentary'} else if (genres.contains
('Drama')) {'Drama'} else if (genres.contains
('Crime')) {'Drama'} else if (genres.contains
('Horror')) {'Horror'} else if (genres.contains
('Game')) {'Game'} else if (genres.contains
('Game Show')) {'Game'} else if (genres.contains
('History')) {'History'} else if (genres.contains
('Wildlife')) {'Wildlife'} else if (genres.contains
('Travel')) {'Travel'} else if (genres.contains
('Food')) {'Food'} else if (genres.contains
('Home and Garden')) {'Home and Garden'} else if (genres.contains
('Animation')) {'Animation'} else if (genres.contains
('Children')) {'Children'} else if (genres.contains
('Family')) {'Family'} else if (genres.contains
('Special Interest')) {'Special Interest'} else if (genres.contains
('Educational')) {'Educational'} else if (genres.contains
('Lifestyle')) {'Lifestyle'} else if (genres.contains
('Mini-Series')) {'MiniSeries'} else if (genres.contains
('Talk Show')) {'Talk Show'} else if (genres.contains
('Sport')) {'Sports'} else if (genres.contains
('Sports')) {'Sports'} else if (genres[0]?.trim())
{'_GENRE_ZERO_'} else '_GENRE'}\{any{((int)y/10)+'0s'} {any{((int)airdate.year/10)+'0s'}{airdate.year}} {'_YEAR'}}\{norm = {it.upperInitial() .lowerTrail()

	.replaceAll(/[`´‘’“”""]/, "'") 
	.replaceAll(/[:]/, "- ") 
	.replaceAll(/[;]/, ", ") 
	.replaceAll(/[?|]/, "!") 
	.replaceAll(/ /, " ")
	.replaceAll(/[*\s]+/, " ")
	.replaceAll(/\b[0-9]+(?i:Th|Nd|Rd|St|S)\b/, { it.lower() })
	.replaceAll(/\b[IiVvXxLlCcMm]+\b/, { it.upper() })

	.replaceAll(/\b(?i:N'| a| on| or| of| the| an| and| at| by| with| if| in| is| to| than| for| are| vs| O'| 'Til|'m|'d|'ll|'ve|' It)+\b/, {it.lower() })
	.replaceAll(/\b(?i:aka|bff|chp|bmx|cmt|nfl|mtv|mvp|nyc|fbi|cia|gb|usa|cctv|csi|fyi|nasa|nypd|bbc|vcr|vhs|dvd|nz|uk|ncis|csi|pd|ocd|wwi|wkrp|bbq| tv|sg-1|fm|dj|diy|qi|rv|sos|ufo|wwi|wwii|wpc|hd| cp| cn| er| la| dds)\b/, { it.upper() })
	.replaceAll(/(Dc)/, "DC").replaceAll(/(Dci)/, "DCI").replaceAll(/(Md)/, "MD").replaceAll(/(NY)/, "NY").replaceAll(/(Uk)/, "UK").replaceAll(/(Anzac)/, "ANZAC")
	.replaceAll(/(Rupaul)/, "RuPaul").replaceAll(/(Versus)/, "vs").replaceAll(/(LIL')/, "Lil'")
	.replaceAll(/(Mca)/, "McA") .replaceAll(/(Mcb)/, "McB") .replaceAll(/(Mcc)/, "McC") .replaceAll(/(Mcg)/, "McG") .replaceAll(/(Mcm)/, "McM") .replaceAll(/(Mcn)/, "McN") .replaceAll(/(Mct)/, "McT")
	.replaceAll(/(Au )/, "AU ")	.replaceAll(/( Pm)/, " PM")
	
	.replace(" ($y)",'')
	.replaceTrailingBrackets()
	}
	; norm(n).replaceFirst(/^(?i)(The|A|An)\s(.+)/, /$2, $1/) }



{
//
Section 2
1) Find the start year of the series & append to the (above) name.
1a) If there is anything in the year (y) then use it.
1b) Else if there is anything in the airdate then use it.
1c) Else use 'YEAR'.
-) NEED TO ADD A WAY TO ELIMINATE DUPLICATE YEARS (i.e. XYZ (2014) (2014, ABC...))
2) Find the broadcasting network of the show; else use 'NETWORK'.
3) Find the dbase ID#; else use '#ID#'.
	.replaceAll(/\b[Tv]+\b/, { it.upper() }).replaceAll(/\b[T.v.]+\b/, { it.upper() })
	.replaceAll('a.M.','A.M.').replaceAll('p.M.','P.M.')
}

({any{y} {any{airdate.year}{airdate.year}} {'YEAR'}}, 
{any{info.network} {'NETWORK'}}, 
#{any{info.id}{'ID#'}})

{
//
Section 3
1) Set special episode numbers (should be good with the s00e00, but just in case)
2) Set episode number format as SeasonXXEpisodeXX
3) Same operators as above to clean up episode title name (t)
}

{episode.special ? 'S00E'+special.pad(2) : s00e00} - {norm(t)}

{
//
Section 4
1) If filename (fn) contains phrase 'CD' or 'Part' or 'Pt', change to/keep as 'Pt' so a split episode doesn't try to get overwritten.
}

{fn.contains('CD1') ? ' (Pt1)':""}{fn.contains('CD2') ? ' (Pt2)':""}{fn.contains('CD3') ? ' (Pt3)':""}
{fn.contains('Part1') ? ' (Pt1)':""}{fn.contains('Part2') ? ' (Pt2)':""}{fn.contains('Part3') ? ' (Pt3)':""}
{fn.contains('Pt1') ? ' (Pt1)':""}{fn.contains('Pt2') ? ' (Pt2)':""}{fn.contains('Pt3') ? ' (Pt3)':""}
{fn.contains('(1)') ? ' (Pt1)':""}{fn.contains('(2)') ? ' (Pt2)':""}{fn.contains('(3)') ? ' (Pt3)':""}
{fn.contains('3D') || fn.contains('3-D') ? ' '+'3D':""}
{' [' + fn.matchAll(/extended|uncensored|remastered|unrated|uncut|alternate|director's.cut|special.edition|unsold.pilot|unaired.pilot|unaired.episode/)*.upperInitial()*.lowerTrail().sort().join(', ').replaceAll(/ [._]/, " ") + ']'}

{
//
Section 5
1) Check airdate for the episode airdate; if exist, use it; else check starting year; if exist, use it; else use 'NO_AIRDATE'.
2) Where no airdate exists, but year exists, use year & add '-xx-xx' to illustrate it's a fix for a missing airdate.
3) Check if any genres exist; if so, use the first 3; if not, use 'NO_GENRE'.
}

({any{airdate}{any{y}{'AIRDATE'}}}) {genres.take(3) ?: '[GENRE]'}


{
//
Section 6
1) Match the network against my CSV file to get country of origin - if not found in CSV, use the network field; research & add that network to the CSV to update.
}

({csv('B:/Downloads/Filebot/TV Networks List.csv').get(info.network) ?: info.network})
User avatar
rednoah
The Source
Posts: 22984
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Request a little help from a GUI format guru!

Post by rednoah »

These examples should help:

String literal replace:

Code: Select all

{'(Au)'.replace('(Au)', '(AU)')}
In String literal you don't need to worry about ()

Regex replace:

Code: Select all

{'test (au)'.replaceAll(/[(](?i:UK|US|AU)[)]/, {it.upper()})}
Since () has special meaning in regular expressions, you need to escape it with \ or use [] character class. The (?i) flag forces case-insensitive matching.

AM/PM is a bit more tricky.

Code: Select all

{'A Madman 9 AM to 5 P.M.'.replaceAll(/\b(A|P)[.]?(M)\b[.]?/,{it[1]+'.M.'})}
The thing to note is that regular expression . means <any character> so if you want a literal dot, you need to escape it.

If you wanna upper case all roman numerals is probably easier to specify the ones you care about token by token. Though I'm sure a regex pattern would be shorter.

Code: Select all

{'star wars v viii dc'.replaceAll(/(?i:VI*|DC|DX)/,{it.upper()})}
Replace McSomething

Code: Select all

{'Mctaggart mcDuck mcd MCHAMMER'.replaceAll(/\b(?i:Mc[a-z]+)\b/, {'Mc'+it[2..-1].upperInitial()})}
:idea: Please read the FAQ and How to Request Help.
User avatar
rednoah
The Source
Posts: 22984
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Request a little help from a GUI format guru!

Post by rednoah »

If you have issues with {genre} you can try the latest build (r2747) which should work better for TVRage. Please note that I highly recommend using TheTVDB instead though.

Examples for how to deal with undefined values can be found here:
http://www.filebot.net/forums/viewtopic.php?f=5&t=1895

e.g.

Code: Select all

{any{genre}{'NOGENRE'}}
:idea: Please read the FAQ and How to Request Help.
User avatar
rednoah
The Source
Posts: 22984
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Request a little help from a GUI format guru!

Post by rednoah »

This expression will give you a Map for all lines where <before ;> maps to <after ;>

Code: Select all

csv('/path/to/data.csv')
The rest is just Groovy code. If you want to map some regular expression to some value you'll probably need to iterate through each mapping until you find a match.

This might help:
http://groovy.codehaus.org/JN1035-Maps
:idea: Please read the FAQ and How to Request Help.
DavidRTurner
Power User
Posts: 85
Joined: 01 Feb 2014, 16:59

Re: Request a little help from a GUI format guru!

Post by DavidRTurner »

I've been working on this since I posted...

I couldn't get your am/pm code to work fully, but it seems to be working with this code:

Code: Select all

.replaceAll(/[a|A]\.M\./, "AM").replaceAll(/[p|P]\.M\./, "PM")
The McX... now works, thank you!

The (AU) seems OK, but I know I have redundant, perhaps duplicate, code I'm still streamlining.


It seems now, that TVRage has added some split-genres, i.e. "Military/War" - and the forward slash is killing me...
I've searched high & low for a way to split a string around a forward slash - either to create 2 'genres' from the one split-genre, or even just to see if one of them is in my list.

So - if a show has genres:
"Documentary, Action, Military/War" - can I somehow read the 4 words as separate genre items?

Or, can I at least check for the word "War" alone?
I am unable to find a conclusive answer for how to treat a forward slash as a character (as opposed to a folder path delimiter), and how to return a substring of a field, when splitting on a given character.

So - I'd prefer to list 4 genres from the 4 words; but I'd be OK replacing the Military/War one with just one of the words.
User avatar
rednoah
The Source
Posts: 22984
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Request a little help from a GUI format guru!

Post by rednoah »

1.
The conclusive answer to having / in the filename is that you can't have / in the filename.

2.

Code: Select all

{'Military/War'.replace('/', '-')}
:idea: Please read the FAQ and How to Request Help.
DavidRTurner
Power User
Posts: 85
Joined: 01 Feb 2014, 16:59

Re: Request a little help from a GUI format guru!

Post by DavidRTurner »

rednoah wrote:1.
The conclusive answer to having / in the filename is that you can't have / in the filename.
I know this ;) ; but whoever maintains TVRage's dbase apparently doesn't know this, or has never used its API.

2.

Code: Select all

{'Military/War'.replace('/', '-')}
I know this, too... ;)
What I am searching for is a generic way to parse a string on a given character - I know there are numerous ways to do this; but I have so far had no need for it. Now I do!
The slash, as a special (reserved?) character, might make this particular parsing attempt impossible - certainly there are a lot of complaints on Javascript forums about it!). Still, I try...

Since I now have a number of shows where TVRage is the only data source, and several of them have split-genres, I need to:
-manually name/sort these files, or
-manually work my format around the genres for this set of files, and fix it each time depending on how many genres are returned, or
-manually enter all their split genres (which will change over time, requiring format maintenance), or
-find a generic parsing format that can include the slash


For the time being, I'll work around the issue using the Replace function; I'm trying to learn RegEx to fix as many possible issues in my format as possible... but this one character seems to cause a lot of grief... :(
User avatar
rednoah
The Source
Posts: 22984
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Request a little help from a GUI format guru!

Post by rednoah »

Same thing as REGEX:

Code: Select all

{'Military/War'.replaceAll('/', '-')}
/ is not a special character in REGEX.

But if /.../ is used to define String that defines the REGEX then it has to be escaped of course:

Code: Select all

{'Military/War'.replaceAll(/\//, '-')}

Merry Xmas!
:idea: Please read the FAQ and How to Request Help.
DavidRTurner
Power User
Posts: 85
Joined: 01 Feb 2014, 16:59

Re: Request a little help from a GUI format guru!

Post by DavidRTurner »

Thank you!!

I was finally able to get something working, as I decided not to set specific strings for the genre check.
If TVRage just changed a bunch of genres, they might change a bunch more; so a general function is preferable so that any major re-sort doesn't have to be manually checked.

While this code may be kludgy, it's working... :D

I want to list (up to) 3 genres in the filename.
-TVRage recently (in the last few days?) changed a number of genres to 'multiple' genres, with a forward-slash in the name (such as 'Military/War'). Groovy treats this as a folder path character.
-I need to check if the count of genres is zero (i.e. if there are no genres) - fill in '_GENRE' instead.
-Then replace the front-slashes with carets, to avoid the slashes defining a folder path.

Code: Select all

{if(genres[0] == null)
	{} else 
	{var genresString = genres[0].replaceAll(/\//, '^')}}
{if(genres[1] == null)
	{} else 
	{var genresString = genres[0].replaceAll(/\//, '^')+', '+genres[1].replaceAll(/\//, '^')}}
{if(genres[2] == null)
	{} else 
	{var genresString = genres[0].replaceAll(/\//, '^')+', '+genres[1].replaceAll(/\//, '^')+', '+genres[2].replaceAll(/\//, '^')}}

[{genresString}]
User avatar
rednoah
The Source
Posts: 22984
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Request a little help from a GUI format guru!

Post by rednoah »

Your format makes my eyes bleed. And why use a really weird character such as caret ^ (which is illegal on some operating systems / file system implementations and most definitely will fuck you eventually) if you can use slash - instead?


Here's how I'd write it:

Code: Select all

{genres*.replaceAll(/[^\w\s]/, '-').take(3)}
With default:

Code: Select all

{(genres ?: ['No Genres'])*.replaceAll(/[^\w\s]/, '-').take(3)}
:idea: Please read the FAQ and How to Request Help.
DavidRTurner
Power User
Posts: 85
Joined: 01 Feb 2014, 16:59

Re: Request a little help from a GUI format guru!

Post by DavidRTurner »

rednoah wrote:Your format makes my eyes bleed.
Well, I DID say it was 'kludgy'... I admitted it may not be efficient, but I'm still working to understand the code first, and make it work, then make it simpler.
Performance is not my concern while I'm still trying to make things work; functionality is.
*SIDEBAR: is there a size limit to the content of the EDIT FORMAT field? As I try to understand it, I add a lot of comments. With the comments in my code, it's over 10,000 characters & FB rejects it. If I bring it in under ~9,600, it's fine - ?limit?
And why use a really weird character such as caret ^ (which is illegal on some operating systems / file system implementations and most definitely will fuck you eventually) if you can use slash - instead?
I'm doing this for personal use with this one application. The caret is a personal choice, to indicate a 'combined' (or as I previously referred, 'split') entry.
The caret represents something different from a dash (for my purposes).
Here's how I'd write it:

Code: Select all

{genres*.replaceAll(/[^\w\s]/, '-').take(3)}
With default:

Code: Select all

{(genres ?: ['No Genres'])*.replaceAll(/[^\w\s]/, '-').take(3)}
I'll take that advice, thanks! I was unaware that the '*' could be used in this way, and it seems to be working.

I can now clarify my big issue better:
I primarily use TheTVDB, but I have several hundred shows not listed there, that do exist on TVRage, so I must continue to use it when TheTVDB brings no results.
When I use TVRage, and there is no genre listed, I get a NullPointerException.
The series may exist, but if the genre does not, all the code I've seen in the forums, and even your suggestions, just DO NOT replace the null with 'No Genres' - it simply fails.

I think this is in part (or whole) due to TVRage's recent dbase updates - genres have been 'combined' (hence the issue with the slash/caret), many networks have changed, and some other information is different it was than several weeks ago when I last ran through my collection.

To clarify: In my testing; when Filebot now tries to return a genre from TVRage data, when there is no genre listed on the site, FB craps out with a NullPointerException. This does NOT happen with TheTVDB; it seems to be specific to TVRage. When TheTVDB genre field is empty, I get the (correct) 'NO GENRE' result in my format.
So - data from the search of the TVRage API is (apparently) corrupted as it returns, IF the genre field is empty.
It stops FB from continuing.

EXAMPLE: Celebrity Rehab Presents Sober House (2009, NETWORK) S01E01 - Season 1, Episode 1 (2009-01-15) [].avi http://www.tvrage.com/Celebrity_Rehab_P ... ober_House
-was previously sorted successfully; no network found, so 'NETWORK' filled in; no genre found, but instead of filling in 'NO GENRE', it returned a blank/null (between the []). This was what caused me grief originally, not understanding why a blank/null was not replaced with 'NO GENRE'. Still, a blank field was better than an error.
-NOW, while there is still no genre in the TVRage dbase, FileBot fails with this NullPointException.
-I THINK it's something to do with the data (or lack of data) in the genre that is causing the issue, so I have focused my efforts on the {genre} field. It's the only one I'm pulling that (I believe) might be set up as an array (so perhaps the problem is particular to arrays?).


Previously, I've said FB seems to treat a null entry for the genres field (no genres in it) differently than a 'blank genre'. It's as if TVRage's genre field is using a non-recognizable character in place of an empty field.
*Is this possible?

For a long time, I thought it was an issue with the way I was checking the returned genre; now I think it must be deeper than that - my system/software, TVRage's API server perhaps - since I'm not seeing any other comments like this in the forums, I'm more inclined to believe it's on my system.
I've updated both 32- and 64- bit Java on my Win7-64 Pro PC; no change.

I'm now trying to find a way to determine if a return shows a failure ('before it fails'); then act on that.
That is; "If an error exists, give me a warning message, rather than crapping out because of the error. Otherwise, do the thing..."
Excel does this with "=iferror(x)" - but I'm not having much luck finding a similar functionality in Java.


I appreciate any help & advice - but I'm not begging for answers; I enjoy tweaking code & learning, even though this is taking me months to perfect... :P
And without FileBot's customizable formatting, I wouldn't have ever stumbled into Java-related coding, something I've had interest in learning for a while - so thanks for that!

As much as anything, I'm hoping someone else would be able to find this thread & suggest how they've dealt with similar problems - since I haven't seen any other comments like this, then if this is somehow specific to my system, I guess I'm screwed... but such is life. :cry:
User avatar
rednoah
The Source
Posts: 22984
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Request a little help from a GUI format guru!

Post by rednoah »

1.
I'm not enforcing a limit, but the toolkit might, and limiting a single line text field to ~10.000 chars seems perfectly reasonable choice to me. :D I'd make myself a script that strips away the comments.

2.
There seems to be a bug in the new code when there is no genres breaking parsing for the whole show. Fixed with r2771.

3.
This thread explains how to deal with bindings that unwind the {group} i.e. throw an exception:
viewtopic.php?f=5&t=1895
:idea: Please read the FAQ and How to Request Help.
DavidRTurner
Power User
Posts: 85
Joined: 01 Feb 2014, 16:59

Re: Request a little help from a GUI format guru!

Post by DavidRTurner »

rednoah wrote:1.
I'm not enforcing a limit, but the toolkit might, and limiting a single line text field to ~10.000 chars seems perfectly reasonable choice to me. :D I'd make myself a script that strips away the comments.
Yes, I retain a copy without comments & keep new code reconciled in both, so I can use the uncommented one.
It's only about 4,000 characters without any comments; but I wanted to document each (new) thing I do, until I'm familiar with its function.
2.
There seems to be a bug in the new code when there is no genres breaking parsing for the whole show. Fixed with r2771.
Whew!!! I've updated and I'm so relieved it's not my own system... and that I can still use both dbases... I'll guess that it was TVRage's addition of the slashed/combined genres that glitched your code since nobody foresaw that happening?!?
3.
This thread explains how to deal with bindings that unwind the {group} i.e. throw an exception:
viewtopic.php?f=5&t=1895
I'm familiar with this info, and it's helped; but while I didn't know what was causing the error (fixed with 2771), I was trying 'everything I could think of' including some of my long, extended, klydgy code - so now I can streamline a lot of it back to more simple terms.
User avatar
rednoah
The Source
Posts: 22984
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Request a little help from a GUI format guru!

Post by rednoah »

1.
Do share that 4k format expression sometime. I'd highly discourage anyone from doing anything crazy like that at home though. :D

2.
The issue affected shows that didn't have any genre at all.
:idea: Please read the FAQ and How to Request Help.
DavidRTurner
Power User
Posts: 85
Joined: 01 Feb 2014, 16:59

Re: Request a little help from a GUI format guru!

Post by DavidRTurner »

rednoah wrote:1.
Do share that 4k format expression sometime. I'd highly discourage anyone from doing anything crazy like that at home though. :D
Once cleaned up again, I will (note: if I go to the trouble of slimming it down in more efficient terms, it would be quite a bit smaller). I want to make DevXen & Ithiel go "WTF, is he crazy???" :D
I'd rather get 99% of possible matches covered, rather than 98%... so I'll spend 1000% as much time as I should, to get that 1% extra.
That's probably why I live alone... :geek:
2.
The issue affected shows that didn't have any genre at all.
But TheTVDB data was fine; only the calls to TVRage were causing an error if a genre was missing. This may or may not matter...
User avatar
rednoah
The Source
Posts: 22984
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Request a little help from a GUI format guru!

Post by rednoah »

I'd rather get 99% of possible matches covered, rather than 98%... so I'll spend 1000% as much time as I should, to get that 1% extra.
Guiding principle behind FileBot. :lol:
:idea: Please read the FAQ and How to Request Help.
DavidRTurner
Power User
Posts: 85
Joined: 01 Feb 2014, 16:59

Re: Request a little help from a GUI format guru!

Post by DavidRTurner »

Well, I have to ask for a bit more help.
I've searched high & low for weeks, to no avail.
I just cannot find a way to get this to work.

I use this code:

Code: Select all

Z:\Premiers\SORTED\{if(genres.contains
('Lifestyle')) {'Life'} else if (genres.contains
('Comedy')) {'Comedy'} else if (genres.contains
('Documentary')) {'NonFiction'} else if (genres.contains
...
to assign a folder, named from the genre of my preference, to sort the show.
The If/Else statement runs my priority of genres.
Unfortunately, this is a full half of my format length, and very untidy.

I want to simply call a CSV to check this - the CSV format would be:
Lifestyle;Life
Comedy;Comedy
Documentary;NonFiction
...

I do this with the Network just fine, as I assign a broadcast country based on the network (a file I maintain manually):

Code: Select all

{csv('B:/Downloads/Filebot/TV Network-Country List.csv').get(info.network) ?: 'No Network Listed'}
This CSV contains:
NBC;US
BBC;UK
...

But my problem is - I cannot find a way to iterate through each genre to see if the first item (Lifestyle) exists; and then iterate through all genres again to check the send item (Comedy) and so on.

Essentially, the logic would be something like:

Code: Select all

Count # of genres   <--  check if genres is empty or has some -> {GenresCount = genres.size()} works.
If Count > 0 then    <-- if 1 genre or more
    For x = 1 to Count   <--  loop x times
        csv('B:/Downloads/Filebot/TV Genres.csv').get(info.genres(x-1)) ?: 'No Match'   <-- match genre X against my manual list, and if not exist, 'no match'
    Loop
Endif
Any thoughts??? I'm embarrassed to tell you how many hours I've worked on this one thing... :shock:
User avatar
rednoah
The Source
Posts: 22984
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Request a little help from a GUI format guru!

Post by rednoah »

Here's a demo for what you need:

* sort genres by whether it's in your list or not
* transform genre entries as defined by your map

Code: Select all

{def map=['Science-Fiction':'SciFi']; genres.sort{ map[it] ? 0 : 1 }.collect{ map[it] ?: 'No Match' }}
:idea: Please read the FAQ and How to Request Help.
DavidRTurner
Power User
Posts: 85
Joined: 01 Feb 2014, 16:59

Re: Request a little help from a GUI format guru!

Post by DavidRTurner »

Not sure if this will help... it's a much cleaner way to map the replacement genres, but that's only part of the process I need to figure out.
I'd like to:

1) Map the existing genres for a show to my own ones - this can do that.

But an if/then/else loop is needed here:

2) Check (all the genres for a show) to see if 'Lifestyle' is one of them; then move that show to the Lifestyle folder.
If 'Lifestyle' is not one of the genres, then check them for 'Comedy'... then repeat down the list of my maps until a match is found. When the first match is found, that's the one I want to use as the 'main' genre for the show, and it goes to that folder.

3) If the list is finished, and it has not found a matching genre, then I want 'No Match' - so I can later go to the 'No Match' folder, update the list with whatever genres are on the show (this is a separate part of my format), and re-run those shows.

4) If there is NO genre on the show, then I want it going into a 'NO GENRE' folder.


I have this all working fine, with the clumsy {if(genres.contains('X')) {'Y'} else if....}, and my CSV, format.
But a cleaner, tidier method is what I'm trying to find. Since I have several CSVs built into the format already, another one would simply help save space in the format itself.


The fact that my preferred genres are NOT in any sorted order is what's confounding my logic.
I'll keep plugging away at it...
User avatar
rednoah
The Source
Posts: 22984
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Request a little help from a GUI format guru!

Post by rednoah »

I'm pretty sure my example above solves (1) (2) (3) and (4) so what's not working?
:idea: Please read the FAQ and How to Request Help.
DavidRTurner
Power User
Posts: 85
Joined: 01 Feb 2014, 16:59

Re: Request a little help from a GUI format guru!

Post by DavidRTurner »

It appears to me that this will
-sort the genres after re-mapping them to my liking (1),
-test if there are no genres at all (4),
-and mark the show with non-matched genres as No Match (3),

but I can't see it helping me select a genre based on a rule I set up (2).

I think what this is doing is 1)sorting the genres, 2)mapping them to new genres, 3)checking if the genres are all in the mapping - and display them, 4)otherwise call 'no match'.

But I don't want to (check if all the dbase's genres for the show are listed in my map).
I want to re-map the genres to new ones, then check if "X" exists with the show; if so, use it as the genre & stop looking at the genres. If it is not there, THEN check if "Y" exists, and use it instead; if it's not there, THEN check if "Z" is one of them... and so on.

If I trusted the dbase's entries to be in most-appropriate order for each show, I'd simply use the 1st genre for the folder sorting. But as example; a fake-reality show that makes fun of contestants, could be "Reality, Game, Comedy, Hidden Camera" - and I want all humourous shows to be generally classified as 'Comedy', so 'Comedy' is high on my priorities list. Which is why I want to see if Comedy is one of the genres, before looking for other genres.

Am I missing something; or have I been too vague in my descriptions of what I need?
I really appreciate the help, but I've spent frightening numbers of hours scouring websites for tips, none of which seem to work for precisely what I'm trying to do.

I'm almost at the point of changing the genres from my CSV or a map & input the new genres into a list, then pulling another CSV to rank the priorities of the genres into another list, then selecting the highest single priority genre from that list. There must be a simpler way to iterate through the data, or to set up the data better.

I'm still looking, but any help is still very much appreciated.
User avatar
rednoah
The Source
Posts: 22984
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Request a little help from a GUI format guru!

Post by rednoah »

it's sorting the {genres} BY {whether or not it exists in your custom mappings table}, so if genres contains an entry that is also contained in your mappings table this entry will be first. Original order is of course respected as long as it doesn't conflict with the new primary order, so if genres does not contain your custom mapping the genres list will remain unchanged.

e.g.

Code: Select all

{def map=['Science-Fiction':'SciFi']; ['1st Gen', 'Science-Fiction'].sort{ map[it] ? 0 : 1 }.collect{ map[it] ?: 'No Match' }}
Result => [SciFi, No Match] and the 1st entry is obviously what we want ;)

Code: Select all

{def map=['Science-Fiction':'SciFi']; ['1st Gen', 'Science-Fiction'].sort{ map[it] ? 0 : 1 }.collect{ map[it] ?: 'No Match' }.first()}
Result => SciFi


Look at the example. How is this not the solution you asked for? :D
:idea: Please read the FAQ and How to Request Help.
DavidRTurner
Power User
Posts: 85
Joined: 01 Feb 2014, 16:59

Re: Request a little help from a GUI format guru!

Post by DavidRTurner »

OK; I think I see how this works now. But, it's not QUITE doing what I need.
If this can be used for my purpose, I'm still unable to see it (so I apologize) :?
I'll re-phrase my issue again...

Here's a sample show:
7.39, The S01E01 - Episode 1 [Drama, Mini-Series, Romance]

Here's a sample of this code with my map*:

Code: Select all

{
def map=['Romance':'Life','Mini-Series':'Mini','Action':'Act','Crime':'Cri'];
genres.sort{ map[it] ? 0 : 1 }
     .collect{ map[it] ?: 'No Match' }
     .first()
}
*Note that my map items are not in any alphabetic order - they are in order of my personal preference. <-- this may be the thing that's confounding the logic.

The show has genres of Drama, Mini-Series & Romance.

This is returning:
"I found Mini-Series and Romance, but not Drama; I've sorted the map items, and since Mini-Series is alphabetically first, I will use that."

I want it to return:
"I found Mini-Series and Romance, but not Drama; I haven't bothered sorting the map items because you don't need that done; since Romance is the first item in your map, I will use it, and call it Life."

So; I THINK that removing the .sort will allow it to do what I need. But I've experimented & can't make it work without the .sort
This assumes that when Groovy checks the mapped items, it does so one-line-at-a-time.
Does Groovy work like this*?

*If Groovy doesn't work that way, then I'm on the wrong track; but I could use this to sort my new genres into a new list, then match that list against a genres-preference list to select the preferred one.
It's 2 steps vs 1, but it's still better than the long, 4k-character if/then/else check that I'm still using.
User avatar
rednoah
The Source
Posts: 22984
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Request a little help from a GUI format guru!

Post by rednoah »

So you want to sort by the position in your mapping table? That's easy too, just a bit more verbose.

Code: Select all

{def map=['Romance':'Life','Mini-Series':'Mini','Action':'Act','Crime':'Cri']; ['Drama', 'Mini-Series', 'Romance'].findResults{ map[it] }.sort{ map.values().toList().indexOf(it) }.first()}
Result => Life


Keep in mind that {genres} order is not specified, it may be alphabetic, it may be by priority of the genre (which my previous format tries to preserve), it is whatever is returned by the database.
:idea: Please read the FAQ and How to Request Help.
DavidRTurner
Power User
Posts: 85
Joined: 01 Feb 2014, 16:59

Re: Request a little help from a GUI format guru!

Post by DavidRTurner »

YES!
The .indexOf combined with the map, is, I guess, what I needed... and I'm guessing it's building a temporary list from the map (.toList) to do the checking. I can use that in other ways, too!!!

'Position' is the word I'd neglected to use - I kept referring to 'priority' or 'preference'... when 'position' would have illustrated it better.

Thanks so much!! I'm now up to about 99.3% of what I need... :)


EDIT:
This mapping method has cut 1,900 characters from my format (and made it considerably more functional for me), allowing me to focus on the few remaining pieces.

So now that I'm at 99.3% complete, I want to reach 99.9%... :oops:
Just (EDIT) 1 more question left in my list... and I think I'll be there!

Keeping split-title parts
I have used this code

Code: Select all

{norm(t).replaceAll(/[(]([0-9])[)]/, {group, match -> ' (Part '+match.pad(1)+')' })}
to replace '(1)' and '(2)' from an episode title, to show '(Part 1)' and '(Part 2)'.
When the title is the same text in both except for the (1) and (2), and I have a double-episode, the format (or perhaps FB internally?) drops the 'parts' & keeps just the title.
So if
S01E01 - Pilot (1) and
S01E02 - Pilot (2), and
I have a double-episode S01E01-E02, it gets named
S01E01-E02 - Pilot,
ignoring the part #s.
I would like to show it as
S01E01-E02 - Pilot (Part 1 & 2).

It does, correctly, show this, when the title text is different in the parts:
S01E01-E02 - Pilot-Start (Part1) & Pilot-Finish (Part2)

This will help me with 3- and 4- part episodes (true, there aren't many...) to help indicate if one is missing (i.e. if I see a show with S01E01-E02 - Pilot (Part 1 & 2) & S01E04 - Pilot (Part 4), I'll be looking for 3.
This won't (can't?) be automatic, and yes, the missing S01E03 should alert me; but the ability to combine same-named fields, would be nice anyway.

With these 2 resolved, I think my format will be close to 99.9% - as complete as (humanly) possible.
Of course, it might not be as complete as (red-ly) possible, but I think 99.9% might be good enough... :mrgreen:
Post Reply