FileBot

Posted: **15 Apr 2019, 23:41**

More and more streaming services exist. More and more releases come from them.
The current {source} binding takes care of identifying the broad "WEBDL" or "WEBRip" and similar, but does not include where exactly from.
I'll share my solution to this particularly pointless problem.

Code: Select all

      { def file = new File('/scripts/websources.txt')
        def websources = file.exists() ? readLines(file).join("|") : null // to avoid non-existing paths failing the whole block
        def isWeb = (source ==~ /WEB.*/)
        // or def isWeb = source.matches(/WEB.*/) don't know which one is preferrable
        def lfr = { if (isWeb) fn.match(/($websources)\.(?i)WEB/) } // logo free release
        return allOf{fn.match(/(?i)(UHD).$source/).upper()}{lfr}{source}.join(".") } // it also specifies that source is downscaled from a 4K stream

the magic happens in the websources.txt file which contains a fairly comprehensive list of shortcode for websources.

Code: Select all

9NOW
AE
AUBC
AMBC
AS
AJAZ
AMZN
AMC
ATK
ANPL
ANLB
AOL
ARD
iP
BRAV
CNLP
CN
CBC
CBS
4OD
CHGD
CMAX
CNBC
CC
CCGC
COOK
CMT
CRKL
CR
CSPN
CTV
CUR
CWS
DSKI
DHF
DEST
DDY
DTV
DISC
DSNY
DIY
DOCC
DPLY
ETV
ETTV
EPIX
ESPN
ESQ
FAM
FJR
FOOD
FOX
FREE
FYI
GLBL
GLOB
GO90
GC
HLMK
HBO
HGTV
HIDI
HIST
HULU
TOU
IFC
ID
iT
ITV
KNOW
LIFE
LN
MNBC
MTOD
MTV
NATG
NBA
NBC
NF
NFLN
NFL
NICK
NRK
PLUZ
PBS
PBSK
PSN
RSTR
RTE
SBS
SESO
SHMI
SPIK
SNET
SPRT
STAN
STZ
SVT
SWER
SYFY
TBS
TEN
TFOU
TLC
TRVL
TUBI
TV3
TV4
TVL
VH1
VICE
VMEO
UFC
UKTV
UNIV
VLCT
VIAP
VRV
WNET
WME
WWEN
XBOX
YHOO
RED
ZDF

Posted: **24 Apr 2019, 19:59**

Aren't those release groups?
I don't know all of those, but AMZN and ETTV are release groups.

Posted: **24 Apr 2019, 20:09**

Never heard about them.
In any case, they're also commonly used to identify the source of WEB-DL and WEBRip releases:
Roswell.New.Mexico.S01.1080p.AMZN.WEBRip.DDP5.1.x264-NTb
NTb = release group, AMZN.WEBRip = websource

Family.Food.Fight.AU.S01.720p.9NOW.WEBRip.AAC2.0.x264-RTN
RTN = release group, 9NOW.WEBRip = websource

Fangbone.S01.1080p.NF.WEBRip.DD5.1.x264-RCVR
RCVR = release group, NF.WEBRip = websource

Club.de.Cuervos.Presents.The.Ballad.of.Hugo.Sanchez.S01.SPANISH.1080p.NF.WEB-DL.DD5.1.x264-NTb
NTb = release group, NF.WEB-DL = websource

just the first few on google search.
If the above don't work then my format has an issue.

Posted: **25 Apr 2019, 06:42**

I'm considering add the more popular patterns to the {source} list. At least patterns like AMZN.WEBRip make sense to me.

Your full list is too large for me though, can you recommend a small section of post popular once where it makes sense that {source} includes that?

Posted: **26 Apr 2019, 00:57**

I'm not sure I'd include it in {source}, I would consider this an additional, optional detail.
Besides the possibility of this info not always being present (which could fit into the allOf{b1}{b2} pattern), I see this information more akin to the {group} binding, and potentially implementable in a similar manner (remote, updatable file from which to retrieve a pattern).

The reasoning is that there may be new providers (e.g. Disney this year) with massive amount of viewers and the list would need to be updated to match that.
In addition, I believe the list was also created to distinguish releases of shows (and increasingly movies) which were released on a popular provider first and were subsequently re-released by a proprietary streaming service.

Anyway, I guess the most common ones would correspond to the most used streaming services (Amazon, Netflix, Hulu, Google PLAY, Crunchy Roll, iTunes, Vimeo, YouTube RED [yes, season 2 of Cobra Kai was released as Cobra.Kai.S02.1080p.RED.WEB-DL...]); however many acronyms also correspond to tv networks (HBO, STRZ [Starz], SYFY, CC [Comedy Central], CSPN [Cspan, who the hell releases these??], DSNY [Disney]) and it may be worth adding some of the biggest ones (I'm thinking about CW for the Arrow/Flash/Supergirl/etc. or HBO for example).

Posted: **26 Apr 2019, 07:07**

Alright, lets not make any premature decisions then, see how things develop, collect some sample data, and then see what makes the most sense.

Posted: **25 May 2020, 04:36**

https://en.wikipedia.org/wiki/Pirated_m ... reviations

Might be helpful

Posted: **01 Jun 2020, 17:09**

devster wrote: ↑15 Apr 2019, 23:41 More and more streaming services exist. More and more releases come from them.
The current {source} binding takes care of identifying the broad "WEBDL" or "WEBRip" and similar, but does not include where exactly from.
I'll share my solution to this particularly pointless problem.

Code: Select all

      { def file = new File('/scripts/websources.txt')
        def websources = file.exists() ? readLines(file).join("|") : null // to avoid non-existing paths failing the whole block
        def isWeb = (source ==~ /WEB.*/)
        // or def isWeb = source.matches(/WEB.*/) don't know which one is preferrable
        def lfr = { if (isWeb) fn.match(/($websources)\.(?i)WEB/) } // logo free release
        return allOf{fn.match(/(?i)(UHD).$source/).upper()}{lfr}{source}.join(".") } // it also specifies that source is downscaled from a 4K stream

the magic happens in the websources.txt file which contains a fairly comprehensive list of shortcode for websources.

Code: Select all

9NOW
AE
AUBC
AMBC
AS
AJAZ
AMZN
AMC
ATK
ANPL
ANLB
AOL
ARD
iP
BRAV
CNLP
CN
CBC
CBS
4OD
CHGD
CMAX
CNBC
CC
CCGC
COOK
CMT
CRKL
CR
CSPN
CTV
CUR
CWS
DSKI
DHF
DEST
DDY
DTV
DISC
DSNY
DIY
DOCC
DPLY
ETV
ETTV
EPIX
ESPN
ESQ
FAM
FJR
FOOD
FOX
FREE
FYI
GLBL
GLOB
GO90
GC
HLMK
HBO
HGTV
HIDI
HIST
HULU
TOU
IFC
ID
iT
ITV
KNOW
LIFE
LN
MNBC
MTOD
MTV
NATG
NBA
NBC
NF
NFLN
NFL
NICK
NRK
PLUZ
PBS
PBSK
PSN
RSTR
RTE
SBS
SESO
SHMI
SPIK
SNET
SPRT
STAN
STZ
SVT
SWER
SYFY
TBS
TEN
TFOU
TLC
TRVL
TUBI
TV3
TV4
TVL
VH1
VICE
VMEO
UFC
UKTV
UNIV
VLCT
VIAP
VRV
WNET
WME
WWEN
XBOX
YHOO
RED
ZDF

Hello,
I am testing your function to add the video source and it fails me. Has anything changed?

http://prntscr.com/srtyse

The txt is well imported, if I do a return websources it gives me the complete list.

Greetings and thank you!

Posted: **01 Jun 2020, 18:34**

Yes, new versions highlighted some issues in my code, you need to change the following line:

Code: Select all

        def lfr = { if (isWeb) fn.match(/($websources)\.(?i)WEB/) } // logo free release

with something like:

Code: Select all

def lfr
if (isWeb) {
        lfr = fn.match(/($websources)\.(?i)WEB/)
}

or similar.

Posted: **01 Jun 2020, 19:10**

devster wrote: ↑01 Jun 2020, 18:34 Yes, new versions highlighted some issues in my code, you need to change the following line:
Code: Select all
        def lfr = { if (isWeb) fn.match(/($websources)\.(?i)WEB/) } // logo free release
with something like:
Code: Select all
def lfr
if (isWeb) {
        lfr = fn.match(/($websources)\.(?i)WEB/)
}
or similar.

Hello, thanks for answering. I have this and it doesn't work

Code: Select all

{
	def file = new File('C:/websources.txt')
	def websources = file.exists() ? readLines(file).join("|") : null 
	def isWeb = (source ==~ /WEB.*/)
	def lfr
	if (isWeb) {
		lfr = fn.match(/($websources)\.(?i)WEB/)
	}
	return allOf{fn.match(/(?i)(UHD).$source/).upper()}{lfr}{source}.join(".")
}

Now the return doesn't work either. I have tried return "hi" and it returns nothing.

Posted: **01 Jun 2020, 23:40**

Sorry, no clue as to why, you'll probably have to debug on your machine.
The following:

Code: Select all

{
	def isWeb = true
	def lfr
	if (isWeb) {
		lfr = 'hi'
	}
}

works, so it's likely something with the file or the regex or something else.
As a suggestion you can use

Code: Select all

lfr = any{fn.match(/($websources)\.(?i)WEB/)}{null}

as it unwinds gracefully.

Posted: **02 Jun 2020, 00:33**

Code: Select all

{
	any{ def file = new File("C:/websources.txt")
	def websources = file.exists() ? lines(file).join('|') : null 
	source ==~ /WEB.*/ ? lfr = fn.match(/($websources)\.(?i)WEB/) : ''

	return allOf{fn.match(/(?i)(UHD).$source/).upper()}{lfr}{source}.join('.')
	}{source}
}

output e.g.

NF.WEB-DL

Posted: **02 Jun 2020, 09:38**

Solved! I have removed the parentheses and it works correctly.

Code: Select all

def lfr = fn.match(/$websources\.(?i)WEB/)

Posted: **02 Jun 2020, 14:40**

I've updated the WEB-DL {source} pattern to make this work more easily out-of-the-box:
https://github.com/filebot/data/blob/ma ... ources.txt

Posted: **10 Jun 2020, 06:06**

Is there a way I can either revert back to an older "{source}" scheme or something, recent changes to the {source} has botched my naming scheme up and I see no way to fix it other than removing my source stuff and also writing up a bunch of "replaceall" stuff to fix various things or something ...

1. Way I used to have it when something was missing from the {source} I would have it name the {source} just "WEB-DL" with

Code: Select all

{"${self.source ?: 'WEB-DL'}"}

this was mostly when the download had the source as just "web", now it no longer seems to do this and just puts the source as "web" when I much prefer it to say "WEB-DL".. That'd require a "replaceall" for each source that wasn't there before that I'd prefer to have just set as "WEB-DL" ontop of trying to make sure that the name of the actual show or episode didn't have that bit from the "replaceall" in there as well..

2. Way I used to have it correlates with the above, any "webdl" that I had downloaded I actually want to have the "-" in between the "web" and the 'dl" part, this requires another "replaceall" making at least a minimum of two to bring back my naming scheme a little bit.

3. Way I used to have it when something had a websource like AMZN or any of the others there was a space between the AMZN bit and the source bit, now is no longer the case as they are now seemingly squished together, once again requiring a "replaceall".

This was my current websource / sources bit from my script that was working perfectly fine until this recent change in {source}

Code: Select all

{ def websources = new File('path/to/websource.txt')
                    def websource = websources.exists() ? lines(websources).join("|") : null // to avoid non-existing paths failing the whole block

                    def lfr = { fn.match(/\.($websource)\./) }
                    return allOf{fn.match(/(?i)(UHD).$source/).upper()}{hc}{lfr}{"${self.source ?: 'WEB-DL'}"}.join(" ") }

Posted: **10 Jun 2020, 06:41**

ZeroByDivide wrote: ↑10 Jun 2020, 06:06 when the download had the source as just "web", now it no longer seems to do this and just puts the source as "web" when I much prefer it to say "WEB-DL"

Have you tried the recently introduced {vs} source class binding yet?
viewtopic.php?t=11265

ZeroByDivide wrote: ↑10 Jun 2020, 06:06 Way I used to have it when something had a websource like AMZN or any of the others there was a space between the AMZN bit and the source bit, now is no longer the case as they are now seemingly squished together

The updated pattern requires a separator between AMZN and WEB.DL and so if things are seemingly squished together now, then it's most likely your code that's making it so.

Posted: **10 Jun 2020, 07:00**

rednoah wrote: ↑10 Jun 2020, 06:41
ZeroByDivide wrote: ↑10 Jun 2020, 06:06 when the download had the source as just "web", now it no longer seems to do this and just puts the source as "web" when I much prefer it to say "WEB-DL"
Have you tried the recently introduced {vs} source class binding yet?
viewtopic.php?t=11265

I actually just now found that, as I was looking through naming schemes again over at http://www.filebot.net/naming.html and so far it seems to at least be working how I prefer. Not sure how I missed it though all this time since it seems it was introduced back in November of last year xD

rednoah wrote: ↑10 Jun 2020, 06:41
ZeroByDivide wrote: ↑10 Jun 2020, 06:06 Way I used to have it when something had a websource like AMZN or any of the others there was a space between the AMZN bit and the source bit, now is no longer the case as they are now seemingly squished together
The updated pattern requires a separator between AMZN and WEB.DL and so if things are seemingly squished together now, then it's most likely your code that's making it so.

even doing just plain old {source} in the filebot GUI editor produces something like

Code: Select all

AMZNWEBDL

even though the file name would be something like

Code: Select all

Random.File.Name.720p.AMZN.WEB-DL.DD+5.1.H.264.mkv

so filebot would go about producing something like

Code: Select all

Random Name AMZNWEBDL.mkv

for example: https://i.imgur.com/jVUyOiX.png

Posted: **10 Jun 2020, 07:36**

well I guess there are a few caveats to {vs} vs using {source}.. is there a way I could make it so that it only uses {vs} for any releases that just have web/web-dl for the source, while everything else uses {source}...

I'm tinkering around with if then else conditions a bit now, but if you can come up with something possibly that would work nicely then I would be very happy

Posted: **10 Jun 2020, 07:54**

{vs} is definitely more appropriate if you want to check for specific values, because {vs} is limited to specific values, while {source} could yield anything that matches the patterns.

{source} works correctly as far as I can tell:

Code: Select all

$ filebot -mediainfo *.mkv --format "{source} | {fn}"
AMZN.WEB-DL | Random.File.Name.720p.AMZN.WEB-DL.DD+5.1.H.264

What specific version / revision are you using?

Code: Select all

$ filebot -version
FileBot 4.9.1 (r7372) / OpenJDK Runtime Environment 14 / Windows 10 10.0 (amd64)

Posted: **10 Jun 2020, 08:02**

rednoah wrote: ↑10 Jun 2020, 07:54 {source} works correctly as far as I can tell:
Code: Select all
$ filebot -mediainfo *.mkv --format "{source} | {fn}"
AMZN.WEB-DL | Random.File.Name.720p.AMZN.WEB-DL.DD+5.1.H.264
What specific version / revision are you using?
Code: Select all
$ filebot -version
FileBot 4.9.1 (r7372) / OpenJDK Runtime Environment 14 / Windows 10 10.0 (amd64)

Well that would explain it, for some reason my filebot was still on version 4.9.0, I had to go into the microsoft store and manually update it since for some reason or another microsoft store doesn't want to auto update stuff -.- anyways that fixed that, tbh now that that issue I can just replace the "." with a space and from what I can tell things will hopefully be back to how they used to be for my naming scheme.

Posted: **10 Jun 2020, 08:04**

Another problem solved by just updating. My favourite kind.

Posted: **20 Jan 2021, 14:37**

Hi,
Quick question: is it possible to only get the websource abbreviation, so I could name my files something like xxx{“.$websource”}{“.$vf”}{“.vs”}.xxx, only thing I found is to use source.vf.vs, but that gives: xxx.amzn.web-dL.1080p.web-dl.xxx and it would be a little exhausting to use a lot of replacealls. Finally this would also give me files like: xxx.Bluray.1080p.bluray.xxx
Thanks in advance,
FunnyFunk

Posted: **20 Jan 2021, 15:24**

You mean {vs} source class as opposed to {source} source match?
viewtopic.php?t=11265

Posted: **20 Jan 2021, 15:49**

First of all, thanks for your quick reply.
My problem actually is at the beginning if I put source I get a file like this: xxx.amzn.web-dl.1080p.web-dl.xxx and I don’t want that first web-dl. Like I said previously the problem gets worse if it isn’t a web release, than I have something like xxx.bluray.1080p.bluray.xxx or xxx.hdtv.1080p.hdtv.xxx
FunnyFunk

Posted: **20 Jan 2021, 15:54**

Use only {vs} or {source} at the position where you want to insert one or the other, but don't do it twice in two different positions. You may need to use fn.match() with a custom pattern if you need something more specific.

FileBot

Web Sources

Web Sources

Re: Web Sources

Re: Web Sources

Re: Web Sources

Re: Web Sources

Re: Web Sources

Re: Web Sources

Re: Web Sources

Re: Web Sources

Re: Web Sources

Re: Web Sources

Re: Web Sources

Re: Web Sources

Re: Web Sources

Re: Web Sources

Re: Web Sources

Re: Web Sources

Re: Web Sources

Re: Web Sources

Re: Web Sources

Re: Web Sources

Re: Web Sources

Re: Web Sources

Re: Web Sources

Re: Web Sources