webservice.fanart.tv sending back 0 bytes

All your suggestions, requests and ideas for future development
kim
Power User
Posts: 856
Joined: 15 May 2014, 16:17

webservice.fanart.tv sending back 0 bytes

Post by kim » 13 Sep 2017, 15:42

when I scrape a show on thetvdb it downloads 0 byte artwork files can anyone confirm this ?
it was ok 1 week ago
the log url's check out ok

I have no problem on themoviedb

kim
Power User
Posts: 856
Joined: 15 May 2014, 16:17

Re: thetvdb artwork problem

Post by kim » 13 Sep 2017, 16:29

if I try "script fn:artwork.tvdb"
I get this:
Match Series Name => [Cheers] [Cheers]
Query Series => [Cheers]
D:\_DoWnLoaD\Done\tv\Cheers => Search by Cheers
D:\_DoWnLoaD\Done\tv\Cheers => Cheers
Generate Series NFO: Cheers [77623]
Fetching D:\_DoWnLoaD\Done\tv\Cheers\poster.jpg => [poster/680x1000, en, 10.0, http://thetvdb.com/banners/posters/77623-7.jpg]
Fetching D:\_DoWnLoaD\Done\tv\Cheers\banner.jpg => [series/graphical, en, 10.0, http://thetvdb.com/banners/graphical/77623-g.jpg]
Fetching D:\_DoWnLoaD\Done\tv\Cheers\fanart.jpg => [fanart/graphical/1920x1080, en, 9.5, http://thetvdb.com/banners/fanart/original/77623-10.jpg]
Fetch resource: http://webservice.fanart.tv/v3/tv/77623 ... 2f392c2deb
Received 0 bytes
Fetch failed: http://webservice.fanart.tv/v3/tv/77623 ... 2f392c2deb
java.io.IOException: Response data is null: tv/77623 => http://webservice.fanart.tv/v3/tv/77623 ... 2f392c2deb
at net.filebot.CachedResource.lambda$get$1(CachedResource.java:96)
at net.filebot.Cache.computeIf(Cache.java:90)
at net.filebot.CachedResource.get(CachedResource.java:82)
at net.filebot.web.FanartTVClient.getArtwork(FanartTVClient.java:45)
at net.filebot.web.ArtworkProvider$getArtwork.call(Unknown Source)
at Script2.fetchSeriesFanart(Script2.groovy:112)
at Script2$_fetchSeriesArtworkAndNfo_closure5$_closure23.doCall(Script2.groovy:176)
at Script2$_fetchSeriesArtworkAndNfo_closure5.doCall(Script2.groovy:176)
at Script2$_fetchSeriesArtworkAndNfo_closure5.doCall(Script2.groovy)
at net.filebot.cli.ScriptShellBaseClass.tryLogCatch(ScriptShellBaseClass.java:141)
at Script2.fetchSeriesArtworkAndNfo(Script2.groovy:154)
at Script1$_run_closure1$_closure10.doCall(Script1.groovy:51)
at Script1$_run_closure1$_closure10.doCall(Script1.groovy)
at net.filebot.cli.ScriptShellBaseClass.tryLogCatch(ScriptShellBaseClass.java:141)
at Script1$_run_closure1.doCall(Script1.groovy:50)
at net.filebot.cli.ScriptShellMethods.eachMediaFolder(ScriptShellMethods.java:218)
at Script1.run(Script1.groovy:7)
at net.filebot.cli.ScriptShell.evaluate(ScriptShell.java:64)
at net.filebot.cli.ScriptShell.runScript(ScriptShell.java:74)
at net.filebot.cli.ArgumentProcessor.runScript(ArgumentProcessor.java:127)
at net.filebot.cli.ArgumentProcessor.run(ArgumentProcessor.java:29)
at net.filebot.Main.main(Main.java:111)

IOException: Response data is null: tv/77623 => http://webservice.fanart.tv/v3/tv/77623 ... 2f392c2deb
Done ヾ(@⌒ー⌒@)ノ

kim
Power User
Posts: 856
Joined: 15 May 2014, 16:17

Re: thetvdb artwork problem

Post by kim » 13 Sep 2017, 16:49


User avatar
rednoah
The Source
Posts: 17355
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: thetvdb artwork problem

Post by rednoah » 14 Sep 2017, 01:48

Still not working? The resources in the log all seem to work now from my location.
:idea: Please read the FAQ and How to Request Help.

kim
Power User
Posts: 856
Joined: 15 May 2014, 16:17

Re: thetvdb artwork problem

Post by kim » 14 Sep 2017, 13:34

no change
Fetch resource: http://webservice.fanart.tv/v3/tv/77623 ... 2f392c2deb
Received 0 bytes
Fetch failed: http://webservice.fanart.tv/v3/tv/77623 ... 2f392c2deb
java.io.IOException: Response data is null: tv/77623 => http://webservice.fanart.tv/v3/tv/77623 ... 2f392c2deb
at net.filebot.CachedResource.lambda$get$1(CachedResource.java:96)

kim
Power User
Posts: 856
Joined: 15 May 2014, 16:17

Re: thetvdb artwork problem

Post by kim » 14 Sep 2017, 13:41

can you/I make it so it bypasses fanart.tv ?
there is no need to use it i think... only when

Code: Select all

// fetch fanart
		['hdclearart', 'clearart'].findResult{ type -> fetchSeriesFanart(seriesDir.resolve('clearart.png'), seriesId, type, null, override, locale) }
		['hdtvlogo', 'clearlogo'].findResult{ type -> fetchSeriesFanart(seriesDir.resolve('logo.png'), seriesId, type, null, override, locale) }
fetchSeriesFanart(seriesDir.resolve('landscape.jpg'), seriesId, 'tvthumb', null, override, locale)
// fetch season fanart
		if (seasonDir != seriesDir) {
			fetchSeriesFanart(seasonDir.resolve('landscape.jpg'), seriesId, 'seasonthumb', season, override, locale)
}
and I do not use this at all

btw: I to can click/use the links from log in the browser and it works, but in the filebot script it fails

User avatar
rednoah
The Source
Posts: 17355
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: thetvdb artwork problem

Post by rednoah » 14 Sep 2017, 15:01

You could use Fiddler to debug the HTTP requests. Presumably, CloudFlare is more likely to block requests made by tools such as FileBot while allowing requests from browsers when the load on the site is too high.

You can only disable --def artwork=n altogether.
:idea: Please read the FAQ and How to Request Help.

kim
Power User
Posts: 856
Joined: 15 May 2014, 16:17

Re: thetvdb artwork problem

Post by kim » 14 Sep 2017, 16:25

have you tried "script fn:artwork.tvdb" ?
because it sounds like you say "there is no problem" ?

As you can read it's not just me with the problem and I can't find any info on fanart.tv or thetvdb saying there is a problem..

In my custom scripts I do not use fanart.tv at all and the problem is the same, 0 byte files

I think I found the problem:
on fanart.tv and thetvdb
if you try the links from the log in browser it will change it from http to https...
so can you change it to https ?

EDIT:
YES, I can confirm fanart.tv and thetvdb on https is working

User avatar
rednoah
The Source
Posts: 17355
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: thetvdb artwork problem

Post by rednoah » 15 Sep 2017, 03:20

If it works in the browser, but not from within FileBot, then the most likely explanation is that some requests are blocked while others aren't. This is what CloudFlare does and unfortunately there's not much I can do about that.

To confirm whether or not CloudFlare is indeed the culprit, you can use Fiddler to see the HTTP requests, so you can have a look at the HTTP error response and see what it says.

The other explanation is that fanart.tv is sending back 302 REDIRECT messages and FileBot somehow doesn't follow them. Not sure if that's the case. Again, Fiddler will tell you exactly what's going on.

:idea: I am replying with my mobile phone and won't be able to do any testing or debugging on my desktop computer for the next 1-2 days.
:idea: Please read the FAQ and How to Request Help.

kim
Power User
Posts: 856
Joined: 15 May 2014, 16:17

Re: thetvdb artwork problem

Post by kim » 15 Sep 2017, 04:49

HTTP/1.1 301 Moved Permanently

User avatar
rednoah
The Source
Posts: 17355
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: thetvdb artwork problem

Post by rednoah » 15 Sep 2017, 05:33

I see. I guess it's not following redirects then. I'll have a look at the code on Sunday.
:idea: Please read the FAQ and How to Request Help.

User avatar
rednoah
The Source
Posts: 17355
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: thetvdb artwork problem

Post by rednoah » 17 Sep 2017, 14:48

Since webservice.fanart.tv doesn't rewrite HTTP to HTTPS anymore now, I guess this problem has resolved itself. I'll set it to use HTTPS for future revisions.
:idea: Please read the FAQ and How to Request Help.

kim
Power User
Posts: 856
Joined: 15 May 2014, 16:17

Re: thetvdb artwork problem

Post by kim » 17 Sep 2017, 15:35

same problem with thetvdb

info:
the get nfo info works and it "looks like" the artwork to, BUT it only writes 0 byte files
(when http, if https = all ok)

User avatar
rednoah
The Source
Posts: 17355
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: thetvdb artwork problem

Post by rednoah » 17 Sep 2017, 16:38

Can you give me the URLs please?
:idea: Please read the FAQ and How to Request Help.

kim
Power User
Posts: 856
Joined: 15 May 2014, 16:17

Re: thetvdb artwork problem

Post by kim » 17 Sep 2017, 19:45

look above

why don't you just try scraping a show yourself ?
HTTP/1.1 301 Moved Permanently
Location: https://www.thetvdb.com/banners/posters/77623-7.jpg
Server: cloudflare-nginx
CF-RAY: 39fe94eae4143ca1-CPH
0

User avatar
rednoah
The Source
Posts: 17355
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: thetvdb artwork problem

Post by rednoah » 18 Sep 2017, 03:14

I'll need to recreate a test case and reproduce the problem. You already have both. In this case, a good bug report will allow to understand and fix the problem directly, instead of spending time on figuring out what the problem is.
:idea: Please read the FAQ and How to Request Help.

SuperDuper
Posts: 1
Joined: 18 Sep 2017, 04:08

Re: thetvdb artwork problem

Post by SuperDuper » 18 Sep 2017, 04:24

I just wanted to confirm that I'm having this exact same problem as well. At least for the past 3 or 4 days, FileBot has been creating 0-byte files for the artwork.

If it helps, I run the following script from qBittorrent:

Code: Select all

C:\filebot -script C:\amc.groovy --output "D:\outputFolder" --action copy --conflict auto -non-strict --log-file amc.log --def excludeList=amc.excludes unsorted=y music=y artwork=y "ut_dir=%F" "ut_kind=multi" "ut_title=%N" "ut_label=%L" seriesFormat="TV Shows/{n}/{episode.special ? 'Special' : 'Season '+s.pad(2)}/{n} - {episode.special ? 'S00E'+special.pad(2) : s00e00.lower()} - {t.replaceAll(/[`´‘’ʻ]/, /'/).replaceAll(/[!?.]+$/).replacePart(', Part $1')}{'.'+lang}" musicFormat="Music/{n}/{album+'/'}{pi.pad(2)+' - '}{t}"
Server returned HTTP response code: 500 for URL: https://assets.fanart.tv/fanart/tv/2817 ... acc213.jpg
java.io.IOException: Server returned HTTP response code: 500 for URL: https://assets.fanart.tv/fanart/tv/2817 ... acc213.jpg
at org.apache.commons.io.FileUtils.copyURLToFile(FileUtils.java:1478)
at net.filebot.cli.ScriptShellMethods.saveAs(ScriptShellMethods.java:345)
at Script2.fetchSeriesFanart(Script2.groovy:118)
at Script2$_fetchSeriesArtworkAndNfo_closure5.doCall(Script2.groovy:177)
at Script2$_fetchSeriesArtworkAndNfo_closure5.doCall(Script2.groovy)
at net.filebot.cli.ScriptShellBaseClass.tryLogCatch(ScriptShellBaseClass.java:127)
at Script2.fetchSeriesArtworkAndNfo(Script2.groovy:153)
at Script1$_run_closure55$_closure135$_closure142.doCall(Script1.groovy:437)
at Script1$_run_closure55$_closure135.doCall(Script1.groovy:435)
at Script1$_run_closure55.doCall(Script1.groovy:432)
at Script1.run(Script1.groovy:412)
at net.filebot.cli.ScriptShell.evaluate(ScriptShell.java:62)
at net.filebot.cli.ScriptShell.runScript(ScriptShell.java:72)
at net.filebot.cli.ArgumentProcessor.runScript(ArgumentProcessor.java:114)
at net.filebot.cli.ArgumentProcessor.run(ArgumentProcessor.java:29)
at net.filebot.Main.main(Main.java:120)
Done ヾ(@⌒ー⌒@)ノ

User avatar
rednoah
The Source
Posts: 17355
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: thetvdb artwork problem

Post by rednoah » 18 Sep 2017, 05:05

5xx error codes would indicate an unrelated server-side issue. For 5xx error codes the most likely explanation is an issue at assets.fanart.tv or at the CloudFlare edge cache.

:idea: The errors discussed above are about 3xx response codes which is a different class of problem which has been fixed with the latest revision.
:idea: Please read the FAQ and How to Request Help.

denywinarto
Donor
Posts: 28
Joined: 30 Jul 2013, 01:15

Re: thetvdb artwork problem

Post by denywinarto » 20 Sep 2017, 01:23

same issue here, so no fix yet?

User avatar
rednoah
The Source
Posts: 17355
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: thetvdb artwork problem

Post by rednoah » 20 Sep 2017, 02:43

This thread discusses a variety of unrelated issues. What exactly is the issue in your case? fanart.tv or thetvdb?
:idea: Please read the FAQ and How to Request Help.

denywinarto
Donor
Posts: 28
Joined: 30 Jul 2013, 01:15

Re: thetvdb artwork problem

Post by denywinarto » 20 Sep 2017, 04:35

rednoah wrote:
20 Sep 2017, 02:43
This thread discusses a variety of unrelated issues. What exactly is the issue in your case? fanart.tv or thetvdb?
i used a script to export fanart.jpg, poster.jpg and banner.jpg from thetvdb
But all 3 are now returning 0 kb files..

User avatar
rednoah
The Source
Posts: 17355
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: webservice.fanart.tv sending back 0 bytes

Post by rednoah » 20 Sep 2017, 04:42

I see. TheTVDB issue has been fixed with the latest revision. You can try the latest jar, or wait for the next stable release.
:idea: Please read the FAQ and How to Request Help.

denywinarto
Donor
Posts: 28
Joined: 30 Jul 2013, 01:15

Re: webservice.fanart.tv sending back 0 bytes

Post by denywinarto » 20 Sep 2017, 06:56

rednoah wrote:
20 Sep 2017, 04:42
I see. TheTVDB issue has been fixed with the latest revision. You can try the latest jar, or wait for the next stable release.
Is Updating the only way to fix this?
Updating filebot breaks my script, i have to keep it under specific version (4.75)
Last edited by denywinarto on 22 Sep 2017, 03:42, edited 1 time in total.

kim
Power User
Posts: 856
Joined: 15 May 2014, 16:17

Re: webservice.fanart.tv sending back 0 bytes

Post by kim » 20 Sep 2017, 21:34

no, you can make you own LOCAL scripts (amc.groovy and htpc.groovy)... where you change http to https on the 2 sites (in "htpc.groovy" look for "...url.saveAs(outputFile)"

BUT it's better to fix the "breaks my script", you can try to ask for help ;)

denywinarto
Donor
Posts: 28
Joined: 30 Jul 2013, 01:15

Re: webservice.fanart.tv sending back 0 bytes

Post by denywinarto » 21 Sep 2017, 04:17

hmm it's been a while since i made the script.. any idea what makes this script broken in newer versions ?

Code: Select all

def override = _args.conflict == 'override'
import static groovy.json.StringEscapeUtils.*


/**
 * XBMC helper functions
 */
def scanVideoLibrary(host, port) {
	tryLogCatch {
		telnet(host, port) { writer, reader ->
			writer.println("""{"jsonrpc":"2.0","method":"VideoLibrary.Scan","id":1}""")
		}
	}
}

def showNotification(host, port, title, message, image) {
	tryLogCatch {
		telnet(host, port) { writer, reader ->
			writer.println("""{"jsonrpc":"2.0","method":"GUI.ShowNotification","params":{"title":"${escapeJavaScript(title)}","message":"${escapeJavaScript(message)}", "image":"${escapeJavaScript(image)}"},"id":1}""")
		}
	}
}



/**
 * Plex helpers
 */


/**
 * TheTVDB artwork/nfo helpers
 */
def fetchSeriesBanner(outputFile, seriesId, bannerType, bannerType2, season, override, locale) {
	if (outputFile.exists() && !override) {
		log.finest "Banner already exists: $outputFile"
		return outputFile
	}

	// select and fetch banner
	def artwork = TheTVDB.getArtwork(seriesId, bannerType, locale)
	def banner = [locale.language, null].findResult { lang -> artwork.find{ it.matches(bannerType2, season, lang) } }
	if (banner == null) {
		log.finest "Banner not found: $outputFile / $bannerType:$bannerType2"
		return null
	}
	log.finest "Fetching $outputFile => $banner"
	return banner.url.saveAs(outputFile)
}

def fetchSeriesFanart(outputFile, seriesId, type, season, override, locale) {
	if (outputFile.exists() && !override) {
		log.finest "Fanart already exists: $outputFile"
		return outputFile
	}

	def artwork = FanartTV.getArtwork(seriesId, "tv", locale)
	def fanart = [locale.language, null].findResult{ lang -> artwork.find{ it.matches(type, season, lang) } }
	if (fanart == null) {
		log.finest "Fanart not found: $outputFile / $type"
		return null
	}
	log.finest "Fetching $outputFile => $fanart"
	return fanart.url.saveAs(outputFile)
}

def fetchSeriesNfo(outputFile, i, locale) {
	log.fine "Generate Series NFO: $i.name [$i.id]"
	def xml = XML {
		tvshow {
			title(i.name)
			sorttitle([i.name, i.startDate as String].findAll{ it?.length() > 0 }.findResults{ it.sortName('$2') }.join(' :: '))
			year(i.startDate?.year)
			rating(i.rating)
			votes(i.ratingCount)
			plot(i.overview)
			runtime(i.runtime)
			mpaa(i.certification)
			id(i.id)
			i.genres.each{
				genre(it)
			}
			thumb(i.bannerUrl)
			premiered(i.startDate)
			status(i.status)
			studio(i.network)
			tvdb(id:i.id, "https://www.thetvdb.com/?tab=series&id=${i.id}")

			/** Kodi requires an <episodeguide> element with a TheTVDB API (v1) Series Record XML URL **/
			episodeguide {
				url(cache:"${i.id}.xml", "https://www.thetvdb.com/api/1D62F2F90030C444/series/${i.id}/all/${locale.language}.zip")
			}
		}
	}
	xml.saveAs(outputFile)
}
 

def fetchSeriesArtworkAndNfo(seriesDir, seasonDir, seriesId, season, override = false, locale = Locale.ENGLISH) {
	tryLogCatch {
		// fetch nfo
		def seriesInfo = TheTVDB.getSeriesInfo(seriesId, locale)
		fetchSeriesNfo(seriesDir.resolve('tvshow.nfo'), seriesInfo, locale)

		// fetch series banner, fanart, posters, etc
		['680x1000', null].findResult{ fetchSeriesBanner(seriesDir.resolve('poster.jpg'), seriesId, 'poster', it, null, override, locale) }
		['graphical', null].findResult{ fetchSeriesBanner(seriesDir.resolve('banner.jpg'), seriesId, 'series', it, null, override, locale) }

		// fetch highest resolution fanart
		['1920x1080', '1280x720', null].findResult{ fetchSeriesBanner(seriesDir.resolve('fanart.jpg'), seriesId, 'fanart', it, null, override, locale) }

		// fetch season banners
		if (seasonDir != seriesDir) {
			fetchSeriesBanner(seasonDir.resolve('poster.jpg'), seriesId, 'season', 'season', season, override, locale)
			fetchSeriesBanner(seasonDir.resolve('banner.jpg'), seriesId, 'seasonwide', 'seasonwide', season, override, locale)

			// folder image (resuse series poster if possible)
			copyIfPossible(seasonDir.resolve('poster.jpg'), seasonDir.resolve('folder.jpg'))
		}

		// fetch fanart
		['hdclearart', 'clearart'].findResult{ type -> fetchSeriesFanart(seriesDir.resolve('clearart.png'), seriesId, type, null, override, locale) }
		['hdtvlogo', 'clearlogo'].findResult{ type -> fetchSeriesFanart(seriesDir.resolve('logo.png'), seriesId, type, null, override, locale) }
		fetchSeriesFanart(seriesDir.resolve('landscape.jpg'), seriesId, 'tvthumb', null, override, locale)

		// fetch season fanart
		if (seasonDir != seriesDir) {
			fetchSeriesFanart(seasonDir.resolve('landscape.jpg'), seriesId, 'seasonthumb', season, override, locale)
		}

		// folder image (resuse series poster if possible)
		copyIfPossible(seriesDir.resolve('poster.jpg'), seriesDir.resolve('folder.jpg'))
	}
}


/**
 * TheMovieDB artwork/nfo helpers
 */






def copyIfPossible(File src, File dst) {
	if (src.exists() && !dst.exists()) {
		src.copyAs(dst)
	}
}

args.eachMediaFolder{ dir ->
	// fetch only missing artwork by default
	if (!override && dir.hasFile{it.name == 'banner.jpg'}) {
		println "Skipping $dir"
		return
	}
	
	def videos = dir.listFiles{ it.isVideo() }
	def query = _args.query ?: detectSeriesName(videos)
	def sxe = videos.findResult{ parseEpisodeNumber(it) }
	
	if (query == null) {
		query = dir.dir.hasFile{ it.name =~ /Season/ && it.isDirectory() } ? dir.dir.name : dir.name
	}
	
	println "$dir => Search by $query"
	def options = TheTVDB.search(query, _args.locale)
	if (options.isEmpty()) {
		println "TV Series not found: $query"
		return
	}
	
	// sort by relevance
	options = options.sortBySimilarity(query, { it.name })
	
	// auto-select series
	def series = options[0]
	
	// maybe require user input
	if (options.size() != 1 && !_args.nonStrict && !java.awt.GraphicsEnvironment.headless) {
		series = javax.swing.JOptionPane.showInputDialog(null, 'Please select TV Show:', dir.path, 3, null, options.toArray(), series)
		if (series == null) return
	}
	
	// auto-detect structure
	def seriesDir = [dir.dir, dir].sortBySimilarity(series.name, { it.name })[0]
	def season = sxe && sxe.season > 0 ? sxe.season : 1
	
	println "$dir => $series"
	fetchSeriesArtworkAndNfo(seriesDir, dir, series.id, season, false, _args.locale ?: Locale.ENGLISH)
}







Post Reply