webservice.fanart.tv sending back 0 bytes

[kim]

when I scrape a show on thetvdb it downloads 0 byte artwork files can anyone confirm this ?
it was ok 1 week ago
the log url's check out ok

I have no problem on themoviedb

[kim]

if I try "script fn:artwork.tvdb"
I get this:

Match Series Name => [Cheers] [Cheers]
Query Series => [Cheers]
D:\_DoWnLoaD\Done\tv\Cheers => Search by Cheers
D:\_DoWnLoaD\Done\tv\Cheers => Cheers
Generate Series NFO: Cheers [77623]
Fetching D:\_DoWnLoaD\Done\tv\Cheers\poster.jpg => [poster/680x1000, en, 10.0, http://thetvdb.com/banners/posters/77623-7.jpg]
Fetching D:\_DoWnLoaD\Done\tv\Cheers\banner.jpg => [series/graphical, en, 10.0, http://thetvdb.com/banners/graphical/77623-g.jpg]
Fetching D:\_DoWnLoaD\Done\tv\Cheers\fanart.jpg => [fanart/graphical/1920x1080, en, 9.5, http://thetvdb.com/banners/fanart/original/77623-10.jpg]
Fetch resource: http://webservice.fanart.tv/v3/tv/77623 ... 2f392c2deb
Received 0 bytes
Fetch failed: http://webservice.fanart.tv/v3/tv/77623 ... 2f392c2deb
java.io.IOException: Response data is null: tv/77623 => http://webservice.fanart.tv/v3/tv/77623 ... 2f392c2deb
at net.filebot.CachedResource.lambda$get$1(CachedResource.java:96)
at net.filebot.Cache.computeIf(Cache.java:90)
at net.filebot.CachedResource.get(CachedResource.java:82)
at net.filebot.web.FanartTVClient.getArtwork(FanartTVClient.java:45)
at net.filebot.web.ArtworkProvider$getArtwork.call(Unknown Source)
at Script2.fetchSeriesFanart(Script2.groovy:112)
at Script2$_fetchSeriesArtworkAndNfo_closure5$_closure23.doCall(Script2.groovy:176)
at Script2$_fetchSeriesArtworkAndNfo_closure5.doCall(Script2.groovy:176)
at Script2$_fetchSeriesArtworkAndNfo_closure5.doCall(Script2.groovy)
at net.filebot.cli.ScriptShellBaseClass.tryLogCatch(ScriptShellBaseClass.java:141)
at Script2.fetchSeriesArtworkAndNfo(Script2.groovy:154)
at Script1$_run_closure1$_closure10.doCall(Script1.groovy:51)
at Script1$_run_closure1$_closure10.doCall(Script1.groovy)
at net.filebot.cli.ScriptShellBaseClass.tryLogCatch(ScriptShellBaseClass.java:141)
at Script1$_run_closure1.doCall(Script1.groovy:50)
at net.filebot.cli.ScriptShellMethods.eachMediaFolder(ScriptShellMethods.java:218)
at Script1.run(Script1.groovy:7)
at net.filebot.cli.ScriptShell.evaluate(ScriptShell.java:64)
at net.filebot.cli.ScriptShell.runScript(ScriptShell.java:74)
at net.filebot.cli.ArgumentProcessor.runScript(ArgumentProcessor.java:127)
at net.filebot.cli.ArgumentProcessor.run(ArgumentProcessor.java:29)
at net.filebot.Main.main(Main.java:111)

IOException: Response data is null: tv/77623 => http://webservice.fanart.tv/v3/tv/77623 ... 2f392c2deb
Done ヾ(＠⌒ー⌒＠)ノ

[kim]

viewtopic.php?f=6&t=5328

[rednoah]

Still not working? The resources in the log all seem to work now from my location.

[kim]

no change

Fetch resource: http://webservice.fanart.tv/v3/tv/77623 ... 2f392c2deb
Received 0 bytes
Fetch failed: http://webservice.fanart.tv/v3/tv/77623 ... 2f392c2deb
java.io.IOException: Response data is null: tv/77623 => http://webservice.fanart.tv/v3/tv/77623 ... 2f392c2deb
at net.filebot.CachedResource.lambda$get$1(CachedResource.java:96)

[kim]

can you/I make it so it bypasses fanart.tv ?
there is no need to use it i think... only when

Code: Select all

// fetch fanart
		['hdclearart', 'clearart'].findResult{ type -> fetchSeriesFanart(seriesDir.resolve('clearart.png'), seriesId, type, null, override, locale) }
		['hdtvlogo', 'clearlogo'].findResult{ type -> fetchSeriesFanart(seriesDir.resolve('logo.png'), seriesId, type, null, override, locale) }
fetchSeriesFanart(seriesDir.resolve('landscape.jpg'), seriesId, 'tvthumb', null, override, locale)
// fetch season fanart
		if (seasonDir != seriesDir) {
			fetchSeriesFanart(seasonDir.resolve('landscape.jpg'), seriesId, 'seasonthumb', season, override, locale)
}

and I do not use this at all

btw: I to can click/use the links from log in the browser and it works, but in the filebot script it fails

[rednoah]

You could use Fiddler to debug the HTTP requests. Presumably, CloudFlare is more likely to block requests made by tools such as FileBot while allowing requests from browsers when the load on the site is too high.

You can only disable --def artwork=n altogether.

[kim]

have you tried "script fn:artwork.tvdb" ?
because it sounds like you say "there is no problem" ?

As you can read it's not just me with the problem and I can't find any info on fanart.tv or thetvdb saying there is a problem..

In my custom scripts I do not use fanart.tv at all and the problem is the same, 0 byte files

I think I found the problem:
on fanart.tv and thetvdb
if you try the links from the log in browser it will change it from http to https...
so can you change it to https ?

EDIT:
YES, I can confirm fanart.tv and thetvdb on https is working

[rednoah]

If it works in the browser, but not from within FileBot, then the most likely explanation is that some requests are blocked while others aren't. This is what CloudFlare does and unfortunately there's not much I can do about that.

To confirm whether or not CloudFlare is indeed the culprit, you can use Fiddler to see the HTTP requests, so you can have a look at the HTTP error response and see what it says.

The other explanation is that fanart.tv is sending back 302 REDIRECT messages and FileBot somehow doesn't follow them. Not sure if that's the case. Again, Fiddler will tell you exactly what's going on.

I am replying with my mobile phone and won't be able to do any testing or debugging on my desktop computer for the next 1-2 days.

[kim]

HTTP/1.1 301 Moved Permanently

[rednoah]

I see. I guess it's not following redirects then. I'll have a look at the code on Sunday.

[rednoah]

Since webservice.fanart.tv doesn't rewrite HTTP to HTTPS anymore now, I guess this problem has resolved itself. I'll set it to use HTTPS for future revisions.

[kim]

same problem with thetvdb

info:
the get nfo info works and it "looks like" the artwork to, BUT it only writes 0 byte files
(when http, if https = all ok)

[rednoah]

Can you give me the URLs please?

[kim]

look above

why don't you just try scraping a show yourself ?

HTTP/1.1 301 Moved Permanently
Location: https://www.thetvdb.com/banners/posters/77623-7.jpg
Server: cloudflare-nginx
CF-RAY: 39fe94eae4143ca1-CPH
0

[rednoah]

I'll need to recreate a test case and reproduce the problem. You already have both. In this case, a good bug report will allow to understand and fix the problem directly, instead of spending time on figuring out what the problem is.

[SuperDuper]

I just wanted to confirm that I'm having this exact same problem as well. At least for the past 3 or 4 days, FileBot has been creating 0-byte files for the artwork.

If it helps, I run the following script from qBittorrent:

Code: Select all

C:\filebot -script C:\amc.groovy --output "D:\outputFolder" --action copy --conflict auto -non-strict --log-file amc.log --def excludeList=amc.excludes unsorted=y music=y artwork=y "ut_dir=%F" "ut_kind=multi" "ut_title=%N" "ut_label=%L" seriesFormat="TV Shows/{n}/{episode.special ? 'Special' : 'Season '+s.pad(2)}/{n} - {episode.special ? 'S00E'+special.pad(2) : s00e00.lower()} - {t.replaceAll(/[`´‘’ʻ]/, /'/).replaceAll(/[!?.]+$/).replacePart(', Part $1')}{'.'+lang}" musicFormat="Music/{n}/{album+'/'}{pi.pad(2)+' - '}{t}"

Server returned HTTP response code: 500 for URL: https://assets.fanart.tv/fanart/tv/2817 ... acc213.jpg
java.io.IOException: Server returned HTTP response code: 500 for URL: https://assets.fanart.tv/fanart/tv/2817 ... acc213.jpg
at org.apache.commons.io.FileUtils.copyURLToFile(FileUtils.java:1478)
at net.filebot.cli.ScriptShellMethods.saveAs(ScriptShellMethods.java:345)
at Script2.fetchSeriesFanart(Script2.groovy:118)
at Script2$_fetchSeriesArtworkAndNfo_closure5.doCall(Script2.groovy:177)
at Script2$_fetchSeriesArtworkAndNfo_closure5.doCall(Script2.groovy)
at net.filebot.cli.ScriptShellBaseClass.tryLogCatch(ScriptShellBaseClass.java:127)
at Script2.fetchSeriesArtworkAndNfo(Script2.groovy:153)
at Script1$_run_closure55$_closure135$_closure142.doCall(Script1.groovy:437)
at Script1$_run_closure55$_closure135.doCall(Script1.groovy:435)
at Script1$_run_closure55.doCall(Script1.groovy:432)
at Script1.run(Script1.groovy:412)
at net.filebot.cli.ScriptShell.evaluate(ScriptShell.java:62)
at net.filebot.cli.ScriptShell.runScript(ScriptShell.java:72)
at net.filebot.cli.ArgumentProcessor.runScript(ArgumentProcessor.java:114)
at net.filebot.cli.ArgumentProcessor.run(ArgumentProcessor.java:29)
at net.filebot.Main.main(Main.java:120)
Done ヾ(＠⌒ー⌒＠)ノ

[rednoah]

5xx error codes would indicate an unrelated server-side issue. For 5xx error codes the most likely explanation is an issue at assets.fanart.tv or at the CloudFlare edge cache.

The errors discussed above are about 3xx response codes which is a different class of problem which has been fixed with the latest revision.

[denywinarto]

same issue here, so no fix yet?

[rednoah]

This thread discusses a variety of unrelated issues. What exactly is the issue in your case? fanart.tv or thetvdb?

[denywinarto]

This thread discusses a variety of unrelated issues. What exactly is the issue in your case? fanart.tv or thetvdb?

i used a script to export fanart.jpg, poster.jpg and banner.jpg from thetvdb
But all 3 are now returning 0 kb files..

[rednoah]

I see. TheTVDB issue has been fixed with the latest revision. You can try the latest jar, or wait for the next stable release.

[denywinarto]

I see. TheTVDB issue has been fixed with the latest revision. You can try the latest jar, or wait for the next stable release.

Is Updating the only way to fix this?
Updating filebot breaks my script, i have to keep it under specific version (4.75)

[kim]

no, you can make you own LOCAL scripts (amc.groovy and htpc.groovy)... where you change http to https on the 2 sites (in "htpc.groovy" look for "...url.saveAs(outputFile)"

BUT it's better to fix the "breaks my script", you can try to ask for help

[denywinarto]

hmm it's been a while since i made the script.. any idea what makes this script broken in newer versions ?

Code: Select all

def override = _args.conflict == 'override'
import static groovy.json.StringEscapeUtils.*


/**
 * XBMC helper functions
 */
def scanVideoLibrary(host, port) {
	tryLogCatch {
		telnet(host, port) { writer, reader ->
			writer.println("""{"jsonrpc":"2.0","method":"VideoLibrary.Scan","id":1}""")
		}
	}
}

def showNotification(host, port, title, message, image) {
	tryLogCatch {
		telnet(host, port) { writer, reader ->
			writer.println("""{"jsonrpc":"2.0","method":"GUI.ShowNotification","params":{"title":"${escapeJavaScript(title)}","message":"${escapeJavaScript(message)}", "image":"${escapeJavaScript(image)}"},"id":1}""")
		}
	}
}



/**
 * Plex helpers
 */


/**
 * TheTVDB artwork/nfo helpers
 */
def fetchSeriesBanner(outputFile, seriesId, bannerType, bannerType2, season, override, locale) {
	if (outputFile.exists() && !override) {
		log.finest "Banner already exists: $outputFile"
		return outputFile
	}

	// select and fetch banner
	def artwork = TheTVDB.getArtwork(seriesId, bannerType, locale)
	def banner = [locale.language, null].findResult { lang -> artwork.find{ it.matches(bannerType2, season, lang) } }
	if (banner == null) {
		log.finest "Banner not found: $outputFile / $bannerType:$bannerType2"
		return null
	}
	log.finest "Fetching $outputFile => $banner"
	return banner.url.saveAs(outputFile)
}

def fetchSeriesFanart(outputFile, seriesId, type, season, override, locale) {
	if (outputFile.exists() && !override) {
		log.finest "Fanart already exists: $outputFile"
		return outputFile
	}

	def artwork = FanartTV.getArtwork(seriesId, "tv", locale)
	def fanart = [locale.language, null].findResult{ lang -> artwork.find{ it.matches(type, season, lang) } }
	if (fanart == null) {
		log.finest "Fanart not found: $outputFile / $type"
		return null
	}
	log.finest "Fetching $outputFile => $fanart"
	return fanart.url.saveAs(outputFile)
}

def fetchSeriesNfo(outputFile, i, locale) {
	log.fine "Generate Series NFO: $i.name [$i.id]"
	def xml = XML {
		tvshow {
			title(i.name)
			sorttitle([i.name, i.startDate as String].findAll{ it?.length() > 0 }.findResults{ it.sortName('$2') }.join(' :: '))
			year(i.startDate?.year)
			rating(i.rating)
			votes(i.ratingCount)
			plot(i.overview)
			runtime(i.runtime)
			mpaa(i.certification)
			id(i.id)
			i.genres.each{
				genre(it)
			}
			thumb(i.bannerUrl)
			premiered(i.startDate)
			status(i.status)
			studio(i.network)
			tvdb(id:i.id, "https://www.thetvdb.com/?tab=series&id=${i.id}")

			/** Kodi requires an <episodeguide> element with a TheTVDB API (v1) Series Record XML URL **/
			episodeguide {
				url(cache:"${i.id}.xml", "https://www.thetvdb.com/api/1D62F2F90030C444/series/${i.id}/all/${locale.language}.zip")
			}
		}
	}
	xml.saveAs(outputFile)
}
 

def fetchSeriesArtworkAndNfo(seriesDir, seasonDir, seriesId, season, override = false, locale = Locale.ENGLISH) {
	tryLogCatch {
		// fetch nfo
		def seriesInfo = TheTVDB.getSeriesInfo(seriesId, locale)
		fetchSeriesNfo(seriesDir.resolve('tvshow.nfo'), seriesInfo, locale)

		// fetch series banner, fanart, posters, etc
		['680x1000', null].findResult{ fetchSeriesBanner(seriesDir.resolve('poster.jpg'), seriesId, 'poster', it, null, override, locale) }
		['graphical', null].findResult{ fetchSeriesBanner(seriesDir.resolve('banner.jpg'), seriesId, 'series', it, null, override, locale) }

		// fetch highest resolution fanart
		['1920x1080', '1280x720', null].findResult{ fetchSeriesBanner(seriesDir.resolve('fanart.jpg'), seriesId, 'fanart', it, null, override, locale) }

		// fetch season banners
		if (seasonDir != seriesDir) {
			fetchSeriesBanner(seasonDir.resolve('poster.jpg'), seriesId, 'season', 'season', season, override, locale)
			fetchSeriesBanner(seasonDir.resolve('banner.jpg'), seriesId, 'seasonwide', 'seasonwide', season, override, locale)

			// folder image (resuse series poster if possible)
			copyIfPossible(seasonDir.resolve('poster.jpg'), seasonDir.resolve('folder.jpg'))
		}

		// fetch fanart
		['hdclearart', 'clearart'].findResult{ type -> fetchSeriesFanart(seriesDir.resolve('clearart.png'), seriesId, type, null, override, locale) }
		['hdtvlogo', 'clearlogo'].findResult{ type -> fetchSeriesFanart(seriesDir.resolve('logo.png'), seriesId, type, null, override, locale) }
		fetchSeriesFanart(seriesDir.resolve('landscape.jpg'), seriesId, 'tvthumb', null, override, locale)

		// fetch season fanart
		if (seasonDir != seriesDir) {
			fetchSeriesFanart(seasonDir.resolve('landscape.jpg'), seriesId, 'seasonthumb', season, override, locale)
		}

		// folder image (resuse series poster if possible)
		copyIfPossible(seriesDir.resolve('poster.jpg'), seriesDir.resolve('folder.jpg'))
	}
}


/**
 * TheMovieDB artwork/nfo helpers
 */






def copyIfPossible(File src, File dst) {
	if (src.exists() && !dst.exists()) {
		src.copyAs(dst)
	}
}

args.eachMediaFolder{ dir ->
	// fetch only missing artwork by default
	if (!override && dir.hasFile{it.name == 'banner.jpg'}) {
		println "Skipping $dir"
		return
	}
	
	def videos = dir.listFiles{ it.isVideo() }
	def query = _args.query ?: detectSeriesName(videos)
	def sxe = videos.findResult{ parseEpisodeNumber(it) }
	
	if (query == null) {
		query = dir.dir.hasFile{ it.name =~ /Season/ && it.isDirectory() } ? dir.dir.name : dir.name
	}
	
	println "$dir => Search by $query"
	def options = TheTVDB.search(query, _args.locale)
	if (options.isEmpty()) {
		println "TV Series not found: $query"
		return
	}
	
	// sort by relevance
	options = options.sortBySimilarity(query, { it.name })
	
	// auto-select series
	def series = options[0]
	
	// maybe require user input
	if (options.size() != 1 && !_args.nonStrict && !java.awt.GraphicsEnvironment.headless) {
		series = javax.swing.JOptionPane.showInputDialog(null, 'Please select TV Show:', dir.path, 3, null, options.toArray(), series)
		if (series == null) return
	}
	
	// auto-detect structure
	def seriesDir = [dir.dir, dir].sortBySimilarity(series.name, { it.name })[0]
	def season = sxe && sxe.season > 0 ? sxe.season : 1
	
	println "$dir => $series"
	fetchSeriesArtworkAndNfo(seriesDir, dir, series.id, season, false, _args.locale ?: Locale.ENGLISH)
}