Hiring: Automate/Scrape into SQLite3

[skchopperguy]

Hi,

My project is to better manage the files on my NAS, make directory structure/metadata 100% compatible with xbmc, run simple website with sqlite3. Would it be possible to have a filebot script which will:

1. Monitor watch directory (cron ?)
foreach $file{
2. Rename the file from imdb or tvdb{if contains season/episode identifier (like S02E03)
3. Move into tvshows/${tvdbid}/season${xx)/ or movies/${imdbid}/{imdbid}.{n.upperInitial().space('.')}.{y}.{SDHD-?-720p/1080p/DVD}
4. Scrape for MetaData and poster/banner/fanart {only download the first of each image;}
5. Insert imploded comma separated metadata into SQLite3 db
6. Insert each poster, banner, fanart AS BLOB into SQLite3 db

I'm thinking that 5,6 will not be possible but I could be wrong. In the very least, #4 is important as I don't want 40 images inside any directories. Have been toying with the idea of an external bash script to handle #'s 5,6 but I don't know how I'd read the nfo into an array for insert, might use php instead. So that would be quite time consuming for me. Would be cool if filebot could do it all.

I don't consider myself a coder, even though I can muddle through php/shell/html5 pretty well and kinda understand the scripts you have in the repository. Simply put, I'd rather pay for a pro to do this as I don't have time or the inclination to learn Java. I will contribute the final script to the community though as it could be useful to others. Even just as an example.

I'm thinking $150 USD or less. Unless you think that's insulting, then you tell me and we talk

Questions/thoughts/ideas?

[rednoah]

Did you check the AMC script? That pretty much covers 2 through 4, and 1 if it's called on download complete so you don't need any filesystem monitoring.

5 and 6 are not possible directly with filebot. You'd need a standalone tool for this. You could write that in java/python/etc but that's not in the budget.

Maybe there is already some tools for uploading blobs into SQLite3? If I can just call an external program for 5/6 on artwork I have already downloaded anyway then this is entirely feasable.

[skchopperguy]

Hi rednoah,

Yep I looked at the AMC script but I don't always use torrents. When I do, I still manually reveiw and convert file formats first, then put completed into /sorting and original into /archive.

Really I need something automated from the /sorting stage. I ran filebot -script fn:artwork.tvdb /sorting but this seemed to want me to choose each series of the file manually and did not grab any episode nfo's at all. Can't imagine how that would work from command line when I'm sleeping, lol. I ran filebot -script fn:artwork.tmdb /sorting and that did great but soo many backdrops.

I guess I'll have to look around more on this sqlite3 thing and see what I can find. I know an insert can be done from bash using the /path/to/filename.jpg but I've no clue on how to read the nfo file into an array and insert that from bash. So its seeming that all this would need to happen from java/python/php/etc which is out of budget...as the web/db part of my project is the whole reason for having a project

[rednoah]

1. You can do -non-strict with the artwork.* scripts to have it just guess instead of confirming with you.

2. I don't generate episode .nfo files at this point. Wouldn't be so hard to add.

3. If you have a tool for inserting images I'm sure it supports inserting new db rows as well right? FileBot has all the data.

EDIT: The database is hard not just because of coding but mostly for figuring out how the XBMC db works, setting up a test environment, etc. I don't even use XBMC so it's too mafan to get into that.

EDIT2: Though this would be kinda interesting to play with Groovy SQL. I'll just put my quote at 200 EUR. Maybe there's more than one person that would pay for this feature? I feel it's unfair if you'd paid xxx amout of money so that everyone else later can use it for free.

[skchopperguy]

Hi,

I've been playing around the past few days and now have a better understanding of what I need to do. No need to think about the db, I'm writing a php file that will traverse directories and insert the images along with reading the nfo and importing that. Which is all I need really.

So, what you can do for me is basically hack up a new version of the amc.groovy and the lib/htpc.groovy to do #'s 1-4 of the first post. I can zip up an example directory structure for both movies and tvshows which will include the nfo's and any images so you can see where they go and also exactly what information is needed to write the nfo file. Everything will be in its exact place, so it would be safe to follow the zip structure exactly.

I really see this kind of script as the one thing filebot is missing. I'd want this to end up being an awesome script that will basically transform any mess ... into a perfect hierarchy so that xbmc or any other media center does not need to go out on the net and scrape anything as its all local already. Every MC I've worked with does look local first, saving possibly hours of scanning time (mine takes 2.5 hrs) and its still no good.

Anyway, since the sql is not needed and you can literally follow the structure of the zip I can send...would that help to bring down the quote a bit? I'm not trying to be cheap, I don't want you to work for free, its just that everyone will use it later (your advantage) or even pay for it too. I mean just the fact that it would be compatible with the 3 major MC's alone opens up a large audience. People want 'easy'. Maybe make these scripts 'by donation' or something.

Let me know how to send you the zip.

[EDIT]
I've just figured out that this is going to be maddening. I'll be better off just to ditch my limited NAS and make/setup xbmc AS my NAS instead and use the built-in webserver and jsonrpc. Still would be cool to see a script though. I'll give this xbmc/nas thing a whirl over the next week and come back to you on this.

[EDIT 2]
Scratch that idea. The xbmc web-interface just won't do it for me at all. I took a deeper look into the filbot scripts, it started to look straight forward to do what I want (1-4). So now that I have hacked my own, I just have some questions on the scripting, so I will post the Q's here and when I'm done I will make a donation + script.