Migrate from flat movie library to individual folders per movie

All about user-defined episode / movie / file name format expressions
Post Reply
beembum
Posts: 9
Joined: 11 Mar 2021, 00:32

Migrate from flat movie library to individual folders per movie

Post by beembum »

I am new to FileBot and for me it is a one-off migration tool, not intended for recurring use.
I have now spent several hours to figure out what is the right renaming expression for me, and failed.
I experimented, read the documentation and searched around in this forum.
Definitely I need help with this.

I would like to migrate from a relatively flat movie library structure to a structure where each movie is stored in an individual folder. There are some "easy" ways to do that, but they involve lookups to TMDB. I do not want this because my naming scheme and the Kodi Library are based on IMDB data, and often the publication year is not the same in IMDB and TMDB. There are also other differences.

In my case no lookups should be necessary at all, since I have a very clean and working movie file scheme.

So, what I want to do is the following:

For a given movie filename determine whether it contains a file extension,
(i) if it has an extension (e.g. mp4, avi, mkv, iso, img,...): prepend a new folder name as follows:
use everything left to and including the (year) of the movie filename
then append the movie filename {fn} and ".{ext}"

(ii) if it has no extension (BD, DVD folder structure): do not prepend a new folder name and just use the filename {fn} (which actually in this case is a folder name) as is

I started out with {fn.before("\\)")})/{fn}.{ext} for (i), but when I tried to introduce conditionals based on {ext} I always ran into parser errors, no matter what i tried.

I would really appreciate, if someone could help me out here. Although I did not find a solution for this here, I would assume it is interesting for many people who would migrate from a well groomed flat movie library to a subfolder structure.
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Migrate from flat movie library to individual folders per movie

Post by rednoah »

Most of the documentation will not be relevant to you, because your use case isn't a FileBot use case, and rather a generic batch rename use case.


However, FileBot does have Plain File Mode which should cover what you need:
viewtopic.php?t=2072


:idea: If you need help with prototyping a custom format, then we will need example file paths for testing. Ideally, one example file path for each of your conditions, with corresponding desired destination path.
:idea: Please read the FAQ and How to Request Help.
kim
Power User
Posts: 1251
Joined: 15 May 2014, 16:17

Re: Migrate from flat movie library to individual folders per movie

Post by kim »

Try KODI Import-export library/Video (Separate Files):
https://kodi.wiki/view/Import-export_library/Video

then if some is not as you like use Filebot
beembum
Posts: 9
Joined: 11 Mar 2021, 00:32

Re: Migrate from flat movie library to individual folders per movie

Post by beembum »

kim wrote: 11 Mar 2021, 16:03 Try KODI Import-export library/Video (Separate Files):
https://kodi.wiki/view/Import-export_library/Video

then if some is not as you like use Filebot
Hi Kim, thanks but this suggestion is completely besides the point. I intend to import not only in Kodi, but also into other tools. The name changes are motivated by the other tools, not by Kodi, as currently the old names work already fine with Kodi alone.
Thanks anyway and take care
beembum
Posts: 9
Joined: 11 Mar 2021, 00:32

Re: Migrate from flat movie library to individual folders per movie

Post by beembum »

rednoah wrote: 11 Mar 2021, 04:33 :idea: If you need help with prototyping a custom format, then we will need example file paths for testing. Ideally, one example file path for each of your conditions, with corresponding desired destination path.
Thank you so much. You will find the examples in a picture posted at the attached URL.
Left: current
Right: desired new

The new scheme slightly differs from my initial description as I learned about more limitations of the importing tool.

https://anonymousfiles.io/LkzHBSx4/
beembum
Posts: 9
Joined: 11 Mar 2021, 00:32

Re: Migrate from flat movie library to individual folders per movie

Post by beembum »

Are the examples OK for you to understand what I intend?
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Migrate from flat movie library to individual folders per movie

Post by rednoah »

Please provide example file paths as text pasted here in the forums. Sorry for the inconvenience.
:idea: Please read the FAQ and How to Request Help.
beembum
Posts: 9
Joined: 11 Mar 2021, 00:32

Re: Migrate from flat movie library to individual folders per movie

Post by beembum »

That was not just a matter of copy and paste, so it took a while until I had some time to prepare this:

Code: Select all

.../testimport/sd/img/en/The DVD Movie 8 (2026).img
.../newnames/The DVD Movie 8 (2026) - [EN]/The DVD Movie 8 (2026) - [EN].img

.../testimport/sd/img/en/The DVD Movie 7 (2023)/...dvd files...
.../newnames/The DVD Movie 7 (2023) - [EN]/The DVD Movie 7 (2023) - [EN]/...dvd files...

.../testimport/sd/img/fr/The DVD Movie 5 (2018)/...dvd files...
.../newnames/The DVD Movie 5 (2018) - [FR]/The DVD Movie 5 (2018) - [FR]/...dvd files...

.../testimport/sd/img/fr/The DVD Movie 6 - A wonderful Movie (1968).iso
.../newnames/The DVD Movie 6 - A wonderful Movie (1968) - [FR]/The DVD Movie 6 - A wonderful Movie (1968) - [FR].iso

.../testimport/sd/img/fr/The DVD Movie 6 - A wonderful Movie (1968).en.srt
.../newnames/The DVD Movie 6 - A wonderful Movie (1968) - [FR]/The DVD Movie 6 - A wonderful Movie (1968) - [FR].en.srt

.../testimport/sd/rip/fr/The AVI Movie 8 (2002) - PartA.avi
.../newnames/The AVI Movie 8 (2002) - [FR] - PartA.avi

.../testimport/sd/rip/fr/The AVI Movie 8 (2002) - PartB.avi
.../newnames/The AVI Movie 8 (2002) - [FR] - PartB.avi

.../testimport/sd/rip/en/The MP4 Movie 9 (2021).en.srt
.../newnames/The MP4 Movie 9 (2021) - [EN]/The MP4 Movie 9 (2021) - [EN].en.srt

.../testimport/sd/rip/en/The MP4 Movie 9 (2021).mp4
.../newnames/The MP4 Movie 9 (2021) - [EN]/The MP4 Movie 9 (2021) - [EN].mp4

.../hd/img/en/The Bluray Movie 1 (1981).iso	
.../newnames/The Bluray movie 1 (1981) - [EN]/The Bluray Movie 1 (1981) - [EN].iso

.../testimport/hd/img/fr/The Bluray Movie 2 (2012)/...bd files...
.../newnames/The Bluray movie 2 (2012) - [EN+FR]/The Bluray movie 2 (2012) - [EN+FR]/...bd files...

.../testimport/hd/rip/en/Collectionmovie 3 - The MKV Movie Three (2018).mkv
.../newnames/Collectionmovie 3 - The MKV Movie Three (2018) - [EN]/Collectionmovie 3 - The MKV Movie Three (2018) - [EN].mkv

.../testimport/hd/rip/fr/Collectionmovie 3 - The MKV Movie Three (2018) - 3D HSBS.fr.sub
../newnames/Collectionmovie 3 - The MKV Movie Three (2018) - 3D HSBS - [FR]/Collectionmovie 3 - The MKV Movie Three (2018) - 3D HSBS - [FR].fr.sub

.../testimport/hd/rip/fr/Collectionmovie 3 - The MKV Movie Three (2018) - 3D HSBS.mkv
../newnames/Collectionmovie 3 - The MKV Movie Three (2018) - 3D HSBS - [FR]/Collectionmovie 3 - The MKV Movie Three (2018) - 3D HSBS - [FR].mkv
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Migrate from flat movie library to individual folders per movie

Post by rednoah »

A custom format such as this should do the trick:

Code: Select all

.../newnames/{fn}/{fn}
:arrow: Please see Plain File Mode for details.
:idea: Please read the FAQ and How to Request Help.
beembum
Posts: 9
Joined: 11 Mar 2021, 00:32

Re: Migrate from flat movie library to individual folders per movie

Post by beembum »

I am not confused why you let me write up to all relevant examples to then tell me what I believe is evident already from my initial posting. I thought my posting was detailed and blunt and saying that I tried plain file mode and {fn}, but that it did not the trick, and that there are details that need more treatment, and that it is very finnicky to figure out - at least for me.
As a personal feedback: Your answer led me to believe that you needed me to write up all corner cases as examples to be able to provide the suitable expression. The response to that is - to be open and honest - disappointment.
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Migrate from flat movie library to individual folders per movie

Post by rednoah »

I work with what I have. Unfortunately, you didn't provide file paths for testing. I ask for file paths because I need to recreate your file structure for testing, i.e. it needs to be exact and machine-readable. If I cannot recreate your file structure, then my answer will be generic.


There are many ways to export file paths as text, none of them will take more than few seconds, e.g.

Code: Select all

$ find testimport -type f
testimport/sd/img/en/The DVD Movie 7 (2023)/VIDEO_TS/disk.mkv
testimport/sd/img/en/The DVD Movie 8 (2026).mkv

If you export file paths as text, then I can auto-generate your exact file structure for testing in less than a second:

Code: Select all

$ tree -N
.
└── testimport
    └── sd
        └── img
            └── en
                ├── The DVD Movie 7 (2023)
                │   └── VIDEO_TS
                │       └── disk.mkv
                └── The DVD Movie 8 (2026).mkv

After recreating your file structure, I throw in testimport, use F2 Plain File Mode, modify the format, check the proposed file paths, click Rename:

Code: Select all

~/Test/newnames/{fn}/{fn}
Image


Et voilà! Here's the destination file structure we get:

Code: Select all

$ tree ~/Test -N
~/Test
└── newnames
    ├── The DVD Movie 7 (2023)
    │   └── The DVD Movie 7 (2023)
    │       └── VIDEO_TS
    │           └── disk.mkv
    └── The DVD Movie 8 (2026)
        └── The DVD Movie 8 (2026).mkv

:idea: If this is exactly what you want, and it works for all your other files, then no additional file paths for testing are required. If this is not what you want, or if it doesn't work for some files for some reason, then file paths for those cases will be required for testing and fine-tuning.
:idea: Please read the FAQ and How to Request Help.
beembum
Posts: 9
Joined: 11 Mar 2021, 00:32

Re: Migrate from flat movie library to individual folders per movie

Post by beembum »

I am still confused. I gave you exactly old to new path mapping for all corner cases.
You already have all examples that you would need, you just picked 2 and misunderstood them.

What are you missing? You say you are missing file paths and then you pick the only examples that use a placeholder "...dvdfiles..." (likewise ...bdfiles...")?
All the other examples are file paths, right? Why would you not use all thee file paths, if you may have difficulty working with "...dvdfiles..."?

Lets look at this "...dvd files..." example:
.../testimport/sd/img/en/The DVD Movie 7 (2023)/...dvd files...
.../newnames/The DVD Movie 7 (2023) - [EN]/The DVD Movie 7 (2023) - [EN]/...dvd files...

you can leave out the starting "..." if they irritate you:

/testimport/sd/img/en/The DVD Movie 7 (2023)/...dvd files...
/newnames/The DVD Movie 7 (2023) - [EN]/The DVD Movie 7 (2023) - [EN]/...dvd files...

now about "...dvd files...": Maybe you do know how a DVD looks on the actual medium, that there are a bunch of files with arbitrary names that do not matter, i.e. that whatever "...dvdfiles..." actually represents shall be kept 1:1 in the new path. See it as a placeholder filename that is copied unchanged from the current path to the new path.

Your proposed solution is not I am looking for. But, I am repeating myself.

You already know what I am looking for because I gave you exact examples with all corner cases. You just need to look at them.

Lets look at the one example, again:

current:
/testimport/sd/img/en/The DVD Movie 7 (2023)/...dvd files...

new:
/newnames/The DVD Movie 7 (2023) - [EN]/The DVD Movie 7 (2023) - [EN]/...dvd files...

So why would you then propose just to use {fn}/{fn}? It very clearly does - not - produce the new file path.
In case you are not seeing it: there is " - [EN]" in the new path, and that can be inferred from "/en" on the old path (and for other formats as maybe an avi or mkv it could also be detected using the audio tracks).

maybe to me it was just clear to me and I need to spell it out:

each pair of example pathes shows
(1) the current path "/testimport/"...
and
(2) the new path "/newnames" ...

I mean the examples exactly as I wrote them down, so it would not help me if you are leaving out things in the new path, just to be able to use {fn}. You likely may have seen that I already started that way (just see my first posting) and that it is not doing what I would like to have.

Are my examples now understandable to you or do you have concrete questions about my examples?
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Migrate from flat movie library to individual folders per movie

Post by rednoah »

1.
I now see that you're keen on injecting the current folder name (i.e. EN / FR / etc) into the proposed file structure, the {folder} (i.e. parent folder binding) binding can be used for that. You can extend the solution above by adding folder.name.upper() where desired:

Code: Select all

/path/to/newnames/{fn} - [{folder.name.upper()}]/{fn} - [{folder.name.upper()}]
Image
:idea: If you're processing a Folder item, then this folder item is treated as a single unit, conceptually the same as if it was a single file. {fn} will refer to the name of the folder. {folder} will refer to the parent folder of the folder. Move will move the folder as a whole. Copy will copy the folder as a whole.



2.
If you have a standard disk file structure like this:

Code: Select all

The DVD Movie 7 (2023)/VIDEO_TS/VIDEO_TS.BUP
The DVD Movie 7 (2023)/VIDEO_TS/VIDEO_TS.VOB
The DVD Movie 7 (2023)/VIDEO_TS/VIDEO_TS.IFO
then dropping in The DVD Movie 7 (2023) will add The DVD Movie 7 (2023) as a single folder item into the UI as shown in the screenshot above. In this case, ~/Movies/{fn}/{fn} works.


However, if you have a flattened disk file structure like this:

Code: Select all

The DVD Movie 8 (2023)/VIDEO_TS.BUP
The DVD Movie 8 (2023)/VIDEO_TS.VOB
The DVD Movie 8 (2023)/VIDEO_TS.IFO
then dropping in The DVD Movie 8 (2023) will add VIDEO_TS.BUP / VIDEO_TS.VOB / VIDEO_TS.IFO as file items as if you had dropped any generic folder. In this case, ~/Movies/{fn}/{fn} does not work.


In the latter case you have two options:
* make your format handle all the necessary corner cases (possible but tricky, not recommended)
* force Folder Mode (i.e. Link-Drop) when dropping in the disk folders to process each folder as a single item (recommended, but requires a little bit of extra work when selecting and dropping in folders)
:idea: Please read the FAQ and How to Request Help.
beembum
Posts: 9
Joined: 11 Mar 2021, 00:32

Re: Migrate from flat movie library to individual folders per movie

Post by beembum »

OK now we are getting closer, have you looked at the other examples too?
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Migrate from flat movie library to individual folders per movie

Post by rednoah »

Which examples are not already fundamentally covered by the solution above? At this point, you just need to add bits and pieces to cover each of your special wishes. You can cover more or less of those depending on what level of format complexity you are comfortable with.


:idea: Based on your examples, I was able to guess at some of your special wishes:

* Cut away the " - PartA" at the end if it appears in the current file name

Code: Select all

{fn.before(/ - Part[A-Z]$/)}
* Insert " - PartA" at the end if it appears in the current file name:

Code: Select all

{fn.match(/ - Part[A-Z]$/)}
* Insert language suffix for subtitle files:

Code: Select all

{'.'+lang.ISO2}
* Detect audio languages from media content, pick the first 2, use + as separator, make it upper case:

Code: Select all

{audioLanguages.ISO2.take(2).join('+').upper()}
* Insert 3D side-by-side tag:

Code: Select all

{s3d}

:arrow: The next step is loading all your files into FileBot and doing some rapid prototyping using the concepts and components you've learned until everything works the way you want. Please report back with the final format expression you come up, for the next guy stumbling on this thread.
:idea: Please read the FAQ and How to Request Help.
beembum
Posts: 9
Joined: 11 Mar 2021, 00:32

Re: Migrate from flat movie library to individual folders per movie

Post by beembum »

you are asking me which examples wouldn't be covered? You were asking for the examples in the first place :) Funny.

For the bits and pieces to work together I would need conditionals (see my initial posting at the top), and this is where things stopped working due to unclear syntax errors.

As I tried that before and invested reasonable time already, I don't see how this would work out now, as the main problem remains untouched.
User avatar
rednoah
The Source
Posts: 22923
Joined: 16 Nov 2011, 08:59
Location: Taipei
Contact:

Re: Migrate from flat movie library to individual folders per movie

Post by rednoah »

Well, it's going to be a back-and-forth because we can't know what you have nor what you want. We're writing code for a semi-unknown use case. This is the way.


:!: Note that you have not provided a exact file paths nor screenshots of what you're doing or where you're stuck (e.g. compare and contrast my posts with your posts) so there is going to be lots of unnecessary friction caused by us not being on the same page, talking apples and pears. It's a bit baffling, but I guess you're just not comfortable with sharing real file paths? As explained above, a difference in naming ...dvd files... (we still don't know the exact file paths you're working with) can vastly change the behavior. Like when you say "main problem" I still have no idea what you mean. Presumably, disk folders are not loaded as single item as explained above? Maybe? That's my best guess, but I have no way of knowing for sure.


I've recreated your file paths to the best if my knowledge (see screenshot below) and this works as specified:

Code: Select all

/path/to/newnames/
{fn.before(/ - Part[A-Z]$/)} - [{folder.name.upper()}]/
{fn.before(/ - Part[A-Z]$/)} - [{folder.name.upper()}]{fn.match(/ - Part[A-Z]$/)}{'.'+lang.ISO2}

1. Here's a set of files that represents all of the corner cases we need to code for as far as we know:
Image

2. Here's how FileBot will transform those files using the format above:
Image

3. Here's the final result:
Image




EDIT:

I spot an issue with the subtitle file getting it's own folder. Since it's conceptually the same problem as dealing with PartA / PartB we can take care of that like so:

Code: Select all

/
{fn.before(/ - Part[A-Z]$|[.][a-z]{2,3}$/)} - [{folder.name.upper()}]/
{fn.before(/ - Part[A-Z]$|[.][a-z]{2,3}$/)} - [{folder.name.upper()}]{fn.match(/ - Part[A-Z]$/)}{'.'+lang.ISO2}



:!: There is one exception though, this particular example will not work as specified:

Code: Select all

.../testimport/hd/img/fr/The Bluray Movie 2 (2012)/...bd files...
.../newnames/The Bluray movie 2 (2012) - [EN+FR]/The Bluray movie 2 (2012) - [EN+FR]/...bd files...
The code above will produce language tag [FR] based on the folder name fr and not [EN+FR]:

Code: Select all

.../newnames/The Bluray movie 2 (2012) - [FR]/The Bluray movie 2 (2012) - [FR]/...bd files...
:idea: Please read the FAQ and How to Request Help.
Post Reply