I don't know whether this is Windows, uTorrent or Filebot.
uTorrent replies just love to blame Windows for everything; I'm a little skeptical at to whether they are right.
When a download completes it was moved from E:\Downloads to E:\Seeding.
For large files that process could take up to 20 minutes killing all uploads/downloads until it completes.
Then the Automation script was run and the file was then copied to E:\Completed.
For large files that process could take up to 20 minutes killing all uploads/downloads until it completes.
That makes uTorrent useless for anything up to 40 - 45 minuter per large file.
So I no longer move completed torrents; I just run the script over the original download folder. That solved one half of the problem.
Is there a way to eliminate the impact to the system when Filebot is doing it's stuff?
thanks
Copy/Move slow and stops uTorrent dead
Re: Copy/Move slow and stops uTorrent dead
1.
Moving a file of any size on the same filesystem should always take next to 0 milliseconds because the filesystem just changes the file entry. If it takes a long time, that indicates that the files are physically moved (i.e. data is read/written on disk).
Since you're saying that everything is on E drive, that sounds like it's on the same filesystem / drive, which means any move operation should be instant.
2.
I presume you're using the amc script, right? Most people would just use that and have FileBot create duplicates in a different file structure. Again, creating duplicates should be instant if source / destination file are on the same filesystem.
You can use --action hardlink to ensure that FileBot is always creating hardlinks (or fail) instead of physically copying files. Creating a hardlink will always take 0 milliseconds and require no data on disk to be read or written thus causing no impact on your system.
3.
In the end, it's always the underlying filesystem (i.e. Windows) that performs any of those operations. Any tool like utorrent or filebot will just ask Windows to move file from A to B and then Windows will do it, and if Windows does decide to copy & delete instead of move for some reason, there's not much any of us developers can do, because it's something that we won't be able to reproduce on our machines where things just work.
If E drive is an internal NTFS drive, I'd be surprised if something didn't work as expected, but if E is a mapped network drive, then things get more complicated and error prone within the Windows filesystem abstraction layer.
Moving a file of any size on the same filesystem should always take next to 0 milliseconds because the filesystem just changes the file entry. If it takes a long time, that indicates that the files are physically moved (i.e. data is read/written on disk).
Since you're saying that everything is on E drive, that sounds like it's on the same filesystem / drive, which means any move operation should be instant.
2.
I presume you're using the amc script, right? Most people would just use that and have FileBot create duplicates in a different file structure. Again, creating duplicates should be instant if source / destination file are on the same filesystem.
You can use --action hardlink to ensure that FileBot is always creating hardlinks (or fail) instead of physically copying files. Creating a hardlink will always take 0 milliseconds and require no data on disk to be read or written thus causing no impact on your system.
3.
In the end, it's always the underlying filesystem (i.e. Windows) that performs any of those operations. Any tool like utorrent or filebot will just ask Windows to move file from A to B and then Windows will do it, and if Windows does decide to copy & delete instead of move for some reason, there's not much any of us developers can do, because it's something that we won't be able to reproduce on our machines where things just work.
If E drive is an internal NTFS drive, I'd be surprised if something didn't work as expected, but if E is a mapped network drive, then things get more complicated and error prone within the Windows filesystem abstraction layer.
Please read the Re: Copy/Move slow and stops uTorrent dead
Thanks for your quick and detailed reply.
The files start on my E:\ Drive. They will end up on one of several remote server drives depending on what the content is (Music, Movie, TV Show, Documentary, sport or stand up comedy).
I seed to 1:1 for most content except documentaries which I seed to 90:1
Yes, I am using the amc script.
So, the files are COPIED to E:\media, where they will reside until I move them to their remote location (A Windows 2012 R2 server on my network). To use your words - the data is DUPLICATED on the same file system but it is never instant.
I am aware that a move is really nothing more than a rename.
What I find interesting is that I can copy 10 different files from my local drive to a mapped drive or vice versa and it has zero impact on uTorrent. But if uTorrent does the copy it's extremely inefficient. I figured the amc script was essentially a shell running within uTorrent which is why I was seeing the problem still with the script. I was hoping that the script was running external to uTorrent (ie the same as a native local copy). I'm not a developer so I don't know how to describe that in a programming context.
Secondly, I did note that uTorrent did seem to do a full copy and delete when moving files. That's why I stopped using the "Move completed downloads to..." parameter. I, like you, expected the move to be a simple rename, but alas it always copied/deleted the content. As that is not normal Windows behavior I politely suggested to uTorrent that thier program was suspect, which got me nothing but insults and angry denials.
Anyway, if there is no way around this I guess I have to go back to manually doing all the renaming and copying; skip the use of the script. The script is great and it resolves the issue of me ending up with duplicate entries in Kodi after I remember to rename a file that is already in the library.
Using hard links (mklink /h) would seem to be quicker. But I'm not up to speed with hard links. I'll have to have a play with it, but I'm guesing when I copy the linked file to a new location it will copy the original data to the new location from the original file, retaining the new name?
I do use symlinks extensively (mklink /d), which if I recall correctly on one of your posts, you said no one uses
I find that symlinks seem to be making a comeback as they solve the problems of badly written programs that won't let you move the data to a new location. Anyway, off topic
The files start on my E:\ Drive. They will end up on one of several remote server drives depending on what the content is (Music, Movie, TV Show, Documentary, sport or stand up comedy).
I seed to 1:1 for most content except documentaries which I seed to 90:1
Yes, I am using the amc script.
So, the files are COPIED to E:\media, where they will reside until I move them to their remote location (A Windows 2012 R2 server on my network). To use your words - the data is DUPLICATED on the same file system but it is never instant.
I am aware that a move is really nothing more than a rename.
What I find interesting is that I can copy 10 different files from my local drive to a mapped drive or vice versa and it has zero impact on uTorrent. But if uTorrent does the copy it's extremely inefficient. I figured the amc script was essentially a shell running within uTorrent which is why I was seeing the problem still with the script. I was hoping that the script was running external to uTorrent (ie the same as a native local copy). I'm not a developer so I don't know how to describe that in a programming context.
Secondly, I did note that uTorrent did seem to do a full copy and delete when moving files. That's why I stopped using the "Move completed downloads to..." parameter. I, like you, expected the move to be a simple rename, but alas it always copied/deleted the content. As that is not normal Windows behavior I politely suggested to uTorrent that thier program was suspect, which got me nothing but insults and angry denials.
Anyway, if there is no way around this I guess I have to go back to manually doing all the renaming and copying; skip the use of the script. The script is great and it resolves the issue of me ending up with duplicate entries in Kodi after I remember to rename a file that is already in the library.
Using hard links (mklink /h) would seem to be quicker. But I'm not up to speed with hard links. I'll have to have a play with it, but I'm guesing when I copy the linked file to a new location it will copy the original data to the new location from the original file, retaining the new name?
I do use symlinks extensively (mklink /d), which if I recall correctly on one of your posts, you said no one uses
I find that symlinks seem to be making a comeback as they solve the problems of badly written programs that won't let you move the data to a new location. Anyway, off topic Re: Copy/Move slow and stops uTorrent dead
Let's just use --action hardlink or --action symlink then and see what happens.
If there's no IO, then we neatly avoid IO bottlenecks.
If there's no IO, then we neatly avoid IO bottlenecks.
Re: Copy/Move slow and stops uTorrent dead
OK, done. We'll see what happens.
Thanks for your help
Thanks for your help