Synology Filebot Node Copying instead of Hardlinking

[oldsweatyman]

Hi, just got filebot, I'm getting some weird issues with the node and I'd like some help if possible.

Issue: Logs seem to be fine and yet file system of Synology (DS918+) shows that output folder has taken up data.

Settings for FileBotNode: https://snipboard.io/spYhRm.jpg

Log: https://pastebin.com/G5HnQxLK

Synology File System Result: https://snipboard.io/yZbOmr.jpg

Been googling/searching forums for hours, can't find a solution. Maybe it's the permissions? I don't know how to change them to allow me to hardlink. The renaming works fine.

I tried reflink as well. The same thing happens, same log comes out except instead of [HARDLINK] it says [CLONE] as intended of course but it still takes up the physical space.

quick edit:

System Info from FilebotNode:

Code: Select all

FileBot 4.8.6 (r6856)
JNA Native: 6.1.0
MediaInfo: 19.04
7-Zip-JBinding: 9.20
Chromaprint: java.io.IOException: Cannot run program "fpcalc": error=2, No such file or directory
Extended Attributes: OK
Unicode Filesystem: OK
Script Bundle: 2019-10-14 (r601)
Groovy: 2.5.8
JRE: OpenJDK Runtime Environment 13.0.1
JVM: 64-bit OpenJDK 64-Bit Server VM
CPU/MEM: 4 Core / 6 GB Max Memory / 52 MB Used Memory
OS: Linux (amd64)
HW: Linux TheVault 4.4.59+ #24922 SMP PREEMPT Mon Aug 19 12:13:37 CST 2019 x86_64 GNU/Linux synology_apollolake_918+
STORAGE: ext4 [/] @ 1.3 GB | btrfs [/volume1] @ 1.2 TB | btrfs [/volume1/@docker] @ 1.2 TB | btrfs [/volume1/@docker/btrfs] @ 1.2 TB
DATA: /volume1/@appstore/filebot/data/admin
Package: SPK
License: FileBot License P10097800 (Valid-Until: 2020-11-11)
Done ヾ(＠⌒ー⌒＠)ノ

[rednoah]

How did you confirm that the new files indeed do or do not use up physical space space?

https://stackoverflow.com/a/19952240/1514467


Note that all files are hardlinks, and two hardlinks that point to the same physical disk space are indistinguishable, and may very well be counted one or more times towards the total, if the code counting the total file size doesn't explicitly take hardlink duplicates into account.


If you use CoW clones, then that's even more so the case, because files won't have the same inode, don't even have to be binary equal to share most of the disk space. AFAIK, once CoW clones come into play, it becomes conceptually impossible to "correctly" count "used disk space" for a given set of files.


TL;DR It's already working. You're just looking at the wrong metric.

[oldsweatyman]

Hmmm well that's a relief lol. Spent so many hours trying to figure it out.. Thanks for the help.

Do you know of a way for me to check the "real" metric on Synology? Or should I ask on their forums?

[rednoah]

If you login via SSH, then you can use du and df to count file size and see disk usage.


Generally speaking, even though you can't count file size, you can most certainly check your overall disk usage / remaining disk space. Any operation that applies to the whole file system will work. I'm sure Synology DSM has a pie chart for those metrics somewhere.

Let's say you have a 1 TB drive. If you create a new 100 GB file, then the remaining disk space gonna go down by 100 GB, but if you then create a few hundred hardlinks of that 100 GB file, the remaining disk space is not going to change, even though you have hundreds of 100 GB files in your file system.

[oldsweatyman]

Awesome, thank you!

Just to confirm you were certainly correct because my new "media" file size + the "original" file size is more than what is available on my physical storage so, checks out. Lol. I went with reflinks.

Interestingly, the "storage pool" on DSM is shown as full, but the "volume" is not. Just for future reference I suppose.