Page 1 of 1

Synology repository - IP is banned

Posted: 17 Jun 2022, 14:21
by szyb
Hi,

I'm a maintainer of synopackage.com and I need to contact with site administrator (I'm not allowed to use private messages as I am a new user on this forum so writing here on the forum).
The problem is that my hosting IP is banned, so it is not able to get any packages from filebot repository by the synopackage.com website. Would it be possible to add the hosting IPs to the whitelist on the Cloudflare?
If you need more information about the website or how it works, feel free to contact me. I will answer all the questions.
BR,
szyb

Re: Synology repository - IP is banned

Posted: 17 Jun 2022, 16:00
by rednoah
What is the IP that is banned?

Re: Synology repository - IP is banned

Posted: 17 Jun 2022, 18:19
by szyb
The banned IP is: 194.88.154.190, but my hosting is using also this one: 91.219.122.88
BR,
szyb

Re: Synology repository - IP is banned

Posted: 17 Jun 2022, 19:45
by rednoah
I've unbanned 194.88.154.190 via CloudFlare. Looks like it was auto-banned due to unusual traffic originating from there.


:!: Please request each resource no more than 1x per day (and not 1000+ times of times per day; most resources never change; the package index changes 2-3 times per year at most) and make sure to use GET requests with a If-Modified-Since header when requesting a resource you have requested before, so that the edge cache can just respond with 304 Not Modified when nothing has changed, which is almost all the time.

Re: Synology repository - IP is banned

Posted: 17 Jun 2022, 20:26
by szyb
Thank you for unbanning the IP. Below is some explanation of what had happen.

We use our internal cache mechanism which is caching all the responses from servers per CPU architecture & DSM version & distribution channel (stable/beta). The cache expiration is set globally to 10h for all resources (including filebot). So there should be only one request per those three indicators per 10h. However you probably noticed more than that and that happened due to the ban. Every single request to filebot was directed to your end, as the cache on our side was expired and it was a need to refresh it. The CloudFlare service refuses it and the circle has been closed.

I will try to change this behavior for filebot and probably for more repositories, but it will take a while. I'm not sure yet how long it will take, but this will be my priority. Already filed an issue at https://github.com/szyb/synopackage_dotnet/issues/305

I'll keep you informed.

EDIT: I did check my logs and count total number of different indicators (for response caching) and it is about 100 (CPU x DSM version x channel)... so when it comes with 10h expiration, there could be even ~300 requests per day (before ban, and far more after the ban) ... That is pretty much more than I expected.

Does your repository even care about the model's CPU architecture or channel stable/beta? All Synology devices provides such data to all added repositories. I can only see that you distinguish DSM versions: 7 and 6 (separate repository)

Re: Synology repository - IP is banned

Posted: 17 Jun 2022, 22:21
by rednoah
https://get.filebot.net/syno/ and https://get.filebot.net/syno/dsm6.json are just static files. You will always get the same response irregardless of query parameters, request headers, etc. The first is for DSM7 and the second for DSM6.

Re: Synology repository - IP is banned

Posted: 20 Jun 2022, 05:49
by szyb
I've just deployed hot fix (temporary) that will cache filebot responses for 24h and for all CPUs/DSM version/channel. That should give one request per 24h for main repository (for DSM 7) and one per 24h for DSM 6 as I added second repository to the list.
The permanent solution will be done within next days/weeks, but it will behave in the above manner.