Adding servers to the China zone


#207

My post has disappeared so I’ll say it again:

I am pulling out from .cn on Monday in protest of stupid robot monitoring system whose only purpose now seems to be making pretty chart selfies of broken self.
I have been waiting for years but enough is enough.

When monitoring is fixed I might be back.

Leo


#208

I can confirm that its firewall problem on our end and we are gonna try to find good solution for that on next week.


#209

This MUST be done at global level. Only shifting .cn traffic to other Asia countries will cause these already unhealthy zones fail altogether.


#210

Please add our server to CN pool

https://www.pool.ntp.org/scores/185.255.55.20


#211

I think faelix’s proposal would be an easy stopgap to implement. Ntpd can deal with inaccurate NTP servers, it’s certainly much worse to provide only 3-4 overloaded servers than 50 servers that might not be totally accurate.

An hour ago, the zone had 0 active server, now it has 2, this is the worst situation. Simply serving all the servers in the zone would be better than what we’re doing now, until we can implement a long term solution.


#212

Simply serving all the servers in the zone would be better than what we’re doing now, until we can implement a long term solution.

I started monitoring servers in China a few minutes ago. I’ll upload results here at this time of tomorrow.


#213

Didn’t you just explain it yourself?

“Monitoring packets do get lost because they are in contention with other 400mln requests per hour being taken care of.”

The purpose of the monitoring server is not to draw pretty graphs. It’s just a side effect. The real purpose of the monitoring server is to make sure the NTP servers in the pool are healthy. If they seem to be dropping packets, they are clearly not healthy and should be marked as such. If you disagree, I would like to hear your reasoning for why you think it would be OK for NTP servers to drop requests.

You seem to be maxing out your 100Mbit connection’s capacity. I would try to reduce the number of incoming packets to a level that your connection can handle without dropping packets. This may mean that you would be serving, like, 350 million requests per hour, but with the difference that you would be in the pool constantly instead of dropping out every now and then. With this arrangement you would actually get more requests served per day. I don’t know your netspeed setting in the pool, but I would recommend the lowest setting (384kbit). Sadly the situation in the pool is that even with that setting you can still get request rates that exceed 100Mbit/s. In any case, the netspeed setting would be the first one to check. Actually, seeing that you serve the @ zone it looks like your netspeed setting is at least 768kbit/s.

If setting that setting to 384kbit/s is not sufficient, you could ask the pool admins to move that server to serve exclusively .cn traffic, ie. remove “@”, “uk” and “europe”. That would also reduce the request amount a bit, so that you might better be able to stay under 100Mbit/s.

Edit: This same response also applies to @Hedberg.


#214

I have to point out that UDP is not a guaranteed delivery service. UDP packets can get dropped and will get dropped at any bandwidth settings due to collisions. This is stochastic and unavoidable process.

Chinese zone now has three/four servers, all chock full, serving users at wire speeds. Downgrading one or two of them to the level when monitoring station can reliably get through will require reducing their traffic to probably 20-30% of what it is now.
Please explain how forcefully reducing two out of four servers’ load will help the situation?

This is unbelievably backwards. To keep one single monitoring station happy I am supposed to reject 200-300mln requests per hour just to make sure the line is not busy when LA is calling.

I am happy to run servers at full 100Mbps speed. I am only asking to stop dropping me out. You are suggesting to instead reduce server traffic and everything will be fine. But where will rejected users go?

Also, it does not matter what the speed settings servers have if there are only 4 of them in the zone. All four of them will be dished out in the same DNS query.

Leo


#215

@avij
The kbps setting doesnt work very well and not at all when there is only a couple of hosts left in a zone.


#216

Maybe it’s not useful…?
The NTP request from china also kill my VDSL router almost .


The request is not dropped by server , but dropped by router .

I don’t know why my server got so much NTP request from china…
even my server is not in china zone
https://www.ntppool.org/scores/106.104.162.193


#217

It is in the asia zone and when .cn crashes all traffic goes to asia.


#218

You sure about that @Hedberg ?
I have a server in asia, singapore, i don’t see any more traffic than usual, with a bandwidth setting to 500Mbit.
https://www.ntppool.org/scores/209.58.172.142


#219

There have been significant increase in the number of .cn servers in the last two hours.
http://www.leobodnar.com/balloons/files/ntppool/cn_pool_rotation.html
Let’s see how long .cn will last this time.

Note that none of the servers is managing to stay above 10 points mark for too long.
http://www.leobodnar.com/balloons/files/ntppool/ntp_servers_cn.html

Leo

P.S. These pages are not dynamic, I have copied them from my local host.


#220

Leo, what are the funky bars on the pool rotation page? Is is supposed to be country flags or something?


#221

I believe that means the server was returned by some of the cn pool domains (with colors below).

Looks like 4 servers (including mine) were serving the whole zone in the beginning, and returned by all subdomains. The load was about 130Mbps so probably all 100Mbps servers were kicked over and over again. The load of my server is only ~30Mbps now, hopefully that’s the maximum since I have set my server bandwidth to 1000M.


#222

I just purchased a year in hong kong at hostus.us for $25 that I will dedicate to this purpose. It is pending at the moment, I imagine to be setup or something but I will let you know, if they don’t gank me.

Noah
https://logiplex.net
tick.nj.logiplex.net


#223

This right here. Half a Tibyte on 1 gig and 256 MiByte of memory. It’s not much but it should be able to handle a few thousand requests a second.


#224

Got some fresh data directly in China, raw data here.

Server listed here all (were) a part of cn pool, I ran a simple script to monitoring them in China. @gfk You may want look this.


#225

@LeoBodnar: Please, for the sake of everyone, stop demanding. The pool is run by volunteers, including the monitoring system code and hosting. If you are right, explain that you are right in a polite, respectful, and constructive manner, and people will listen.

At the risk of repeating @avij’s post but not doing a good job of it, let’s talk about your figures: your graph above shows your server topping out at 110K rps. All clients which query your server when you’re at maximum rps will get dropped - not just those which get dropped due to occasional link congestion as is the case with all UDP. If the monitoring station is one of those, it will get dropped. If it’s other clients they will get dropped. So from the perspective of those clients, your NTP server isn’t responding. And that’s what we see in the CSV record for your server - occasional I/O timeouts.

What you seem to be arguing for is a special exception for your server, that it should be just left in the pool all the time, regardless of its ability to serve clients, because those clients will try again later, and some of them will likely get through (because with a random-ish distribution of requests, different clients will get dropped each time).

This might seem reasonable to you, but can you see how it might not be ideal for those clients who were hoping to get their NTP requests serviced? If there are 3 hosts which are not dropping packets and yours is, it’s much more reasonable to simply drop yours out of the zone when it reaches saturation point, and put it back in when traffic drops a bit. Alternatively, you could tune your bandwidth session down a bit so that your server is just below the threshold of dropping packets. (I recognise this might be hard with the bandwidth levels currently available in the pool management interface, but that’s something that should be relatively easy to improve in the code.)

Manual handling of individual servers isn’t practical for a project like the NTP pool. If you can think of a better approach that doesn’t require manual handling, but can be implemented automatically by the monitoring system, I’m sure everyone would be keen to see your patches to the pool code.


#226

Ok that VPS I mentioned is setup. It’s a lousy openvz container but IPv4 offset is showing .007 from the USA and IPv6 is showing an offset of .03 from the USA. Not too shabby and it’s up and doing nothing but serving NTP for a year. Too bad I couldn’t do a bunch of fancy scheduling on the machine to amp things up a bit but it shouldn’t need it since ntp is all it is doing.

It’s currently in the asia and hk zones and will have to wait for its score to rise. The address:

tick.hk.china.logiplex.net

I added it at 10mbit initially and I will test loads because I hear so much craziness in that area of the world.

Let me know if there is anything else that I can do.

Edit: the HK server averaged 4.32 MiByte a second on a 100MiByte file test from logiplex.net in NJ/USA so this should suffice but I still need to test load.

Edit: can I take this out of global? I’d like it to be just for china since that was the initial projection of this thread. How do I modify that. Asia might be alright.

Noah
https://logiplex.net