Then you also agree that if everybody followed your advise and changed their settings to the same minimum, then you would receive more traffic since your share of the total resources would rise? Making it even more impossible for newcommers to help.
Fresh data in China here, you can compare it with official monitoring system.
Raw data, about 213k records: csv mysql
Data collected in a server located in UESTC (Unfortuneately, we have no equipment to set an official monitoring station). Data start at 2019-03-02T05:57:53.634Z, end at 2019-03-03T04:48:35.146Z.
No. If you had bothered to read my message to Leo, you would have noticed that the goal was to adjust his netspeed lower so that his servers would serve more traffic, due to his servers getting kicked out of the pool less frequently with the more appropriate netspeed setting. This would help reduce the traffic that would need to be sent to other pool servers.
My server is currently serving 200kpps at ~180Mbps. I just added two more unmetered gigabyte servers with the same config to the pool, please add them to the cn zone too:
https://www.ntppool.org/scores/62.210.205.24
https://www.ntppool.org/scores/78.46.102.180
@felixonmars Thatās a large load. Iām sure everyone is thankful for your support. Are we trying to keep the china pool consistent of servers that are located in china? That is why I purchased the Hong Kong VPS as that was my assumption. I guess if you can take that kind of load, although then even a slight higher offset would be looked beyond in the apparent details of the situation.
Thanks for the help, perhaps ask or someone will add it.
Noah
https://logiplex.net
I guess we are not there yet. I have some small Chinese servers but not at all possible to add to the pool at its current state. I can add some if a server setting to 384Kbps would get only 1~2 Mbps.
I hope we can eventually bring the pool back to maybe 30~40 servers and add more to achieve thatā¦
Right. My VPS canāt handle the current state of the CN zone apparently at 384Kbit from what I am hearing. Iām just hanging low in Asia and HK taking about 3k requests a second/6kpps but I figure itās a start and if we can get more people to donate a low price like $25 for the year I paid for 500GiByte of bandwidth a month, then eventually maybe these HK servers can be merged to the CN zone. Larger servers are of course much more viable and helpful.
I guess weāll have to wait and see.
avij, I donāt need to prove anything to anybody. Do you really think I am driven by stealth marketing? I think I really should be out of here then.
I have been involved in .cn zone bootstrap since its initial effort in 2016 and it became disheartening to see how after initial pileups servers kept falling off and leaving the zone.
Yesterdayās 22 servers have been reduced to 4 this morning. This is understandable, because nothing has fundamentally changed since 2016.
Whom could I ask to remove 85.199.214.100 and 85.199.214.101 from .cn? They are still being rotated in.
Edited: OK, all my servers are out of cn. I hope it will work out for everyone else!
Thanks
Leo
I want to add my two cents here.
- The āNet Speedā setting is misleading.
I took me quite some time digging in the forums to understand it. But any newcomers, especially those with few time (in big companies, etc) or language barrier, will just set it to their serversā wire speed and afterwards either struggle with the low score (and blame the monitoring system) or broken server networking (due to too much traffic). This will eventually make them decide to remove the server and never come back again.
I would suggest to use a different name here. Like a āWeightā, āRelative Bandwidthā or whatever that makes more sense. I am not a native speaker, so you may find a better one than what I have in mind. Or at least the help text in the right should be changed to explicitly remind them thatās NOT the wire speed.
- ntpd is broken on high load.
This is actually hard to figure out and I struggled for quite some time. Please see the following chart:
Before the red bar my server was having a broken firewall setup. And before the green bar, it is a single ntpd instance running with iptables notrack. The ntpd instance drops so many packets that the monitoring ones are dropped from time to time. For details please refer to my other thread: Dropped packets
Between the green and yellow bars it is running with rsntp, which unfortunately resulted in blank lines in the csv log and doomed the score. After the yellow bar it is running with chrony and never had a problem again.
Iām curious if the other 1Gbps server admins are aware of the issue at all, since this can easily be treated as a monitoring system problem as well (because the networking is fine, a few queries to the server will work mostly, and traceroutes to the monitoring station are okay in both direction). I will suggest to try chrony if your server is in the same dilemma.
If later this turns out to be a common problem, I would suggest changes on the software recommendation pages.
- Tolerant of broken net speed setting
I believe @LeoBodnar and everyone else in the thread are willing to help with the issues here, and most of us now have a good understanding of whatās going on in the China pool. However, there are many server admins who are either unaware of the forum, donāt have time to, or just didnāt understand. For a volunteer project I think it is important to make the system more tolerant with the fact that many servers are setting a wrong net speed. This is still the case even if the ānet speedā name was changed.
For these broken settings, would it be a good idea if the admins go ahead and just lower the net speed setting for them? The issues can easily be identified if the server keeps going down right after getting a score of 10. I know some server admins may want to keep a lower load so increasing the limit would be very inappropriate, but I donāt see a problem the opposite way. If this turns out to be effective, we can even automate it.
In case itās a human resource problem, I can offer some help here.
Sure is, and I agree.
Had no idea but I donāt doubt it at all. Itās 2019 and ntpd is not even SMP aware afaik.
I canāt think of a way it would be possible to predict and use precise bandwidth measurements with the architecture of the Internet.
Noah
https://logiplex.net
The problem here is that depending on the number of servers in the pool, the number of requests that you get will vary wildly. It would need to be raised again when/if the situation gets better. Not offering a solution here, just a remark.
I see. Something like an admin-only (or adjust automatically by the monitoring system) degrade level might help here, but that certainly involves some developmentā¦
Or maybe the āsimpleā way: An email sent to the server admin with something like āOur monitoring system indicates that your server is overloaded for quite some time (maybe a fixed number, like 7 days). We have thus lowered its net speed setting to improve the service quality. Please try to increase it again when appropriate.ā
Well, that is one way of reducing traffic to your servers. Letās see how this move affects your score and your concerns regarding the monitoring server.
When you create scenarios for pleasing the monitor consider the following facts.
- ntppool is not in control of total combined number of user requests R coming into specific zone following the DNS lookups
- ntppool is not in control of total combined capacity C of servers in the zone
- if for a specific zone R > C the packets will be lost. This includes both: user requests and monitoring packets
- If for some zone condition 3 above is true, the most efficient allocation of user request density must be in proportion to serverās share of total bandwidth. Efficient here means that total number of dropped requests is minimal (but not zero.)
- In such case all servers will have to drop packets, including monitoring ones.
To make allocation work for a zone for which condition 3 above is true:
- Server bandwidth has to be accurately measured or declared. Any fiddling with capacity setting will increase total number of dropped packets.
- There must be no penalty for a server dropping incoming packets - monitoring or user ones
- Monitor should not remove a server from the pool unless it has failed catastrophically. Server removal will cause all of its allocated packets to be lost - there is no spare capacity to reallocate them to others
The only way of making this work for a zone with R > C is to fix the current monitoring system. Even just turning it off for such a zone is a massive step forward.
Leo
6 Likes
Iād have a few ideas as well, but maybe we should try to reach a consensus about a few requirements before diving into implementation details.
The first one is: Are we allowed to drop requests (by directing it to a server which canāt handle the traffic, perhaps in proportion of their bandwidth setting) if we canāt find a suitable server for responding to those, or should we try harder to find a NTP server for such requests, even if it means we would need to query a server that is far away from the client? Iād lean towards the latter myself.
Since the pool only have tens of servers in China, I think we should ping those server more frequent than we do. Maybe once per miniute? So that we can better profile that server.
And we definatly need a monitor server in China asap. I just check some universityās servers, some of them only response to requests from within China (like s1d.time.edu.cn 4/4 success with ntpdate from Shenzhen, 0/4 from San Jose,CA) . And some response to all request but their return packets were having troubles crossing the wall(eg. s2f.time.edu.cn https://www.pool.ntp.org/scores/202.112.29.82). I think this is a common issue.
My server https://www.pool.ntp.org/scores/120.24.166.46 was not overload for quite some time, but still not able to go above 10. It is hosted by aliyun, suppose to have very nice connection, but the wall is dropping my responses.
one more server for the china zone:
https://www.ntppool.org/scores/116.203.67.76
https://www.ntppool.org/scores/2a01:4f8:1c1c:8038::1
@felixonmars
Yes this is a human resource Problem, like with almost any open Source Project. If you want to contribute, feel free to leave a pull request: https://github.com/abh/ntppool
The Project of adding multiple Monitoring Nodes is an ongoing Project, the beta site had this feature a couple of months, but apparently some more work is needed.
Hi,
two more server for the china zone (fresh installed).
IPv4 & IPv6
Located in europe Paris
https://www.ntppool.org/scores/51.15.241.78
https://www.ntppool.org/scores/2001:bc8:4400:2c00::20:91f
edit: i have removed the server which was in amsterdam. The score was too low all the time.
Would it be against moral to take all china subzones like hk and merge them to cn due to emergency and maybe start a new thread about anyone with big steel anywhere that wants to help. There is so much fiber these days even my $25 a year hk vps I purchased last night which is openvz of all things chimed in at .007 offset in the USA. Maybe this is an option at least temporarily.
Noah
https://logiplex.net
Rofl had to post this irony. I did an ntp server test to test offsets and that $25 a year HK openvz vps showed up in the millionths of a second. Mostly irony Iām sure. An offset of 0.000003