Is it possible to create an ECMP cluster in software on Linux or are hardware required?
@iocc: the trick to ECMP is that you’ve got control of the router delivering the packets to your node(s). So if you’ve got a Linux box acting as the router, with e.g. four nodes connected to it directly (at layer 2) then you can do the following.
In this example, 192.0.2.0/24 is the network to which all the nodes and router are connected. And 198.51.100.100 is the IP address I’m going to publish in the NTP pool:
# node 1
ip addr add 192.0.2.1/24 dev eth0
ip addr add 198.51.100.100/32 dev eth0
# node 2
ip addr add 192.0.2.2/24 dev eth0
ip addr add 198.51.100.100/32 dev eth0
…etc…
# router
ip route add 198.51.100.100/32 nexthop via 192.0.2.1 nexthop via 192.0.2.2 nexthop via 192.0.2.3 nexthop via 192.0.2.4
Technically you don’t need all four nodes on the same network segment — they do need to be connected directly to the router, though, in this example. But you do have to be sure that your router box is performant enough to route the entire NTP traffic (which can be spread across cores by Linux, unlike ntpd serving NTP).
Another setup might be to use a routing protocol like BGP. In that instance, it’s more like “anycast” — except you need to be careful to ensure that the ISP routers will see an equal cost and will install multiple paths (ECMP) to all the node instances.
Nice trick, I installed 8 VMs and got it to work quite nicely.
Waiting for http://www.pool.ntp.org/scores/193.228.143.26 to be inserted in the pool.
I have 8 ethernet ports configured in a LACP so I hope it will be able to handle all the traffic.
1 Like
What is the LACP configured between? Router and VM host? How are the devices at each end of the LACP group splitting the traffic across the links? Using a hash of L2 and/or L3 (src+dst IP+port)? If you’ve got 8x 1G links, you shouldn’t be having any problems… but I notice that the score of 193.228.143.26 has had some drops. Are all eight VMs only being used for this ECMP/anycast node, or are the individual node IP addresses also being used by the pool (in which case, individual nodes might still get overloaded, and cause problems for your ECMP node’s overall score)?
Between my co-los router and the VM host.
Im using layer3+4 as transmit hash policy to get the traffic
shared equal over the ports. And yes they are gigabit.
It worked but only for my own network that will go through the routing table.
For the most part just one of eight got the traffic. I’ll ask my co-lo to
put the routes at their router instead.
To be continued…
Whilst I’ll caveat the next statement before I even make it, generally NTP is not designed to be clustered in this way and it would usually make more sense to add machines as individual servers to the pool, rather than trying to cluster them together behind a single IP address.
Of course, with this specific case, quantity is what’s needed and the benefits outweigh the potential problems. When you are heroically propping up a zone by dragging 100’s of mbits of traffic halfway round the world the impact of using a cluster become less significant 
However, for anyone thinking of doing this on a smaller scale, read up and consider the issues first. e.g. https://serverfault.com/questions/805939/ntp-high-availability-behind-a-virtual-ip
1 Like
In the end I choosed to run rsntp instead of a ECMP cluster, better use of
resources. And its closer to 200 Mbit/s that has been dragging halfway
around the world 
See my post about it here:
1 Like