why does ntpd never work, always have to use ntpdate in cron job

[Red Squirrel]

ntpd is suppose to keep clock in sync with server specified in /etc/ntp.conf right? Because it NEVER works for me. I always end up having to just disable the service and run ntpdate directly once a day in a cron job. This is kind of dirty.

Why doesn't ntpd not work? Every single machine I've ever setup, I've never actually seen that service work properly as a client. I just add my local server as primary in the config file, is there anything else I have to do? Here's an example config file, the only thing I changed is added "server borg.loc" a bit before the half way mark.

# For more information about this file, see the man pages
# ntp.conf(5), ntp_acc(5), ntp_auth(5), ntp_clock(5), ntp_misc(5), ntp_mon(5).

driftfile /var/lib/ntp/drift

# Permit time synchronization with our time source, but do not
# permit the source to query or modify the service on this system.
restrict default kod nomodify notrap nopeer noquery
restrict -6 default kod nomodify notrap nopeer noquery

# Permit all access over the loopback interface.  This could
# be tightened as well, but to do so would effect some of
# the administrative functions.
restrict 127.0.0.1
restrict -6 ::1

# Hosts on local network are less restricted.
#restrict 192.168.1.0 mask 255.255.255.0 nomodify notrap

# Use public servers from the pool.ntp.org project.
# Please consider joining the pool (http://www.pool.ntp.org/join.html).
server borg.loc
server 0.centos.pool.ntp.org iburst
server 1.centos.pool.ntp.org iburst
server 2.centos.pool.ntp.org iburst
server 3.centos.pool.ntp.org iburst

#broadcast 192.168.1.255 autokey        # broadcast server
#broadcastclient                        # broadcast client
#broadcast 224.0.1.1 autokey            # multicast server
#multicastclient 224.0.1.1              # multicast client
#manycastserver 239.255.254.254         # manycast server
#manycastclient 239.255.254.254 autokey # manycast client

# Enable public key cryptography.
#crypto

includefile /etc/ntp/crypto/pw

# Key file containing the keys and key identifiers used when operating
# with symmetric key cryptography. 
keys /etc/ntp/keys

# Specify the key identifiers which are trusted.
#trustedkey 4 8 42

# Specify the key identifier to use with the ntpdc utility.
#requestkey 8

# Specify the key identifier to use with the ntpq utility.
#controlkey 8

# Enable writing of statistics records.
#statistics clockstats cryptostats loopstats peerstats
~

borg.loc has ntpd running as a server and client and is considered the reliable time source. If I do "ntpdate borg.loc" it syncs.

I want to get at the bottom of this as I'm starting to think this may be the cause of my rsync issues. When I use rsync without --checksum, it just copies everything.

 

[Fallen Kell]

Your file "looks" alright. Are you certain you have ntpd service turned on (run a "service ntpd status")?

Once you know your service is running, can you list the output of the "ntpq -p" command? It should show you that you are peered with borg.loc as well as each of the centos pools, and it should tell you what stratum each of those systems are reporting to be according to your client connection (stratum is the value under "st" in the ntpq output). If their stratum is not lower than 10 (maybe 16?), I believe your system will not trust them as a valid timeserver.

You may want to add the following options to the ntp server you want to connect to:
iburst minpoll 4 maxpoll 8

Those should be overkill in terms of syncing time with it checking at least every 256 seconds (2^"<maxpoll value>" seconds).

I suspect your issue has to do with getting a valid peer, with "ntpq" being the command to show you what is going on to debug. The other possibility is that ntpd will not sync your time if your current clock is too far out of sync with the servers (I think too far may be in the order of a couple minutes but I don't remember off the top of my head). Best to turn off the ntpd process, run a "ntpdate", and then restart your ntpd service to prevent that issue.


Also, your "local server" being listed first has nothing to do with it being the "primary". The lowest stratum server will be the server trusted for the most accurate time.

 

[Red Squirrel]

Yep it's running, this is the output on one of the machines:

[root@appdev debug]# ntpq -p
     remote           refid      st t when poll reach   delay   offset  jitter
==============================================================================
+borg.loc        129.6.15.27      2 u  233 1024  377    0.217   -2.407   1.158
*69.28.67.44     209.51.161.238   2 u  806 1024  377   12.808   -1.929   1.559
+S0106c04a00f34a 128.233.150.93   2 u  838 1024  377   79.873   -2.681   2.437
-ks4001083.ip-19 140.203.204.77   2 u  609 1024  367   19.271   -2.509   2.237
-alpha.codatory. 164.244.221.197  2 u  292 1024  377   17.576   -1.136   1.373
[root@appdev debug]#

Edit: Just noticed the IP address for borg.loc is completely wrong. No idea where it's getting that. It's a local server and the IP is 10.1.1.10. That may be my problem. Why is it not getting the right IP?

Over time it will still drift by a few seconds though. This is on all machines, except for the ones where I have ntpdate setup in a cron job, but since that's kind of dirty I want to try to eliminate needing that.

I'll try that setting you told me and see how that goes.

 

[dave_the_nerd]

Are these VMs? Are the hosts clocks all synced up right?

I also ended up using ntpdate in cron for most of our systems, but the drift got a LOT better when I doublechecked the hypervisors and fixed a couple of them.

 

[Fallen Kell]

You might want to also check that your drift file actually exists. This issue goes back a couple years and may have been fixed in more recent versions of ntp, but at one point in time, you needed to manually touch the driftfile first to create it, as the daemon would not do so.

You might also want to enable some more detailed logging data by adding the following 2 lines to the ntp.conf:

logconfig=all
logfile /var/log/ntpd.log

 

[Red Squirrel]

Some of the servers are VMs, some arn't. I've never actually seen this work either way. The actual NTP server is physical. Though it's a really old OS, it's FC9. Been wanting to retire that box but did not get a chance yet, need to create a new VM and reconfigure everything that server does. Could the fact that it's running a really old version of NTP be an issue? Does the drift file have to be created on the server or the clients, and where?

 

[TheRyuu]

This is a cheat answer and does not answer the original question but I would recommend OpenNTPD[1] over ntpd (your distro should have an official package). It's implementation is simpler and should work just fine if all you need to do is sync from some time server.

[1] https://wiki.archlinux.org/index.php/OpenNTPD

 

[Fallen Kell]

Does the drift file have to be created on the server or the clients, and where?

The driftfile should exist on all clients (even the server if the server is a client of another ntp server, which it should be unless you are on a stand-alone network without connection to the internet). The driftfile's location is specified in your ntp.conf file, in your case:

driftfile /var/lib/ntp/drift

 

[Red Squirrel]

Seems the drift files exist. I'll wait and see if the setting changes I made fixed it. I only did a few servers for now to see.

Also should I be using a GPS antenna or anything for better accuracy? The idea behind this is I want to be able to use rsync without --checksum, but right now my time sync between my servers is not accurate enough and it just keeps copying everything because the time stamps arn't exact between the source and destination even if the files did not change.

 

[manly]

My co-worker explained to me that ntpd skews (aka drift) your host time to synchronize it. By design, this is to prevent any big "jumps" in time that could affect running applications in unexpected ways. So if your host is minutes off of the NTP server, ntpd will not just reset your time the way ntpdate does. It's partly a question of trust, if the delta is large-ish, how does the host know which server has the correct time?

In theory, if you use ntpdate once, ntpd should be doing its job thereafter...

 

[tomt4535]

Yep it's running, this is the output on one of the machines:

[root@appdev debug]# ntpq -p
     remote           refid      st t when poll reach   delay   offset  jitter
==============================================================================
+borg.loc        129.6.15.27      2 u  233 1024  377    0.217   -2.407   1.158
*69.28.67.44     209.51.161.238   2 u  806 1024  377   12.808   -1.929   1.559
+S0106c04a00f34a 128.233.150.93   2 u  838 1024  377   79.873   -2.681   2.437
-ks4001083.ip-19 140.203.204.77   2 u  609 1024  367   19.271   -2.509   2.237
-alpha.codatory. 164.244.221.197  2 u  292 1024  377   17.576   -1.136   1.373
[root@appdev debug]#

Edit: Just noticed the IP address for borg.loc is completely wrong. No idea where it's getting that. It's a local server and the IP is 10.1.1.10. That may be my problem. Why is it not getting the right IP?

Over time it will still drift by a few seconds though. This is on all machines, except for the ones where I have ntpdate setup in a cron job, but since that's kind of dirty I want to try to eliminate needing that.

I'll try that setting you told me and see how that goes.

The refid field is the reference that the NTP server on that line is getting its time from, not the IP address of the "borg.loc" server.