NTP high offset | no servers can be used, exiting

sudo /usr/sbin/ntpdate -q nxdbsr01.paramount.com

server 192.168.148.165, stratum 2, offset 45.247135, delay 0.09427
29 Jul 17:49:48 ntpdate[11404]: step time server 192.168.148.165 offset 45.247135 sec

sudo /usr/sbin/ntpdate -d nxdbsr01.paramount.com

29 Jul 17:49:54 ntpdate[11405]: ntpdate 4.2.0a@1.1190-r Thu May 13 04:29:07 EDT 2010 (1)
Looking for host nxdbsr01.paramount.com and service ntp
host found : nxdbsr01.paramount.com
transmit(192.168.148.165)
receive(192.168.148.165)
transmit(192.168.148.165)
receive(192.168.148.165)
transmit(192.168.148.165)
receive(192.168.148.165)
transmit(192.168.148.165)
receive(192.168.148.165)
transmit(192.168.148.165)
server 192.168.148.165, port 123
stratum 2, precision -20, leap 00, trust 000
refid [192.168.148.165], delay 0.09438, dispersion 0.00768
transmitted 4, in filter 4
reference time:    d5a18991.35bf5946  Mon, Jul 29 2013 17:38:41.209
originate timestamp: d5a18c60.e4c42635  Mon, Jul 29 2013 17:50:40.893
transmit timestamp:  d5a18c32.e169c23b  Mon, Jul 29 2013 17:49:54.880
filter delay:  0.09541  0.09489  0.09438  0.09535
         0.00000  0.00000  0.00000  0.00000
filter offset: 45.95513 45.96297 45.97070 45.97821
         0.000000 0.000000 0.000000 0.000000
delay 0.09438, dispersion 0.00768
offset 45.970707

29 Jul 17:49:54 ntpdate[11405]: step time server 192.168.148.165 offset 45.970707 sec

sudo /usr/sbin/ntpdate -U nxdbsr01.paramount.com

29 Jul 17:50:01 ntpdate[11409]: no servers can be used, exiting

sudo /usr/sbin/ntpq -pn

   remote           refid      st t when poll reach   delay   offset  jitter
==============================================================================
 192.168.148.165 128.138.141.172  2 u   31   64  377   69.683  33347.4 19119.3
 10.40.143.51    69.25.96.13      2 u   26   64  377    0.350  46917.9 29341.6
 10.40.143.52    69.25.96.13      2 u   28   64  377    0.342  14097.4 19138.5
 192.168.148.40  129.6.15.29      2 u   21   64  377   69.682  7680.91 23931.3
*127.127.1.0     LOCAL(0)        10 l   26   64  377    0.000    0.000   0.001

cat /etc/ntp.conf:

# restricted by default
restrict default kod nomodify notrap nopeer noquery
restrict -6 default kod nomodify notrap nopeer noquery

# permit access from loopback interface
restrict 127.0.0.1
restrict -6 ::1

# time servers
server nxdbsr01.paramount.com
server nxinfp02.paramount.com
server nxinfp03.paramount.com
server nxinfr04.paramount.com

# local clock
server 127.127.1.0
fudge 127.127.1.0 stratum 10

# drift file
driftfile /var/lib/ntp/drift

sudo /usr/bin/ntpstat && echo $?

sudo /usr/bin/ntpstat && echo $?
synchronised to local net at stratum 11
   time correct to within 75 ms
   polling server every 64 s
0

I am facing this frequently on many of my production servers in my enviroment ..

what could be the prob. ? how could I get this resolved ?

Sometimes an answer is turning off the local clock (127.127.1.0). You might also want to look at the "tinker panic" option.

Honestly speaking, I never turned off local clock .. how to do that ? and what would be the impact of doing this ??

I am reading about tinker panic ..

The 4 peer servers have too different times. Please get them fixed!
I think the offset columns in ntpq command differ too much for stratum 2 peers, so it falls back to the stratum 10 local clock.
Deleting the local clock 127.127.0.1 in ntp.conf would be a bad work-around.

I'll look into it today when I reach office .. and I am not much aware what stratum is, will read about it too ..

any suggestions to resolve this ? OR any links to read? Please dont point me to that article by RHN, that is not helping me ..

Stratum is the "distance" (number of hops) from the reference clock.
The lower the stratum, the closer it is to the reference clock.

I think that you would be far better getting the time sources that you will be relying on into step first, else you will never know where you are.

Imagine that you get your time from NTPserver1 and that has no connection to NTPserver2 and they drift apart. Now you will need to consider which is the correct one and which clients refer to which. Worse still, if your NTPserver1 drops and you automatically detect and use the time from NTPserver2, you could see a dramatic and unplanned time step, perhaps even moving the clock backwards by several minutes and that could be devastating to any applications of schedulers that rely on accurate time.

Basically, you need to:-

1. Get your time servers agreeing on what your company standard time is first and keeping in sync with each other (and perhaps an internet or radio source)
2. Use ntpdate to step your server into line with any of them - pick a quiet time to do this.
3. Start your NTP client (ntpd or xntpd usually) to keep in sync with them.

I hope that this helps.

Robin
Liverpool/Blackburn
UK