I am trying to configure AIX 6.1 using syslogd to send syslog event to syslog server configured on RHEL. However, RHEL never receives the events.
I have tried to redirect the syslog event on AIX to a local file and successful. Only forwarding to remote server fails.
Firewall between these servers are opened and allowed connection through.
I have tried to check the /var/log/messages but could not find any useful information.
Can anyone give me an advice for this situation?
The syslog message transfer is UDP so doesn't shake hands and open up a TCP connection with packets being acknowledged etc., it's just a fire-and-forget style transmission, so it can make it a bit tricky to diagnose. I have a few to questions pose in response first:-
What have you tried so far? Any variations?
Is the receiving side listening on the correct port? Default is UDP 514.
What output/errors do you get (if any) in your local syslog files? They must exist before the daemon starts to let it write to them. I think AIX refuses to create a new file.
Can you show us the lines you think are relevant from /etc/syslogd.conf
Are there firewalls in play here, either on the sending/receiving servers or something on the network (e.g. a DMZ)
I have tried to configure remote logging, local logging. Local logging success but remote failed.
Yes, the logger server is listening on port 514 as default. and we are receiving data from other UNIX Server but only AIX is having issue.
I do not see any error while starting syslog file. Furthermore, creating local syslog file is success.
For the firewall, I am confirming with the firewall team but they say that it has been opened for port 514 to go through. I need to double check again but it seems firewall team is doing good job and the port should be opened as they said.
And for /etc/syslog.conf content, it is very simple as below:
local0.debug @<Syslog server>
--> This one failed.
I have test with local logger as below content:
local0.debug /tmp/syslog.log
--> It works.
Just one question: I could not find the syslog daemon log file on AIX server. What is the default location of this log file? Is it /var/log/messages? I think if AIX does not send the syslog event then it should be syslogd issue but could not locate the log files to troubleshoot it.
Feel free to let me know any suggestion if you have I really appreciate it.
hmm, you (or, rather, the firewall boys) do understand there are two rules needed, yes? One for the connection in one direction itself, the second for the acknowledgements back to the client. Maybe this point is moot but i have seen many things in my life and this wouldn't be the strangest.
Have you made sure the name of the target system is a FQDN? Does the AIX system have proper name resolution for the target? You might, for testing purposes, put the IP address directly there and/or put the host name of the logging server into /etc/hosts .
Also notice that syslog needs reverse lookup too, which is often "forgotten" by novice network admins. If you check the name resolution always try the nslookup not only for the hostname but also for the corresponding IP address. Both have to be successful.
How is syslog started? Do a
lssrc -ls syslogd
and post the output. You can use the chssys command to change the way syslogd is started by the system resource controller. See the man page for syslogd for specifics, especially the -d (debug) switch. Also notice that after every change to /etc/syslog.conf you need to restart the subsystem with
thanks bakunin for very detail I am a newbie in AIX and really appreciate for all help.
Getting back to my query and bakunin's suggestion, I have tried to stop and start the syslogd service after making change of syslog.conf file (using stopsrc -s syslogd and then startsrc -s syslogd. I am not quite sure about refresh -s syslogd as I have seen some posts about this command sometimes may get naughty and do not refresh the conf file at all )
i will double check the firewall configuration and the syslogd service status tomorrow and get back to you the result after that.
Thank you for your help in advance.
regards,
Michael
You might want to read about the "System Resource Controller" or SRC, for short. It is sort of a super-daemon to start other daemons (called "subservers" or "subsystems", without wanting to go into details). Here is a (very short and incomplete)
Introduction to System Resource Controller (SRC) Commands
How these subsystems are started is defined with the command mkssys and stored in a "registry"-like data storage called ODM. You can change an already defined subsystem with the command chssys and remove a subsystem with the command rmssys . Display a subsystems properties with the command lssrc -ls <subsystem> .
Starting such a subsystem is done with startsrc , stopping with stopsrc and displayed what is running with lssrc -a . refresh reinitialises an already started subsystem if it supports signal communication (basically this sends a kill -1 ), but some subsystems won't allow that, which is why you have to do a stopsrc and then a startsrc in such a case.
You might also get away with kill -HUP process-id to get syslogd to re-read it's config file rather than a whole stop-start effort. Check the man page for syslog.conf to see if that is allowed (or just syslogd if there isn't a separate man page for the config file)
happy lunar new year to all of you i am just getting back from vacation.
I have checked with firewall team and it is getting open as expected. And syslog events still disappear on Syslog server.
I have tried to use tcpdump on AIX system and see that there is no packet sent from this AIX to Syslog server. the command i have used is: tcpdump -vv dst host <Syslog server>
There is one point that I am quite confuse: this server is operation behind a NAT. Does this affect to syslog operation?
The second thought is that the config file is getting wrong. Once i issued the command startsrc -s syslog, it say that the subsystem has started completed successfully along with syslogd PID.
however, i have read some posts and it say: The syslog.conf file MUST be configured with TAB, not space as space will be considered as a comment in syslog.conf. Is that true?
I have put space at the beginning and changed to TAB today. is there any way if i can check there is any space still left in the config file?
Really appreciate for your help.
Regards,
Michael.
There is one point that I am quite confuse: this server is operation behind a NAT. Does this affect to syslog operation?
[/quote]
Not the syslog directly, but the network connecction as a whole. Can you (for instance) do ftp , telnet , ssh or any other protocol to the destination host. Can you ping it?
Notice that - as there is a firewall in place - the firewall rules might have to be modified to allow that. My suggestion is to allow ssh -connection for testing purposes and, once this (and everything else) works, they can be removed again.
If ou use the best editor there is (i am talking about vi , of course) you can enter the command:
:set list
To display tab characters as ^I , so you will see the difference. Use
:set nolist
to switch that off again. Enter ":" in command mode (this is what you are in when you start it) to get to the colon-prompt.
just me being silly, but I hope you are using @some.sys.log.server and not (literally) @<some.sys.log.server> as I doubt the < and > are understood correctly.
Further, you asked about default locations.
AIX only "recently" started with any logs on by default putting the aso logs in /var/log/aso/ ..."
My default is the directory /var/log/syslog (with /var/log as it's own filesystem).
And I also recommend using tcpdump to check that it is leaving the host, and you can see what ip addresses and port numbers are used.
Further, you might try sending messages to AIX syslogd as a different test of the internal firewall.
I really appreciate it.
)
i am just getting back from vacation.