UNIX - AIX - Syslog messages

Dear Community,

I have just registered to the forum, and I am very glad I have joined.

I have a question about the syslog logging facility.

I am doing a security assessment for some servers that run on AIX. The admins have set up a syslog server and the following parameters are enabled in syslog.conf:

local0.crit /dev/console
local0.info /var/hacmp/adm/cluster.log
user.notice /var/hacmp/adm/cluster.log
daemon.notice /var/hacmp/adm/cluster.log

How can I know in detail (everything that is logged, not just a general description) which actions are being logged, by the syslog service on the syslog server?

Thank you!

AIX has at least 3 logging facilities and 1 auditing facility.

Logging facilities are:

  • syslog
  • errlog
  • alog

All 3 facilities make a little different things. Through alog you can see e.g. boot-time or console messages. Errlog logs mostly hardware-related messages. Syslog is a standard UNIX logging mechanism.

If you do a security assessment, the usual question is - what must be logged in your branch and in your country? There are a lot of different regulations for different branches (financial, healthcare, stock exchanges, human resources, and so on) and in different countries - rules for an US-based company are not the same as rules for an European- or a Chinese company.

All logging and auditing facilities must be configured according to your local requirements. It is not where a sysadmin has right to say something. It is your local authorities who say the word.

After you have your requirements it is easier to decide what should be written in a configuration file to receive the information you require on the central logging server. If you can't analyze the received information, because you've never seen UNIX logs, you have to hire some experienced guy or can try to post example log entries (if they are not tooo sensitive for your organization) here and forum members will try to explain the logs to you.

Although I don't want to just say "Read the manuals", the manual page has a good description of all the configuration options.

There isn't an AIX one on this site, but you can read this:- Solaris syslog.conf manual

The IBM one (depending on version) can be found here:- AIX 6 syslog.conf manual

It really depends what you are trying to achieve and at what detail. You can get too much and struggle to cope (disk space, too much to filter etc.) so can you define what you need and we can work on the best options from there.

Robin

Hey, guys thank you so much for you replies, but the fact is that I have read extensively all the official manuals. Although the types and priorities of messages are defined in high-level areas, such as user-related, or for audit purposes, the problem is that I could not find a comprehensive list of the exact actions that are being logged. e.g. what does the user.notice logs in detail.

I hope that makes my question more specific.

The entries you have posted are default entries, that come with the installation of IBM HACMP or now called PowerHA, the high availability software from IBM - check this: IBM Info Center.

There is no exact complete list, that defines which application writes into those facilities as far as I know. Though it is common sense, that a mail server will most likely write to it's own logs or write to the syslog daemon and sowith addressing the config lines for the mail facility. Best may be to filter the log and see what is written there, usually there will some kind of source, like [kernel] in Linux or a daemon that writes there. But if nothing happens and it is not written there, I doubt you will easily find out what could write there. Some software writes there, other has it's own logs etc.

Maybe IBM developers know, which parts of their software will write to which facility. Or maybe some admin here can give a hint which things are written there just by experience, but I doubt it will be an exact full list.
You can see those facilities like mail, user, daemon, ... as kind of buckets or filters which any application can "address" in the way that it flags it's message with the appropriate facility and criticality when it writes to the system's syslog daemon. The severities are just an additional filter to sort, where which type of messages should go.

There is no such comprehensive list. Every application can call the functions openlog() and syslog(), specifying facility and priority for syslog messages they like. You can also do the same thing from the command line:

$ logger -p user.notice It was just an example
$ logger -p kern.crit Was it kernel critical message or not?

It might pay to understand how the syslog works:

On one side you have applications (this is rather losely defined, system tools might be among them), which use system calls to issue messages. Messages are grouped by "facilities" (local, user, audit, ....) and "severity" (info, warn, crit, ...). Each message belongs to exactly one facility and one severity.

Syslog is a daemon - syslogd - which collects all these messages. The syslog.conf file describes what syslogd should do with them: write them to a (certain) file, ignore them or send them to a remote destination. Notice, that the severity levels are ordered. Every rule for a certain level also affects all higher severities. Let us analyze your own example:

local0.crit /dev/console
local0.info /var/hacmp/adm/cluster.log

This means: all messages from the "local0" facility with severity "info" or higher ("info" the second-lowest severity, "debug" the lowest) go to the file /var/hacmp/adm/cluster.log . The messages from the "local0" facility with severity "crit" and higher also go the the system console and you will see them when you open the console from the HMC.

What exactly causes a certain message to be issued is a decision of the respective application: it issues a message and says this should go to facility "this" and with severity "that". Syslog then only collects this message and consults its rulebook syslog.conf about what to do with it.

I hope this helps.

bakunin

2 Likes

Thank you everybody for your time and effort to help me. I believe that I now get it.

Mostly thank you bakunin, for the clarification of how syslog operates. That was also my estimation of how this daemon works, but after your comment I get it.

As a side-note, I think that the best way to continue with my assessment, as far as syslog is concerned, is to check for some critical services (e.g. login) in which message category they store what kind of messages.

If you want to know what applications are posting some 'wildcard' entries can help with that.
Below I have commented out all applications at 'debug' level - which is everything. Instead I collect everything at info level - except for local4 (which is used by ipsec logging) - I leave that one at debug level.

# *.debug;local4.none   /var/log/syslog/blockip.log 
*.info;local4.none      /var/log/syslog/blockip.log 
local4.debug    /var/log/syslog/local4.log 

Looking at the combined log you can see which facility is concerned, as well as the severity of the message. From this you can decide how to accommodate different facilities in separate files and/or with differing severity.

michael@x071:[/var/log/syslog]tail blockip.log
Jul  8 09:00:00 x071 mail:warn|warning sendmail[8913086]: gethostbyaddr(IPv6:::1) failed: 1
Jul  8 09:00:00 x071 mail:info sendmail[8913086]: t689004j8913086: from=daemon, size=1131, class=0, nrcpts=1, msgid=<201507080900.t689004j8913086@aixtools.com>, relay=root@localhost
Jul  8 09:00:00 x071 mail:info sendmail[9371720]: t689004j8913086: to=michael, ctladdr=daemon (1/1), delay=00:00:00, xdelay=00:00:00, mailer=local, pri=121131, dsn=2.0.0, stat=Sent
Jul  8 09:00:10 x071 user:info syslog: /etc/ifconfig -a
Jul  8 09:00:28 x071 auth|security:info sshd[5964030]: Accepted publickey for michael from 192.168.129.5 port 64940 ssh2: DSA SHA256:eOZ7yejpirXRJDLw99kR2AzZzuj/o/KNJGd9r70EHds
Jul  8 09:04:57 x071 mail:info pop3d[9109536]: Moved 1407 bytes of new mail to /home/michael/mbox from /var/spool/mail/michael host= x005.home.local [192.168.129.5]
Jul  8 09:04:57 x071 mail:info pop3d[9109536]: Auth user=michael host=x005.home.local [192.168.129.5] nmsgs=1/1
Jul  8 09:04:57 x071 mail:info pop3d[9109536]: Logout user=michael host=x005.home.local [192.168.129.5] nmsgs=0 ndele=1
Jul  8 09:09:33 x071 auth|security:crit su: BAD SU from michael to root at /dev/pts/1
Jul  8 09:09:38 x071 auth|security:notice su: from michael to root at /dev/pts/1

Finally, an easy way to know how syslog is currently running is the command

michael@x071:[/var/log/syslog]lssrc -s syslogd -l

With, e.g., the following result

Subsystem         Group            PID          Status 
 syslogd          ras              3407976      active
Syslogd Config   *.info;local4.none     /var/log/syslog/blockip.log  
Syslogd Config   local4.debug   /var/log/syslog/local4.log  
Syslogd Config   local1.debug   /var/log/syslog/local1.log  
Syslogd Config   user.warn      /var/log/httpd/rootvg/user.log 
Syslogd Config   mail.info      /var/log/syslog/mail.log rotate size 250k files 4 
Syslogd Config   aso.notice /var/log/aso/aso.log rotate size 1m files 8 compress 
Syslogd Config   aso.info /var/log/aso/aso_process.log rotate size 1m files 8 com 
Syslogd Config   aso.debug /var/log/aso/aso_debug.log rotate size 32m files 8 com