Requesting help to understand rsyslog logging server configuration

Hi,

I am learning to configure rsyslog server. while configuring i am facing some issues.
please help me with proper rsyslog.conf configuration, by which logs are logged only at one place at log server within a particular directory with different log names.With no logging at client itself.

Configuration at server:

[root@rhel7-test-3 ~]# cat /etc/rsyslog.conf
# rsyslog configuration file

# For more information see /usr/share/doc/rsyslog-*/rsyslog_conf.html
# If you experience problems, see http://www.rsyslog.com/doc/troubleshoot.html

#### MODULES ####

# The imjournal module bellow is now used as a message source instead of imuxsock.
$ModLoad imuxsock # provides support for local system logging (e.g. via logger command)
$ModLoad imjournal # provides access to the systemd journal
$ModLoad imklog # reads kernel messages (the same are read from journald)
$ModLoad immark  # provides --MARK-- message capability

# Provides UDP syslog reception
$ModLoad imudp
$UDPServerRun 514
#$InputUDPServerBindRuleset remote

# Provides TCP syslog reception
#$ModLoad imtcp
#$InputTCPServerRun 514

$template RemoteLogs, "/var/log/remoteclients/%HOSTNAME%/%PROGRAMNAME%.log"
*.* ?RemoteLogs

#### GLOBAL DIRECTIVES ####

# Where to place auxiliary files
$WorkDirectory /var/lib/rsyslog

# Use default timestamp format
$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat

# File syncing capability is disabled by default. This feature is usually not re                                 quired,
# not useful and an extreme performance hit
#$ActionFileEnableSync on

# Include all config files in /etc/rsyslog.d/
$IncludeConfig /etc/rsyslog.d/*.conf

# Turn off message reception via local log socket;
# local messages are retrieved through imjournal now.
$OmitLocalLogging on

# File to store the position in the journal
$IMJournalStateFile imjournal.state


#### RULES ####

# Log all kernel messages to the console.
# Logging much else clutters up the screen.
#kern.*                                                 /dev/console

# Log anything (except mail) of level info or higher.
# Don't log private authentication messages!
*.info;mail.none;authpriv.none;cron.none                /var/log/messages

# The authpriv file has restricted access.
authpriv.*                                              /var/log/secure

# Log all the mail messages in one place.
mail.*                                                  -/var/log/maillog


# Log cron stuff
cron.*                                                  /var/log/cron

# Everybody gets emergency messages
*.emerg                                                 :omusrmsg:*

# Save news errors of level crit and higher in a special file.
uucp,news.crit                                          /var/log/spooler

# Save boot messages also to boot.log
local7.*                                                /var/log/boot.log


# ### begin forwarding rule ###
# The statement between the begin ... end define a SINGLE forwarding
# rule. They belong together, do NOT split them. If you create multiple
# forwarding rules, duplicate the whole block!
# Remote Logging (we use TCP for reliable delivery)
#
# An on-disk queue is created for this action. If the remote host is
# down, messages are spooled to disk and sent when it is up again.
#$ActionQueueFileName fwdRule1 # unique name prefix for spool files
#$ActionQueueMaxDiskSpace 1g   # 1gb space limit (use as much as possible)
#$ActionQueueSaveOnShutdown on # save messages to disk on shutdown
#$ActionQueueType LinkedList   # run asynchronously
#$ActionResumeRetryCount -1    # infinite retries if host is down
# remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional
#*.* @@remote-host:514
# ### end of the forwarding rule ###

why syslog server is creating folder for itself within "remote-clients"

[root@rhel7-test-3 log]# ls -l remote-clients
total 0
drwx------. 2 root root 40 Jan 20 14:12 rhel7-client
drwx------. 2 root root 80 Jan 20 14:12 rhel7-test-3
[root@rhel7-test-3 remote-clients]# cd rhel7-client
[root@rhel7-test-3 rhel7-client]# ls -ltr
total 8
-rw-r--r--. 1 root root 242 Jan 20 14:12 systemd.log
-rw-r--r--. 1 root root 120 Jan 20 14:12 CROND.log

from client,

[root@rhel7-client ~]# logger -p mail.info "testing mail log"

then at log server,

root@rhel7-test-3 rhel7-client]# ls -ltr
total 20
-rw-r--r--. 1 root root  163 Jan 20 14:13 root.log
-rw-r--r--. 1 root root  322 Jan 20 14:13 goa.log
-rw-r--r--. 1 root root  682 Jan 20 14:15 postfix.log
-rw-r--r--. 1 root root 1867 Jan 20 14:19 systemd.log
-rw-r--r--. 1 root root  918 Jan 20 14:19 CROND.log

Till it seems ok, but

From Client,

[root@rhel7-client log]# logger "hello--test test-1"
[root@rhel7-client log]# tail messages
Jan 20 14:21:47 rhel7-client root: hello--test test-1

From Server,
logger message is logged at root.log

[root@rhel7-test-3 rhel7-client]# more root.log
2017-01-20T14:21:47+05:30 rhel7-client root: root [1444]: logger "hello--test test-1" [0]

logger message is logged also at /var/log/messages of rsyslog server.

[root@rhel7-test-3 rhel7-client]# tail /var/log/messages
Jan 20 14:21:01 rhel7-client systemd: Started Session 19 of user user1.
Jan 20 14:21:47 rhel7-client root: hello--test test-1

why it is logging thrice.

rsyslog.conf at client,

[root@rhel7-client /]# cat /etc/rsyslog.conf
# rsyslog configuration file

# For more information see /usr/share/doc/rsyslog-*/rsyslog_conf.html
# If you experience problems, see http://www.rsyslog.com/doc/troubleshoot.html

#### MODULES ####

# The imjournal module bellow is now used as a message source instead of imuxsock.
$ModLoad imuxsock # provides support for local system logging (e.g. via logger command)
$ModLoad imjournal # provides access to the systemd journal
$ModLoad imklog # reads kernel messages (the same are read from journald)
$ModLoad immark  # provides --MARK-- message capability

# Provides UDP syslog reception
$ModLoad imudp
$UDPServerRun 514

# Provides TCP syslog reception
$ModLoad imtcp
$InputTCPServerRun 514

#### GLOBAL DIRECTIVES ####

# Where to place auxiliary files
$WorkDirectory /var/lib/rsyslog

# Use default timestamp format
$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat

# File syncing capability is disabled by default. This feature is usually not required,
# not useful and an extreme performance hit
#$ActionFileEnableSync on

# Include all config files in /etc/rsyslog.d/
$IncludeConfig /etc/rsyslog.d/*.conf

# Turn off message reception via local log socket;
# local messages are retrieved through imjournal now.
$OmitLocalLogging on

# File to store the position in the journal
$IMJournalStateFile imjournal.state


#### RULES ####

# Log all kernel messages to the console.
# Logging much else clutters up the screen.
#kern.*                                                 /dev/console

# Log anything (except mail) of level info or higher.
# Don't log private authentication messages!
*.info;mail.none;authpriv.none;cron.none                /var/log/messages

# The authpriv file has restricted access.
authpriv.*                                              /var/log/secure

# Log all the mail messages in one place.
mail.*                                                  -/var/log/maillog


# Log cron stuff
cron.*                                                  /var/log/cron

# Everybody gets emergency messages
*.emerg                                                 :omusrmsg:*

# Save news errors of level crit and higher in a special file.
uucp,news.crit                                          /var/log/spooler

# Save boot messages also to boot.log
local7.*                                                /var/log/boot.log
user.info /var/log/user.log
ftp.info /var/log/ftp.log
local6.err      /var/log/errorandabove
local6.=err     /var/log/onlyerror
local6.*        /var/log/alllocal6
# ### begin forwarding rule ###
# The statement between the begin ... end define a SINGLE forwarding
# rule. They belong together, do NOT split them. If you create multiple
# forwarding rules, duplicate the whole block!
# Remote Logging (we use TCP for reliable delivery)
#
# An on-disk queue is created for this action. If the remote host is
# down, messages are spooled to disk and sent when it is up again.
$ActionQueueFileName fwdRule1 # unique name prefix for spool files
$ActionQueueMaxDiskSpace 1g   # 1gb space limit (use as much as possible)
$ActionQueueSaveOnShutdown on # save messages to disk on shutdown
$ActionQueueType LinkedList   # run asynchronously
$ActionResumeRetryCount -1    # infinite retries if host is down
# remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional
#*.* @@remote-host:514
*.* @192.168.234.146:514
# ### end of the forwarding rule ###

Please help to identify the issue with my configuration file.

Basically, on each individual server, the only active line you need is

*.*       @syslog.my.company.server

You do the filtering/splitting on the the receiving server. If you match the message multiple times, you will record it multiple times. Do you have an example of the duplicated messages? If they are in different files, then remember that you are usually recording messages that are of the specified level and above, e.g. *.info will also pick up *.warn messages. You can specifically exclude hight levels, if that's what you want.

This link is useful. Sending Messages to a Remote Syslog Server

I would suggest using a DNS address in case you want to move your central syslog server, however some people worry that this requires the address to resolve as the rsyslogd starts, so I know that many people use a plain IP address.

I hope that this helps,
Robin