Rsyslog setup issue

UNIX for Advanced & Expert Users
1

Hi,
I have one CentOS server, with rsyslog setup on it. Splunk forwarder is the app running on it. As per below sh script, it is supposed to send two logs to app log with date suffix, for example - /export/home/splunk/logs/sendSyslogActionAlert_20230920.log

[root@linux121213 ~]# cat /export/home/splunk/splunk/etc/apps/app_customalertactions/bin/sendSyslogAlarm.sh
#!/bin/bash

export logging_agent="/usr/bin/logger"

if [ $# -ne 8 ];
then
        echo "Usage $(basename "${0}") SearchKey Message Product Customer System Application Carrier ImpairmentDescription"
        echo "Passed parameters are :: $*"
        exit 1
fi

export searchKey=${1}
export message=${2}
export product=${3}
export customer=${4}
export system=${5}
export application=${6}
export carrier=${7}
export impairmentDescription=${8}

mkdir -p "${HOME}"/logs
echo "$(date): command:: $logging_agent SearchKey=\"${searchKey}\" Message=\"${message}\" Product=\"${product}\" Customer=\"${customer}\" System=\"${system}\" Application=\"${application}\" Carrier=\"${carrier}\" ImpairmentNo=\"${impairmentNo}\" ImpairmentDescription=\"${impairmentDescription}\"" >> "${HOME}"/logs/sendSyslogActionAlert_"$(date +%Y%m%d)".log
$logging_agent  "SearchKey=${searchKey} Message=${message} Product=${product} Customer=${customer} System=${system} Application=${application} Carrier=${carrier} ImpairmentDescription=${impairmentDescription}"
echo "Complete $(date): command:: $logging_agent SearchKey=\"${searchKey}\" Message=\"${message}\" Product=\"${product}\" Customer=\"${customer}\" System=\"${system}\" Application=\"${application}\" Carrier=\"${carrier}\" ImpairmentNo=\"${impairmentNo}\" ImpairmentDescription=\"${impairmentDescription}\"" >> "${HOME}"/logs/sendSyslogActionAlert_"$(date +%Y%m%d)".log
[root@linux121213 ~]#

I can see only one line reaches to /var/log/messages. That is fine with me. But I want to understand, where is it defined that it should go to /var/log/messages, along with /export/home/splunk/logs/sendSyslogActionAlert_20230920.log

[root@linux121213 ~]# cat /export/home/splunk/logs/sendSyslogActionAlert_20230920.log | grep -i IMPAIRIMENT | tail -2
Wed Sep 20 10:42:01 UTC 2023: command:: /usr/bin/logger SearchKey="2023-09-20-T10_2_NGIA" Message="Alarm created for (CallSetup) in the last (5) minutes, (2) Impairments were detected, from the total of (3) Attempts. The IMP% is (0.667) which is greater or equal to the Thershold% (0.1), for Customer (NGIA), Product (NGCS) by (Splunk) System. Please use this Alarm-ID (2023-09-20-T10_2_NGIA) for investigation" Product="NGCS" Customer="NGIA" System="Splunk" Application="IMPAIRIMENT DETECTION" Carrier="CTLXXDESM01" ImpairmentNo="" ImpairmentDescription="Impairment Detected During Call Setup"
Complete Wed Sep 20 10:42:01 UTC 2023: command:: /usr/bin/logger SearchKey="2023-09-20-T10_2_NGIA" Message="Alarm created for (CallSetup) in the last (5) minutes, (2) Impairments were detected, from the total of (3) Attempts. The IMP% is (0.667) which is greater or equal to the Thershold% (0.1), for Customer (NGIA), Product (NGCS) by (Splunk) System. Please use this Alarm-ID (2023-09-20-T10_2_NGIA) for investigation" Product="NGCS" Customer="NGIA" System="Splunk" Application="IMPAIRIMENT DETECTION" Carrier="CTLXXDESM01" ImpairmentNo="" ImpairmentDescription="Impairment Detected During Call Setup"
[root@linux121213 ~]#
[root@linux121213 ~]# cat /var/log/messages | grep "IMPAIRIMENT DETECTION" | grep "10:42:01"
Sep 20 10:42:01 linux121213-prod aupadhya: SearchKey=2023-09-20-T10_2_NGIA Message=Alarm created for (CallSetup) in the last (5) minutes, (2) Impairments were detected, from the total of (3) Attempts. The IMP% is (0.667) which is greater or equal to the Thershold% (0.1), for Customer (NGIA), Product (NGCS) by (Splunk) System. Please use this Alarm-ID (2023-09-20-T10_2_NGIA) for investigation Product=NGCS Customer=NGIA System=Splunk Application=IMPAIRIMENT DETECTION Carrier=CTLXXDESM01 ImpairmentDescription=Impairment Detected During Call Setup
[root@linux121213 ~]#

My problem is, there is another server, which is not able to send these logs to /var/log/messages. If I know that, I can apply same login to that non-working server too.
Any advice please?

Thanks

2

I think in CentOS 7 journald is the primary system logger. It does not produce the traditional log files, and the journal DB can be accessed by the command journalctl
Confirm that journald.conf configures forwarding to rsyslog, and /etc/rsyslog.conf or /etc/rsyslog.d/* loads imjournal or imuxsock. See

I have no good knowledge of splunk; it could use either the files created by rsyslog or the journal DB.

3

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.