Read file line by line and process the line to generate another file

Hi,
i have file which contains data as below(Only sample shown, it may contain more data similar to the one shown here)

i need to read this file line by line and generate an output file like the one below

i.e based on N value the number of MSISDNs will vary, if N=1 then the following MSISDN will be only one and the next line is SMSMSG. I need to take MSISDN1 value and its corresponding SMSMSG in the next line and form it like an equation MSISDN=SMSMSG

similar way if N=5 then the next five lines will have the MSISDNs 1 to 5 and i need to make them as one value by separating them by + ( Not adding) i.e MSISDN1+MSISDN2+MSISDN3+MSISDN4+MSISDN5=SMSMSG( the message in the next line)

i have started in the following way ...but logically not correct i hope...is there any better way to do this.

#!/bin/bash
# Shell script utility to read a file line line.
# Once line is read it can be process in processLine() function
# You can call script as follows, to read myfile.txt:
# ./readline myfile.txt
# Following example will read line from standard input device aka keyboard:
# ./readline

# User define Function (UDF)
processLine(){
  line="$@" # get all args
  #  just echo them, but you may need to customize it according to your need
  # for example, F1 will store first field of $line, see readline2 script
  # for more examples
  # F1=$(echo $line | awk '{ print $1 }')

 # echo $line
}
 
### Main script stars here ###
# Store file name
FILE=""
 
# Make sure we get file name as command line argument
# Else read it from standard input device
if [ "$1" == "" ]; then
   FILE="/dev/stdin"
else
   FILE="$1"
   # make sure file exist and readable
   if [ ! -f $FILE ]; then
        echo "$FILE : does not exists"
        exit 1
   elif [ ! -r $FILE ]; then
        echo "$FILE: can not read"
        exit 2
   fi
fi
# read $FILE using the file descriptors
 
# Set loop separator to end of line
BAKIFS=$IFS
IFS=$(echo -en "\n\b")
exec 3<&0
exec 0<"$FILE"
while read -r line
do
        # use $line variable to process line in processLine() function
       n=""
       n=$(echo $line | /usr/xpg4/bin/awk -F"=" -v Ncnt="N" '$1==Ncnt { print $2 } ')
       if [ "$n" != "" ]; then
       for (( c=1; c<=$n; c++ ))
       do
          ......here i have to read the next line and filter out MSISDNs and SMSMSG
        processLine $line
done
exec 0<&3
 
# restore $IFS which was used to determine what the field separators are
IFS=$BAKIFS
exit 0

pls help me...

Try this

awk '/N=/ {
  val=substr($0,3);for(i=0;i<val;i++){
    pre="";if(i>0){pre="+"};getline;split($0, arr, "=");str=str pre arr[2];}
  getline;split($0, arr, "=");str=str"="arr[2];print str;str="";
}' file

regards,
Ahamed

1 Like

Hi,

yes its ok....thanks

Hi,

for this input file its malfunctioning. why so?

the result is coming out like the one below.

what could be issue....

awk -F= '/MSISDN/{s=(s=="")?$2:s"+"$2}/SMSMSG/{s=s"="$2;print s;s=""}' file

Hi

now the output is

i want the output as

one way,

awk '{ printf "%s", $0; getline; printf "%s\n", $0 }' inputfile

Hello, aemunathan:

Since your initial attempt is a shell script, I thought you'd find the following simple sh script instructive.

#!/bin/sh

while IFS== read -r key value; do
    case $key in
        MSISDN*) s=$s+$value ;;
         SMSMSG) printf '%s=%s\n' "${s#+}" "$value"
                 s= ;;
    esac
done

It reads from stdin, though I'm sure you can modify it to handle filename arguments. If there's anything about it that you don't understand, just ask.

Regards,
Alister

hi,

i hope i understood ur example. and i tried to add the input and output file as below, but it doesn't work.

#!/bin/sh
while IFS== read -r key value; do
    case $key in
        MSISDN*) s=$s+$value ;;
         SMSMSG) printf '%s=%s\n' "${s#+}" "$value"
                 s= ;;
    esac
done < /aemu/smsT.ini > /aemu/SMSfileGen.txt

if i use -r option am getting

after removing -r am getting

Pls help me

Which shell and operating system are you using? I tested with bash and it worked perfectly on the sample data you provided in a couple of earlier posts. Perhaps your /bin/sh isn't up to the task. Although the script should be compatible with a reasonably posix-compliant sh, perhaps you could try changing the first line to #!/bin/bash, as in your original script.

1 Like

Hi,

yes i tried already with ksh. its running but the output is the same...one thing i observed, if i do a more input file name (smsT.ini) and copy the content of the file and create a new file name(smsT.txt) and paste it there and use that file it gives the required output...File size also getting decreased from the original file size value since while pasting all spaces are replaced to single space.

How to come out of this magical moments

tried as below but no hope

#!/bin/ksh
#sed -e "s/[ <tab>]*//g" /aemu/smsT.ini > /aemu/smsT.txt
while read line
do
echo $line | tr -s " "
done < /aemu/smsT.ini > /aemu/smsT.txt
while IFS== read -r key value; do
    case $key in
        MSISDN*) s=$s+$value ;;
         SMSMSG) printf '%s=%s\n' "${s#+}" "$value"
                 s= ;;
    esac
done< /aemu/smsT.txt > /aemu/SMSfileGen.txt

---------- Post updated 04-19-11 at 03:58 PM ---------- Previous update was 04-18-11 at 11:27 PM ----------

Hi,

Nothing is helping the cause...
pls tell me to read this output file so that i can assign the first row value to one variable and second row to another variable and repeat the process. Am going to use only two variables.

assign like the one below.

---------- Post updated at 06:09 PM ---------- Previous update was at 03:58 PM ----------

Hi,

what is the mistake in the below code ?

num=2
count=$num
while read line
do
if [ "$count" != "1" ]; then
strMSID=$(echo $line)
count=`expr $count - 1`
else
lines=`echo $line | tr -s " "`
smstext=`echo $lines | sed -e 's/:/-/g'`
strsms=`echo $smstext | sed -e 's/ /+/g'`
echo $strsms
count=$num
echo "GET  /cgi-bin/sendsms?username=an&password=ad&to=8445&text=$strsms  HTTP/1.1"
(sleep 4; echo "GET  /cgi-bin/sendsms?username=an&password=ad&to=8445&text=$strsms  HTTP/1.1"; echo ""; sl
eep 5) | telnet 10.10.6.6 9080
fi
done< /home/SMS/SMSfile.txt

it give echoes out like

Pls help me out..

I think your file must have some control characters in it that are not visible and also not copied into your example text.
That would explain why the code works when I use it on the examples given. Perhaps try to remove the control characters and then try the awk command again?

1 Like

Hi

Its really great finding. Yes there was some hidden characters. ^M

and its removed and working fine.....

Its working for me though

/user/ahamed $ awk '/N=/ {
  val=substr($0,3);for(i=0;i<val;i++){
    pre="";if(i>0){pre="+"};getline;split($0, arr, "=");str=str pre arr[2];}
>   val=substr($0,3);for(i=0;i<val;i++){
>     pre="";if(i>0){pre="+"};getline;split($0, arr, "=");str=str pre arr[2];}
>   getline;split($0, arr, "=");str=str"="arr[2];print str;str="";
}' file> }' file
278=1415-3P-G in BSC jo Road II down at 17:01:14 18/04/2011
90278=1415-3P-G in jo Road II UP at 17:13:14 18/04/11

regards,
Ahamed

---------- Post updated at 09:04 AM ---------- Previous update was at 09:02 AM ----------

Didn't realize the thread was 2 page long and issue got resolved.

regards,
Ahamed

local $/="[IMS]";
while(<DATA>){
	chomp;
	if(my @arr=$_=~/(?:MSISDN1=(\d+)\n)?(?:MSISDN2=(\d+)\n)?(?:MSISDN3=(\d+)\n)?(?:MSISDN4=(\d+)\n)?(?:MSISDN5=(\d+)\n)?(?:MSISDN6=(\d+)\n)?SMSMSG(=.*)/){
		my @brr = grep {$_ ne ""} @arr;
		print join "+", @brr[0..$#brr-1];
		print $brr[$#brr],"\n";
	}
	
}