Merge lines together in unix

UNIX for Beginners Q & A UNIX for Dummies Questions & Answers
1

I have a file like this. Pls help me to solve this in ksh
(I should look for only Message : 111 and need to print the start time to end time
Need to ignore other type of messages. Ex: if first message is 111 and second message is 000 or anything else then ignore the 2nd one and print start time of the first message. )

Example:

08-24-10 10:22:34,xxxMessage : 111
08-24-10 10:24:38,xxbMessage : 111
08-24-10 10:24:39,xxcMessage : 111
08-24-10 10:24:40,xxdMessage : 000 
08-24-10 11:28:11,xxeMessage : 111
08-24-10 11:32:35,xxfMessage : 111

The output should be

08-24-10 10:22:34 to 08-24-10 10:24:39
08-24-10 11:28:11 to 08-24-10 11:32:35
2 
# cat infile
08-24-10 10:22:34,xxxMessage : 111
08-24-10 10:24:38,xxbMessage : 111
08-24-10 10:24:39,xxcMessage : 111
08-24-10 10:24:40,xxcMessage : 000
08-24-10 11:28:11,xxeMessage : 111
08-24-10 11:32:35,xxfMessage : 111

# ./justdoit
08-24-10 10:22:34 to 08-24-10 10:24:39
08-24-10 11:28:11 to 08-24-10 11:32:35

## justdoit ##
#!/bin/bash
notdel=111
rm -f tmpX 2>/dev/null
while read -r l
  do
    if [[ $(echo "$l" | grep $notdel) ]] ; then
     echo "$l" >> tmpX
    else
     cnt=$(cat tmpX | wc -l)
     y="";   x="N;"
      while [ $(( cnt -= 1 )) -gt 1 ]
       do
        y="$y$x";
       done
       sed -e '{/111/N;}' -e "{/111/!d;$y;bx}"  -e :x -e '{s/^\([^,]*\).*\n\([^,]*\).*$/\1 to \2/}'  tmpX
       rm -f tmpX
    fi
  done<infile
cnt=$(cat tmpX | wc -l)
y="";x="N;"
   while [ $(( cnt -= 1 )) -gt 1 ]
    do
     y="$y$x";
    done
      sed -e '{/111/N;}' -e '{s/^\([^,]*\).*\n\([^,]*\).*$/\1 to \2/}'  tmpX
3

I think this is a bit easier to understand:

#!/usr/bin/env ksh
while read line
do
        if [[ $line == *" : 111" ]]              # line with your flag
        then
                if [[ -z $start_time ]]          # dont have start; save start
                then
                        start_time="${line%%,*}"   # ditch all from first comma to end of line
                else                                   # have start, just update end
                        end_time="${line%%,*}"
                fi
        else                                   # not our line, print out last start/end pair
                if [[ -n $end_time ]]             # but only if we have a pair and not printed
                then
                        printf "%s  to  %s\n" "$start_time" "$end_time"    # print times
                        start_time=""                # next time we see is a start time
                        end_time=""
                fi
        fi
done <input-file
if [[ -n $end_time ]]             # handle last case, but only if both seen
then
        printf "%s  to  %s\n" "$start_time" "$end_time"    # print times
fi
4

agama,

Thanks a lot !
I am having some problem with the above code. The above code works only for few cases. Please see the below example.

2010-08-26 11:22:34,611 INFO   [Thread-xxxx]- [(xxxx,2723)Service Message : 111
2010-08-26 11:28:11,573 INFO   [Thread-abcd]- [(xxxxx,2723)Service Message : 000
2010-08-26 11:32:35,187 INFO   [Thread-zs]- [(xxxxx,2581)Service Message :   |LS
2010-08-26 11:33:35,189 INFO   [Thread-zs]- [(xxxxx,2581)Service Message : 111
2010-08-26 11:33:35,189 INFO   [Thread-zs]- [(xxxxx,2581)Service Message : 111
2010-08-26 11:34:36,189 INFO   [Thread-zs]- [(xxxxx,2581)Service Message : 111
2010-08-26 11:35:37,573 INFO   [Thread-abcd]- [(xxxxx,2724)Service Message : |LS
2010-08-26 11:36:38,189 INFO   [Thread-zs]- [(xxxxx,2585)Service Message : 111

Output should be :

2010-08-26 11:22:34
2010-08-26 11:33:35 to 2010-08-26 11:34:36
2010-08-26 11:36:38
5 
# cat infile
2010-08-26 11:22:34,611 INFO [Thread-xxxx]- [(xxxx,2723)Service Message : 111
2010-08-26 11:28:11,573 INFO [Thread-abcd]- [(xxxxx,2723)Service Message : 000
2010-08-26 11:32:35,187 INFO [Thread-zs]- [(xxxxx,2581)Service Message : |LS
2010-08-26 11:33:35,189 INFO [Thread-zs]- [(xxxxx,2581)Service Message : 111
2010-08-26 11:33:35,189 INFO [Thread-zs]- [(xxxxx,2581)Service Message : 111
2010-08-26 11:34:36,189 INFO [Thread-zs]- [(xxxxx,2581)Service Message : 111
2010-08-26 11:35:37,573 INFO [Thread-abcd]- [(xxxxx,2724)Service Message : |LS
2010-08-26 11:36:38,189 INFO [Thread-zs]- [(xxxxx,2585)Service Message : 111

# ./justdoit
2010-08-26 11:22:34
2010-08-26 11:33:35 to 2010-08-26 11:34:36
2010-08-26 11:36:38

 
## justdoit ##
#!/bin/bash
notdel=111
rm -f tmpX 2>/dev/null
while read -r l
  do
    if [[ $(echo "$l" | grep $notdel) ]] ; then
     echo "$l" >> tmpX
    else
     cnt=$(cat tmpX 2>/dev/null | wc -l )
     y="";   x="N;"
      while [ $(( cnt -= 1 )) -gt 1 ]
       do
        y="$y$x";
       done
       sed -e '{/111/N;}' -e "{/111/!d;$y;bx}"  -e :x -e '{s/^\([^,]*\).*\n\([^,]*\).*$/\1 to \2/}' tmpX 2>/dev/null | sed 's/,.*//'
       rm -f tmpX
    fi
  done<infile
cnt=$(cat tmpX 2>/dev/null | wc -l )
y="";x="N;"
   while [ $(( cnt -= 1 )) -gt 1 ]
    do
     y="$y$x";
    done
      sed -e '{/111/N;}' -e '{s/^\([^,]*\).*\n\([^,]*\).*$/\1 to \2/}'  tmpX 2>/dev/null | sed 's/,.*//'
6

Yes, the code sample I gave assumed that there was always a start/end pair. You didn't mention how to handle the case that you've pointed out.

Small change below handles this case now.

#!/usr/bin/env ksh
while read line
do
        if [[ $line == *" : 111" ]]              # line with your flag
        then
                if [[ -z $start_time ]]          # dont have start; save start
                then
                        start_time="${line%%,*}"   # ditch all from first comma to end of line
                else                                   # have start, just update end
                        end_time="${line%%,*}"
                fi
        else                                   # not our line, print out last start/end pair
                if [[ -n $end_time ]]             # print pair if we have both
                then
                        printf "%s  to  %s\n" "$start_time" "$end_time"    # print times
                else
                      if [[ -n $start_time ]]
                      then
                        printf "%s\n" "$start_time"
                      fi
                fi
                start_time=""                # next time we see is a start time
                end_time=""
        fi
done <input-file
if [[ -n $end_time ]]             # handle last case, but only if both seen
then
        printf "%s  to  %s\n" "$start_time" "$end_time"    # print times
else
        if [[ -n $start_time ]]
        then
           printf "%s\n" "$start_time"
        end
fi
7 
awk 'BEGIN{c=0;start="";end=""}
{
   if ($NF=="111")
      { split($2,a,",")
        if (c==0) start=$1 FS a[1]
        else end=$1 FS a[1]
        c++;next;
      }
   else {
          if (start=="") next
          if (end=="") print start
          else print start, "to", end
          c=0;start="";end="";next
       }
}
END { if (c>0)           
          {if (end=="") print start
          else print start, "to", end}
    }
' infile
8

Thanks to agama, rdcwayx, ygemici.
All three codes are working fine.
Thanks a lot!