Replacing data of output file with input

Hi,

I have a ksh which peocess and get me data from 3 days...
ie if i process it on jan 28.. it gets data for 25, 26 and 27.... the process run every day and get previous 3 days data...all this data is appened to a file lets call time.out
Now time.out cannot have deplicate data so what i want is when ever the appeneding is done i want look for the date in the file if it matches with the date the data is holding then i want it to be replace with new data

the time.out is like this

.
.
20090124,00:02:31,00:00:11,2776,00:01:38,00:01:31,00:56:36,108938, 
20090125,00:02:31,00:00:11,2776,00:01:38,00:01:31,00:56:36,108938, 
20090126,00:02:33,00:00:35,8187,00:01:42,00:01:32,02:02:08,321055, 

while in the script we have the current data for which the process is running save in CUR_MAINT_DATE..
the way the output file is written is

 
Read.ksh `basename ${0}` $1 $2 ${CUR_MAINT_DATE} > runtime.out
echo >> time.out; tr -d '\n' < runtime.out >> time.out

The the script runs it initially get data of 25 ... now i want the script to cross check with the time.out file if the data is already present for that date then it have to replace that line with the data in runtime.out.... and so on till it comes to current maintaince date 27 as no data is there for 27 in time.out the script have put it in the time.out

can anyone say me how i can replace the data in output file with data form input

I've taken your input:

and created two files. The first test1.dat is as above. The second, test2.dat I took the first line above starting with "20090124" and changed that to "20090127" to spoof a date on the 27th, and moved it to the end so it looks like this:

20090125,00:02:31,00:00:11,2776,00:01:38,00:01:31,00:56:36,108938, 
20090126,00:02:33,00:00:35,8187,00:01:42,00:01:32,02:02:08,321055, 
20090127,00:02:31,00:00:11,2776,00:01:38,00:01:31,00:56:36,108938, 

Then I ran the following 'comm' command:

comm test1.dat test2.dat | tr -d '\t'

and got the following output:

20090124,00:02:31,00:00:11,2776,00:01:38,00:01:31,00:56:36,108938, 
20090125,00:02:31,00:00:11,2776,00:01:38,00:01:31,00:56:36,108938, 
20090126,00:02:33,00:00:35,8187,00:01:42,00:01:32,02:02:08,321055,
20090127,00:02:31,00:00:11,2776,00:01:38,00:01:31,00:56:36,108938, 

Which could be put into it's own file.

comm prints output in three columns. Col 1 is data only in the first file,
col 2 is data only in the second file and Col 3 is data in both files. The "columns" are created using tabs, and there is not more than one column's worth of data on a line. So by removing the tabs, I have output that lines up neatly, and gives me all the unique and non duplicated data from the previous two files.

I am sorry to say but i dint understand anything....

I dont want to compare file and remove duplicates....i want to cheack if that pattern is already present in the file then remove it and replace it with the input....if not just place the input...
The partten here CUR_MAINT_DATE... i want something with grep, sid and awk...

Thanks for the help

Seemed overkill to me to replace a line of data with an exact duplicate, when you can use a command that will get you only the newest data. 'comm' has options on the output.

What if that made things more complicated than they need to be?

Granted I could be missing something that you either haven't mentioned, or I just don't undestand your explanation very well. But if I understand you correctly, the file that should have all the data added to it is 'time.out' and that data is coming from 'runtime.out'

If the structure of both files are the same as you posted in your first post, then the comm command will work for you.

comm -23 runtime.out time.out >> time.out

You don't even need to run it through tr to get rid of tabs, since only the data in runtime.out not appearing in time.out will be printed in the first column w/o any added tabs.

Yes you are missing something that... time.out file is very old file and it have data form past 5 years... so it have to be the final file and the reason i have used this code is
echo >> time.out; tr -d '\n' < runtime.out >> time.out
because the data in runtime.out is like this

20090127,
00:02:33,
00:03:04,
62153,
00:01:39,
00:02:25,
01:39:55,
266355,
 ,
00:09:33,
00:00:22,
00:03:17,
00:00:02,
00:05:01,
 ,
 ,
 ,
00:03:21,
00:07:40,
 ,
00:03:38,
00:36:42,
 ,
 ,

Okay, suspected the second part, but you hadn't mentioned the format of runtime.out before so there was no way to know.

As I used comm with the -23 option, it wouldn't care about the old data of time.out at all, so that data would be safe, it would simply get the newest data from runtime.out, but the different file structure is a deal breaker unless you want to create a temporary file to hold data that can be comm'd to time.out.

I realize now that the reason you append a new line before inserting the data from runtime.out piped through tr to time.out is because after the last insertion, there is no newline since tr removes all newlines. I assume this is done on each and every record, so that you do wind up with newlines after each record, except for the last record.

I'd advisde changing this order if possible so the last record has a newline appended to it also, or that would be a deal breaker if you try to find a solution involving 'sed', because sed ignores lines w/o a newline at the end.

So if you make the line look as follows:

tr -d '\n' < runtime.out >> time.out;echo >> time.out

And then at the command prompt, one time only, before you run the collection script.

$> echo >> time.out

Then your good to go if you want to find a sed solution. That, however will take more time than I have today for me to look into it.

That really nice of you... thanks a lot...
I dont mind adding another tem file in between runtime.out and time.out if that will slove the problem... I just want some solution this as i have to forward the script by tomm morning to someone else...if not i will be a dealbreaker... I would prefer a sed... but i dont mind any other solution... till it works... please help me...

I took your point to note and i did little home work.... Comm command compares 2 files... i just dont want comparsion.... actually what i want is replacing....
ie for example... in time.out file there is alread data for date 25 now read.out again got 25.... i want to replace the data of 25 in time.out with runtime.out irrespective of there being any change in the data... and if there is no 25 in time.out then write it .... thats excatly what i want....Please say me that there some way i can do this and help me out... please....

If I understand your question correctly, you want to search for the date in time.out and if it is present, update it with the line from runtime.out or append to time.out, you may try the below code:

tr -d '\n' < runtime.out >temp.out
grep -q $CUR_MAINT_DATE time.out
if [ $? -eq 0 ]
   echo |cat time.out -|sed "s/^$CUR_MAINT_DATE.*/`cat temp.out`/"|awk '{ if( NR==1 ) printf $0; else printf "\n" $0; }'
else
  #append code
fi

This may be easier if you can have newline at the end of time.out file:

sed "s/^$CUR_MAINT_DATE.*/`cat temp.out`/" time.out

The code you gave almost work but there one problem....
i added appened and then to if and the code is like this....

read.ksh `basename ${0}` $1 $2 ${CUR_MAINT_DATE} > runtime.out
tr -d '\n' < runtime.out >temp.out
grep -q $CUR_MAINT_DATE time.out
if [ $? -eq 0 ]
then
   echo |cat time.out -|sed "s/^$CUR_MAINT_DATE.*/`cat temp.out`/"|awk '{ if( NR==1 ) printf $0; else printf "\n" $0; }'
else
  tr -d '\n' < runtime.out >> time.out;echo >> time.out
fi

Actually read.ksh is called by Mail.ksh
when i run Mail.ksh i get output like this

 ./Mail.ksh 1 1
Current maintenance date is 20090126
20090124,00:02:26,00:04:19,86752,00:01:37,00:02:50,01:07:52,150818, ,
20090125,00:02:31,00:00:11,2776,00:01:38,00:01:31,00:56:36,108938, ,
20090126,00:02:33,00:00:35,8187,00:01:42,00:01:32,02:02:08,321055, ,
20090127,00:02:33,00:03:04,62153,00:01:39,00:02:25,01:39:55,266355, ,
Current maintenance date is 20090127
20090124,00:02:26,00:04:19,86752,00:01:37,00:02:50,01:07:52,150818, ,
20090125,00:02:31,00:00:11,2776,00:01:38,00:01:31,00:56:36,108938, ,
20090126,00:02:33,00:00:35,8187,00:01:42,00:01:32,02:02:08,321055, ,
20090127,00:02:33,00:03:04,62153,00:01:39,00:02:25,01:39:55,266355, ,
Current maintenance date is 20090128

where i have to get like this

 ./Mail.ksh 1 1
Current maintenance date is 20090126
Current maintenance date is 20090127
Current maintenance date is 20090128

where all the curent ma... statement is comming becaue i have an echo... which i will remove later...
Ofcose its working as it is appeneding a new line to my time.out at the end ie 28... but i dont know if its replaceing 26 and 27 in time.out... as they are same...for time being...i will test that later with more data...
But please help me in removing this print or echo statement thats comming when i execute the script

Thanks

Bhagya

I made few changes and i got the result as i want.... Thanks a lot for being there for me