Deleting rows from csv file

Shell Programming and Scripting
1

Hello,

I am supposed to process about 100 csv files. But these files have some extra lines at the bottom of the file. these extra lines start with a header for each column and then some values below. These lines are actually a summary of the actual data and not supposed to be processed. These unncessary lines may be different for each files. Data To be deleted are as under:

Yr, Month, Prod
2009, 09, XYX
2008, 04, YYY

Is there any way I can delete these unnecessary lines of each files without opening them individually and manually deleting it?

I Appreciate a help in this.

Regards

2

sed -e '/Yr, Month, Prod/d' -e '/2009, 09, XYX/d' -e '/2008, 04, YYY/d'

3

need -i option in above sed command.

4

True. But this option should be use with care! No try and error type of thing. Better use the option -i with a suffix like -i.bak which will create an copy of original file, just in case... :wink:

theshashi's solution will only work on the sample file given by OP. I understand that the lines following the Yr, Month, Prod line can have different values.

Here is a possible awk solution:

awk '/Yr, Month, Prod/{f=1}!f' file

Which is shorthand for:

awk '/Yr, Month, Prod/{f=1}!f{print}' file
5 
 
sed '/Yr, Month, Prod/', $d' file
6

Basically my requirement is to delete all the lines that follows the below mentioned line. ( delete should include the below line also)

Please help

"Yr, Month, Prod"

7

Did you tried the awk and sed solutions given above?

---------- Post updated at 05:46 PM ---------- Previous update was at 05:42 PM ----------

There is a small typo in edidataguy's solution:

sed '/Yr, Month, Prod/', $d' file
                      ^
# should be

sed '/Yr, Month, Prod/,$d' file
8

This sed command actually do a display of the file contents till that line. But it does not physically remove those lines from the file. Is it that i have to redirect output from this sed command to another file ? Or this sed command does the deletion of those lines? I did the same way it was mentioned here.

Thanks

9

Well, as usually with *nix commands the output is, by default, redirected to the screen (stdout). Now, if you want you can redirect it to a file:

sed '......' infile > outfile

That's the safe way. You keep the original file intact and you can inspect the outfile to see if it is like required. All it takes when everything is ok is to cp that outfile onto the original file.

Now, some version of sed permits to make the changes "in place" but be warned: there is no way back.

sed -i  '......' infile

I would recommend to use the option -i with a suffix:

sed -i.bak '......' infile

A backup file infile.bak will be created.