Remove footer record in specific condition

UNIX for Beginners Q & A
1

Hi Experts,

we have a requirement , need your help to remove the footer record in the file.

Input file :

1011070375,,21,,NG,NG,asdfsfadf,1011,,30/09/2017,ACI,USD,,0.28,,,,,,,,,,,,
1011070381,,21,,NG,NG,sgfseasdf,1011,,30/09/2017,ACI,GBP,,0.22,,,,,,,,,,,,
1011070389,,21,,NG,NG,werwfsafds,1011,,30/09/2017,ACI,USD,,0.63,,,,,,,,,,,,
1011070389,,21,,NG,NG,safsdgfe,1011,,30/09/2017,ACI,EUR,,0.63,,,,,,,,,,,,
1011070456,,94,,GB,GB,sfsdsa,1011,,31/10/2017,ACI,GBP,,0.55,,,,,,,,,,,,
JE,11-01-2017/05:38,26504

The above highlighted record needs to remove in the file
instead of "JE", some other values also will come. so start with "JE" will fails to convert dynamically.

Thanks in advance

2

Please re-read the forum rules you agreed to when you joined this forum. Continued refusal to follow the rules may lead to be placed in read-only mode or in being banned from the UNIX & Linux Forums.

If values other than JE are at the start of records that need to be removed, how are those records identified?

Do you always want to remove every line that starts with a J ?

Do you always want to remove every line that starts with an uppercase alphabetic character?

Do you always want to remove every line that starts with a character that is not a numeric character?

Do you always want to remove the last line in every file you process?

Do you always want to remove every line that contains exactly two commas?

What operating system are you using?

What shell are you using?

What have you tried to solve this problem on your own?

3

we are using the following version of os version

Linux uklvaucup01a 2.6.32-696.3.2.el6.x86_64 #1 SMP Wed Jun 7 11:51:39 EDT 2017 x86_64 x86_64 x86_64 GNU/Linux

Your question :
Do you always want to remove every line that contains exactly two commas?

Yes , but need not look all line for 2 commas, usual it will be found at the footer record. these record will not be available start and middle. so our approach is to look tail of the file and matches the pattern as less than 2 or 3 commas .

thanks for your quick response

4

What about the other questions that Don Cragun asked, to help you give people in here a decent specification?

5

If values other than JE are at the start of records that need to be removed, how are those records identified?

Do you always want to remove every line that starts with a J?
No, we need to remove footer record which contains comma less than 3 count
Do you always want to remove every line that starts with an uppercase alphabetic character?
No
Do you always want to remove every line that starts with a character that is not a numeric character?
No
Do you always want to remove the last line in every file you process?
No, because some times they are not giving footer record.in such case the actual value get removed.
Do you always want to remove every line that contains exactly two commas?
Yes. but we can apply the condition as less than 3 counts of comma.
What operating system are you using?

Linux uklvaucup01a 2.6.32-696.3.2.el6.x86_64 #1 SMP Wed Jun 7 11:51:39 EDT 2017 x86_64 x86_64 x86_64 GNU/Linux
What shell are you using?

Linux uklvaucup01a 2.6.32-696.3.2.el6.x86_64 #1 SMP Wed Jun 7 11:51:39 EDT 2017 x86_64 x86_64 x86_64 GNU/Linux
What have you tried to solve this problem on your own?
sorry i haven't any idea to solve this issue.

6

How about

sed -r '$ {s/^([^,]*,){,2}[^,]*$//;T;d;} ' file
7

You could also try:

awk -F, 'NF>3' file