split the file based on the 2nd column passing as a parameter

Shell Programming and Scripting
1

I am unable to spit the file based on the 2nd column passing as a parameter with awk command.

Source file:

�100�,�customer information�,�10000�
�200�,�customer information�,�50000�
�300�,�product information�,�40000�

script: the command is not allowing to pass the parameters with the awk

var=� customer information�
var1= �product information�
awk '{print > $var}' log.txt
awk '{print > $var1}' log.txt

output required

Filename:
Customer_information

�100�,�customer information�,�10000�
�200�,�customer information�,�50000�

product_information

�300�,�product information�,�40000�
2

Hi

Try this:

awk -F, '{x=$2;gsub("\"","",x);sub(" ","_",x);print > x}' log.txt

Guru

3

  1. It is working only for one instance.
    but i have the file customer infomation datail is not replacing all the occurances.
    customer_information_detail

  2. i need to pass parameter in the after splitting file naming convension.

var=1200
awk -F, '{x=$2;gsub("\"","",x);sub(" ","_",x);print > x$var".txt"}' log.txt

final output is

customer_information_detail_1200.txt

i am unable fetch using awk sub and gsub.

Thanks

4 
 awk -v var=1200 'BEGIN{FS=OFS="\""}{gsub(/ /,"_",$4) ;print > $4 "_detail_" var ".txt"}' urfile
5

I am unable to fetch the proper output for the above command.
Detailed information.
My source file:

"0","ANC BC","01","1200",1,"2000-01-11-13:13:13"
1,Customer Information sales Header 1,"Columns_name1","Columns_name2","Columns_name3","Columns_name4"
1,Customer Information sales Header 2,"CustomerID","ICode","InformationDate","Information"
"1","Customer Information Sales Detail","1000","abc","2000-01-31","ccc"
"1","Customer Information Sales Detail","1234","bbb","2000-01-31","aaa"
1,Product Information Header 1,"Columns_name1","Columns_name2","Columns_name3","Columns_name4"
1,Product Information Header 2,"ProductID","ICode","InformationDate","Information"
"1","Product Information Detail","1000","abc","2000-01-31","ccc"
"1","Product Information Detail","1234","bbb","2000-01-31","aaa"
"9",2,3,4

script

a=$codenumber #code number is 1200 -1500
b=$pid #pid is period date 20011212
awk -v var1=$a,var2=$b 'BEGIN{FS=OFS="\""}{gsub(/ /,"_",$0) ;print > $4"_"var1"_"var2".txt"}' myfile_name.txt

Finally i need to the output of only start with Detail file, i dont require all the header and trailer files.

Customer_Information_Sales_Detail_1200_20011212.txt

"1","Customer Information Sales Detail","1000","abc","2000-01-31","ccc"
"1","Customer Information Sales Detail","1234","bbb","2000-01-31","aaa"

Product_Information_Detail_1200_20011212.txt

"1","Product Information Detail","1000","abc","2000-01-31","ccc"
"1","Product Information Detail","1234","bbb","2000-01-31","aaa"

Thanks

6 
awk -v var1=$a -v var2=$b 'BEGIN{FS=OFS=","}{S=$0;gsub(/"/,"");gsub(/ /,"_",$2)}
                  /Detail/ {print S> $2"_"var1"_"var2".txt"}'  myfile_name.txt
7

Hi,

Try this,

sort -t, -k2 testfile | awk -v var1=$a -v var2=$b -F, '/Detail/ {d=$2;gsub(/\"/,"",d);c=d;gsub(/ /,"_",c);print > c"_"var1"_"var2".txt"}'
8

This is the one exactly i am looking for, I have tried with the small source file. it is working file.
but it is failing for another file due the below error message.Did the awk command does not work for bigger size files.
My oprating system is AIX

Error information:

awk: 0602-590 Internal software error in the tostring function on 1,Customer Extension Detail,111_222222222,222222226 333333333333333 ,ABC ,BBC ,2000-01-01,2001-01-01,,,,,,,,,,ABCDSED,ASJKSDF,,,,,.
The input line number is 6.73808e+06. The file is /input/source/file/AAAAABBB_200101_1111.csv.
The source line number is 1.

Script information:

SID=1111
PID=200101
T1=`echo $TargetDir"/"`
 
awk -v var1=$PID -v var2=$SID -v var3=$T1 'BEGIN{FS=OFS=","}{S=$0;gsub(/"/,"");gsub(/ /,"_",$2)} /Detail/ {print S> var3$2"_"var1"_"var2".txt"}' filename.csv

Thanks

---------- Post updated at 11:30 PM ---------- Previous update was at 05:57 AM ----------

Additional information,
I have guessed the below possibilities and cross verified.

  1. Data issue
    Cross checked the source data is having correct format. No issues with the source data.
    Tried to extract with grep for the particular string. i am able to fetch the data. No data issues.

  2. Awk command
    The below command need to change the syntax to execute for the bigger files.
    The below command is working fine with the small files <1GB. Not working more then 1GB file.
    I don't know where the awk execute in the server side - May be environment issue.
    I am unable to tune the below awk command.
    I have cross checked in the forams, i did not find the similar issue/error.

Finally, i have tried all the possibilities like source data, environmental issues (Hardware resource -CPU,Buffer)- every thing looks fine.
But still i am facing this issue. I am unable to figure out where is the problem to execute the below command

Thanks

---------- Post updated 07-15-10 at 03:49 AM ---------- Previous update was 07-14-10 at 11:30 PM ----------

Hi,
I am going to chnage the script

PID=20001010
SID=1200
var1=`echo $file"Detail"` #var1=customer_relation_information_Detail
var2=`echo $var1| awk '{gsub(/_/, " ");print}'` #var2=customer relation information Detail
echo $var2 #customer relation information Detail
 
awk '/$var2/' $InputDir/$file1 > `echo $TempDir/$file"Detail_"$PID"_"$SID".txt"`
#---------------------------------OR-------------------------------------------
awk -v temp=$var2 '/$temp/' $InputDir/$file1 > `echo $TempDir/$file"Detail_"$PID"_"$SID".txt"`
#---------------------------------OR-------------------------------------------
awk -v temp1='$var2' '/$temp1/' $InputDir/$file1 > `echo $TempDir/$file"Detail_"$PID"_"$SID".txt"`

The above three awk commands not taking the parameter even i have defind the pameter it is not fetching the data.

Thanks

---------- Post updated 07-16-10 at 12:18 AM ---------- Previous update was 07-15-10 at 03:49 AM ----------

Hi,

Now the 2nd command is working for fine.

Thanks for sharing the command.

awk -v var1=20001010 -v var2=1000 -v var3=/abc/bbc/file/ -F, '/Detail/ {d=$2;gsub(/\"/,"",d);c=d;gsub(/ /,"_",c);print > var3"/"c"_"var1"_"var2".txt"}' file_name

Now this is resolved.

I dont know how close the thread.
Thanks