Error in .pl

cmccabe · October 18, 2014, 1:19pm

I change the directory to where the .vcf is, but it throws an error, Thank

 #!/bin/bash
echo -n "Enter ID  : "; read id
echo -n "What panel: "; read panel
OMR=Output_Mutation_Report
perl -aF/\\t/ -lne 'BEGIN{%m=map{chomp;s/\cM|\cJ//g;$p=join("\t",(split/\t/)[4,5]);($p,$_)} <>;$m{"#CHROM\tINFO"}=$m{"Chr\tSegment Position"}};/SEGPOS=(\d+)/ || /\t(INFO)\t/ or next;$p=$F[0]."\t".$1;exists $m{$p} and print join("\t",$_,$m{$p})' $id_$panel_$OMR.txt < $id_$panel_$OMR_Filtered.vcf > $id_matched.vcf

 perl ~/match.pl
Enter ID  : W40855
What panel: Marfan20
/home/cmccabe/match.pl: line 5: .vcf: No such file or directory

The original code:

 perl -aF/\\t/ -lne 'BEGIN{%m=map{chomp;s/\cM|\cJ//g;$p=join("\t",(split/\t/)[4,5]);($p,$_)} <>;$m{"#CHROM\tINFO"}=$m{"Chr\tSegment Position"}};/SEGPOS=(\d+)/ || /\t(INFO)\t/ or next;$p=$F[0]."\t".$1;exists $m{$p} and print join("\t",$_,$m{$p})' test_Marfan20_Output_Mutation_Report.txt < test_Marfan20_Output_Mutation_Report_Filtered.vcf > test_matched.vcf

Aia · October 18, 2014, 1:46pm

You need to tell the shell what a variable to be substituted is, since in this case it doesn't know were it stops being a variable.

This, for the shell means:

$id_$panel_$OMR_Filtered.vcf

substitute variable $id_
substitute variable $panel_
substitute variable $OMR_Filtered
Since those variables do not exist it will accept null, leaving only .vcf , which does not exist.

This is the way you have to treat it:

${id}_${panel}_${OMR}_Filtered.vcf

Do the same for the other parameters.

junior-helper · October 18, 2014, 4:05pm

Your script match.pl is in fact a bash script (although it contains one perl command) and thus the script extension .pl is irritating.
You should rename it to match.sh
When you call your script like perl ~/script.<some-extension-here> you are overriding the actual interpreter program ( /bin/bash in this case) and this is the reason why the script is misbehaving, imho.

Can you try bash ~/match.sh ?

cmccabe · October 20, 2014, 10:18am

The same error seems to happen. Thanks.

  bash ~/match.sh
Enter ID  : W40855
What panel: Marfan20
/home/cmccabe/match.sh: line 5: .vcf: No such file or directory

I'm not entirely clear on your post Aia as I am still learning shell scripting. Thanks :).

Corona688 · October 20, 2014, 11:47am

Show the changes you made please, or we are left guessing.

cmccabe · October 20, 2014, 12:16pm

The only change that was made is the file is called match.sh and is called by:

 bash ~/match.sh

The same code as in post 1 is being used with the same error resulting. Thank you :).

 #!/bin/bash
echo -n "Enter ID  : "; read id
echo -n "What panel: "; read panel
OMR=Output_Mutation_Report
perl -aF/\\t/ -lne 'BEGIN{%m=map{chomp;s/\cM|\cJ//g;$p=join("\t",(split/\t/)[4,5]);($p,$_)} <>;$m{"#CHROM\tINFO"}=$m{"Chr\tSegment Position"}};/SEGPOS=(\d+)/ || /\t(INFO)\t/ or next;$p=$F[0]."\t".$1;exists $m{$p} and print join("\t",$_,$m{$p})' $id_$panel_$OMR.txt < $id_$panel_$OMR_Filtered.vcf > $id_matched.vcf

Aia · October 20, 2014, 12:35pm

Add the following `{' and `}' in red.

perl -aF/\\t/ -lne '

BEGIN{ %m = map{
                chomp;
                s/\cM|\cJ//g;
                $p = join ("\t",(split/\t/)[4,5]);
                ($p,$_)
               } <>;
       $m{ "#CHROM\tINFO" } = $m{ "Chr\tSegment Position" }
    };

/SEGPOS = (\d+)/ || /\t(INFO)\t/ or next;
$p = $F[0]."\t".$1;
exists $m{$p} and print join ( "\t",$_,$m{$p} )'
${id}_${panel}_${OMR}.txt < ${id}_${panel}_${OMR}_Filtered.vcf > ${id}_matched.vcf

cmccabe · October 20, 2014, 12:53pm

As match.sh:

 #!/bin/bash
echo -n "Enter ID  : "; read id
echo -n "What panel: "; read panel
OMR=Output_Mutation_Report
perl -aF/\\t/ -lne '
    BEGIN{ %m = map{ chomp;
               s/\cM|\cJ//g;
               $p = join ("\t",(split/\t/)[4,5]);
               ($p,$_) } <>;
          $m{ "#CHROM\tINFO" } = $m{ "Chr\tSegment Position" } };
    /SEGPOS = (\d+)/ || /\t(INFO)\t/ or next;
    $p = $F[0]."\t".$1;
    exists $m{$p} and print join ( "\t",$_,$m{$p} )
${id_}${panel}_${OMR}.txt < ${id}_${panel}_${OMR}_Filtered.vcf > ${id}_matched.vcf

 
bash ~/match.sh
Enter ID : W40855
What panel: Marfan20
/home/cmccabe/match.sh: line 5: unexpected EOF while looking for matching `''
/home/cmccabe/match.sh: line 16: syntax error: unexpected end of file

As match.pl

 #!/bin/bash
echo -n "Enter ID  : "; read id
echo -n "What panel: "; read panel
OMR=Output_Mutation_Report
perl -aF/\\t/ -lne '
    BEGIN{ %m = map{ chomp;
               s/\cM|\cJ//g;
               $p = join ("\t",(split/\t/)[4,5]);
               ($p,$_) } <>;
          $m{ "#CHROM\tINFO" } = $m{ "Chr\tSegment Position" } };
    /SEGPOS = (\d+)/ || /\t(INFO)\t/ or next;
    $p = $F[0]."\t".$1;
    exists $m{$p} and print join ( "\t",$_,$m{$p} )
${id_}${panel}_${OMR}.txt < ${id}_${panel}_${OMR}_Filtered.vcf > ${id}_matched.vcf

 
perl ~/match.pl
Enter ID : W40855
What panel: Marfan20
/home/cmccabe/match.pl: line 5: syntax error near unexpected token `('
/home/cmccabe/match.pl: line 5: `perl -aF/\\t/ -lne 'BEGIN{%m=map{chomp;s/\cM|\cJ//g;$p=join("\t",(split/\t/)[4,5]);($p,$_)} <>;$m{"#CHROM\tINFO"}=$m{"Chr\tSegment Position"}};/SEGPOS=(\d+)/ || /\t(INFO)\t/ or next;$p=$F[0]."\t".$1;exists $m{$p} and print join("\t",$_,$m{$p})' $id_$panel_$OMR.txt < $id_$panel_$OMR_Filtered.vcf > $id_matched.vcfnd print join ( "\t",$_,$m{$p} )'

Thanks :).

Corona688 · October 20, 2014, 1:01pm

Is ${id_} really what you wanted?

Aia · October 20, 2014, 1:06pm

@cmccabe

change the first ${id_} to ${id}_

Also there's a closing (') missing. That was my fault. When trying to add the colors I must have removed it.

exists $m{$p} and print join ( "\t",$_,$m{$p} )'

cmccabe · October 20, 2014, 1:16pm

match.sh:

 #!/bin/bash
echo -n "Enter ID  : "; read id
echo -n "What panel: "; read panel
OMR=Output_Mutation_Report
perl -aF/\\t/ -lne '
    BEGIN{ %m = map{ chomp;
               s/\cM|\cJ//g;
               $p = join ("\t",(split/\t/)[4,5]);
               ($p,$_) } <>;
          $m{ "#CHROM\tINFO" } = $m{ "Chr\tSegment Position" } };
    /SEGPOS = (\d+)/ || /\t(INFO)\t/ or next;
    $p = $F[0]."\t".$1;
    exists $m{$p} and print join ( "\t",$_,$m{$p} )
${id}_${panel}_${OMR}.txt < ${id}_${panel}_${OMR}_Filtered.vcf > ${id}_matched.vcf

 
bash ~/match.sh
Enter ID : W40855
What panel: Marfan20
/home/cmccabe/match.sh: line 5: unexpected EOF while looking for matching `''
/home/cmccabe/match.sh: line 16: syntax error: unexpected end of file

Thanks.

Aia · October 20, 2014, 1:18pm

You're still missing the (') that I added to my previous post.

Here, copy and paste this part:

perl -aF/\\t/ -lne '

BEGIN{ %m = map{
                chomp;
                s/\cM|\cJ//g;
                $p = join ("\t",(split/\t/)[4,5]);
                ($p,$_)
               } <>;
       $m{ "#CHROM\tINFO" } = $m{ "Chr\tSegment Position" }
    };

/SEGPOS = (\d+)/ || /\t(INFO)\t/ or next;
$p = $F[0]."\t".$1;
exists $m{$p} and print join ( "\t",$_,$m{$p} )'
${id}_${panel}_${OMR}.txt < ${id}_${panel}_${OMR}_Filtered.vcf > ${id}_matched.vcf

cmccabe · October 20, 2014, 1:38pm

 #!/bin/bash
echo -n "Enter ID  : "; read id
echo -n "What panel: "; read panel
OMR=Output_Mutation_Report
perl -aF/\\t/ -lne '
    BEGIN{ %m = map{ chomp;
               s/\cM|\cJ//g;
               $p = join ("\t",(split/\t/)[4,5]);
               ($p,$_) } <>;
          $m{ "#CHROM\tINFO" } = $m{ "Chr\tSegment Position" } };
    /SEGPOS = (\d+)/ || /\t(INFO)\t/ or next;
    $p = $F[0]."\t".$1;
    exists $m{$p} and print join ( "\t",$_,$m{$p} )'
${id}_${panel}_${OMR}.txt < ${id}_${panel}_${OMR}_Filtered.vcf > ${id}_matched.vcf

The command runs (match.sh), but before as a one-liner the same file took seconds, it seems now it takes much longer > 10 minutes.

Thanks.

Aia · October 20, 2014, 1:50pm

Before it was working only with these files, right?

test_Marfan20_Output_Mutation_Report.txt test_Marfan20_Output_Mutation_Report_Filtered.vcf

Now, it is working with these files:

${id}_${panel}_${OMR}.txt
${id}_${panel}_${OMR}_Filtered.vcf

Are they the same file? How much difference in size are they, before and now?

The Perl code you presented has not been changed other than adding spaces to improve readability. Spaces do not affect performances at all.

cmccabe · October 20, 2014, 2:08pm

test_Marfan20_Output_Mutation_Report.txt = 21kb

test_Marfan20_Output_Mutation_Report_Filtered.vcf = 66kb

I attached the two files in a zip as well.

 
perl -aF/\\t/ -lne 'BEGIN{%m=map{chomp;s/\cM|\cJ//g;$p=join("\t",(split/\t/)[4,5]);($p,$_)} <>;$m{"#CHROM\tINFO"}=$m{"Chr\tSegment Position"}};/SEGPOS=(\d+)/ || /\t(INFO)\t/ or next;$p=$F[0]."\t".$1;exists $m{$p} and print join("\t",$_,$m{$p})' test_Marfan20_Output_Mutation_Report.txt < test_Marfan20_Output_Mutation_Report_Filtered.vcf > test_matched.vcf

the output test_matched.vcf is created in seconds.

 
bash ~/match.sh
Enter ID : test
What panel: Marfan20

no output file is created as the script
match.sh

 #!/bin/bash
echo -n "Enter ID  : "; read id
echo -n "What panel: "; read panel
OMR=Output_Mutation_Report
perl -aF/\\t/ -lne '
    BEGIN{ %m = map{ chomp;
               s/\cM|\cJ//g;
               $p = join ("\t",(split/\t/)[4,5]);
               ($p,$_) } <>;
          $m{ "#CHROM\tINFO" } = $m{ "Chr\tSegment Position" } };
    /SEGPOS = (\d+)/ || /\t(INFO)\t/ or next;
    $p = $F[0]."\t".$1;
    exists $m{$p} and print join ( "\t",$_,$m{$p} )'
${id}_${panel}_${OMR}.txt < ${id}_${panel}_${OMR}_Filtered.vcf > ${id}_matched.vcf

Thanks.

cmccabe · October 21, 2014, 10:43am

It seemed to work when I made it a one-liner. Thank you for all your help :).