Looping through input/output

UNIX for Beginners Q & A
21

The trace output you showed us in post #11 in this thread indicated that the command:

./match.pl -f 9477885102_R01C02header -g 9477885102_R01C01header -k 1 -l 1 -v '5 6 7' > 0

succeeded, that the next command:

./match.pl -f 9477885102_R02C01header -g 0 -k 1 -l 1 -v '5 6 7' > 1

failed producing the diagnostic:

Must specify file one using -g switch

and, then it appears that the command:

./match.pl -f 9477885102_R02C02header -g 1 -k 1 -l 1 -v '5 6 7' > 0

succeeds (although we didn't look at enough of the trace output to be sure of that).

You say that your script works with an input file named 0 , but you have shown us diagnostic output from the script (as copied above) that clearly shows that it does not. Should we believe what you showed us in post #11 or should we believe what you said in post #20? They are in direct conflict with each other.

22

To solve this issue of mine, I've done something not very elegant: I've written 5.427 commands with all the files to merge and just made an executable file with all the commands copy pasted there. This is how it looks like (for the first 11 files):

./match.pl  -f 9464294024_R01C01header -g 9466656017_R01C01header -k 1 -l 1 -v "5 6 7" > ./result/1
./match.pl  -f 9464294024_R01C02header -g ./result/1 -k 1 -l 1 -v "5 6 7" > ./result/2
./match.pl  -f 9464294024_R02C01header -g ./result/2 -k 1 -l 1 -v "5 6 7" > ./result/3
./match.pl  -f 9464294024_R02C02header -g ./result/3 -k 1 -l 1 -v "5 6 7" > ./result/4
./match.pl  -f 9464294024_R03C01header -g ./result/4 -k 1 -l 1 -v "5 6 7" > ./result/5
./match.pl  -f 9464294024_R03C02header -g ./result/5 -k 1 -l 1 -v "5 6 7" > ./result/6
./match.pl  -f 9464294024_R04C01header -g ./result/6 -k 1 -l 1 -v "5 6 7" > ./result/7
./match.pl  -f 9464294024_R04C02header -g ./result/7 -k 1 -l 1 -v "5 6 7" > ./result/8
./match.pl  -f 9464294024_R05C01header -g ./result/8 -k 1 -l 1 -v "5 6 7" > ./result/9
./match.pl  -f 9464294024_R05C02header -g ./result/9 -k 1 -l 1 -v "5 6 7" > ./result/10

I've used numbers to name the output files as it was easier to loop them for writing. The commands work without any problem. And I've got my output.

I must admit that I do not understand the script you've provided, nor the diagnostics (I'm a beginner, posting in the forum for beginners) and, thus, cannot comment on your statement of your script working or what the diagnostics say. I'm grateful for your time and help, but am a bit baffled as to why the general consensus is that the only possible explanations to this situation are in two possibilities (1) my perl script having a problem or (2) I'm not providing truthful/good information.

23

if you are looking for a loop then you could use a quick 'for' loop in the directory where match.pl file is present.

./match.pl  -f 9464294024_R01C01header -g 9466656017_R01C01header -k 1 -l 1 -v "5 6 7" > ./result/1
for f in `seq 2 10`
do
./match.pl  -f 9464294024_R01C01header -g 9466656017_R01C01header -k 1 -l 1 -v "5 6 7" > ./result/$f
done
24

OK, no problem, but why don't you, on the other hand, just trust Don Cragun that he knows what he talks about? If you just provide him with the necessary information he might just find out (and then explain it to you), but instead:

you just said something amounting to "no, that can't be". First: about the truthfulness or quality of the information: you just might have ommitted some vital point because you didn't recognise it as being vital. I do not say this is the case but it might be. And even if this is the case: you are a beginner - you are supposed to not get it right every time. There are no bad feelings, it is just about finding out the culprit.

Now, the second point: maybe the perl script you have has a problem, maybe not. If it does it would be simple enough to test: rename one of the files to "0" (as the name Dons script used) and try it manually. If you get the same result Dons suspicion about the perl script being the problem is proven and there is no shame in saying so. You might still decide then to use the workaround you used now, but you would learned something about the script. Maybe you get a correct result and that would prove Dons suspicion wrong. In this case just write here "i have tried that with the command XXXX and got the correct result of YYYYY and therefore your reasoning is wrong. This in turn helps us finding out what indeed the problem is and might lead to some (other) solution. This is how an expert would have handled that.

Either way, this is the way we (not only we in this forum, but UNIX admins everywhere) work: we have an unexplainable problem, we create theories of what is going on, then create testbeds to prove/disprove these theories until we finally find a solution. And more than amassing factual knowledge will getting into this mindset help you advancing from beginner to seasoned to expert.

I hope this helps.

bakunin

4 Likes
25

Apologies for any typos.

But was it tested with a file named 0, zero? Looking at your alternative code there is NO 0, zero, file generated at all; your files start with the number 1.

We are not trying to belittle you, just glean as much information as possible to satisfy your requirements. You have no idea how difficult some problems are for volunteers on here, and remember we ARE volunteers, to solve with limited or incorrect information. Until late into this thread no-one even knew about a Perl script and the error report pointed straight to an unknown script at that time. If we had the Perl script to view the Perl experts on here would have probably solved your problem without ANY errors from either the Perl or the Shell scripts well inside in less than 20 odd replies.

There can't be anything so secret about your Perl script that you cannot show it to us.

Your Perl script might need a specific _library_/module and that module may be causing this error, who knows, we don't, as we do not have this script to view.

Please oh please, for everyones' sanity, give as much information as possible so that the big guns on here, who give their time for free, can supply you with a solution ASAP.

26

Thank you all for your comments and suggestions. I do understand that the solution to any problem is proper communication and my point was that since we are communicating in writing only, even with best intentions and the biggest trust in the world, it is easy to get misunderstood or be misunderstood (both ways, not one way only). In any case, I apologise once again for not being able to provide an explanation that would be understandable to everyone or share every script. I do understand that it prevents people in this forum from helping me and have solved the problem in other way. Many thanks everyone for your time and help!

27

My script was just using two output files named 0
and 1 instead of 5,426 output files named ./result/1 through ./result/5246 . For the sequence of commands shown above that your script produces, my script would have created the commands:

./match.pl  -f 9464294024_R01C01header -g 9466656017_R01C01header -k 1 -l 1 -v "5 6 7" > 0
./match.pl  -f 9464294024_R01C02header -g 0 -k 1 -l 1 -v "5 6 7" > 1
./match.pl  -f 9464294024_R02C01header -g 1 -k 1 -l 1 -v "5 6 7" > 0
./match.pl  -f 9464294024_R02C02header -g 0 -k 1 -l 1 -v "5 6 7" > 1
./match.pl  -f 9464294024_R03C01header -g 1 -k 1 -l 1 -v "5 6 7" > 0
./match.pl  -f 9464294024_R03C02header -g 0 -k 1 -l 1 -v "5 6 7" > 1
./match.pl  -f 9464294024_R04C01header -g 1 -k 1 -l 1 -v "5 6 7" > 0
./match.pl  -f 9464294024_R04C02header -g 0 -k 1 -l 1 -v "5 6 7" > 1
./match.pl  -f 9464294024_R05C01header -g 1 -k 1 -l 1 -v "5 6 7" > 0
./match.pl  -f 9464294024_R05C02header -g 0 -k 1 -l 1 -v "5 6 7" > 1

and the trace output log you showed us said that the line shown in red above failed with the diagnostic message:

Must specify file one using -g switch

What output do you get when you run the first three of the above sequence of commands manually?

The commands in your script are logically equivalent to the commands my script was trying to run, but didn't need the extra disk space for 5,245 temporary files that your script produces. If all 5,247 of the commands like those above had succeeded, my script would have then moved the last temporary output file (either 0 or 1 ) to a file named outfile and then would have removed the other temporary file.

I certainly don't claim that all of the code I produce is perfect. But the trace output you showed us from running my script (as you modified it to run your perl script) shows that commands like those shown above were being run and that the command shown in red above failed when there doesn't seem to be any reason that you have explained to us why it should not succeed. My belief is that there is something about the filename 0 that match.pl doesn't like, but without understanding why it doesn't like it (and what other filenames it doesn't like), I don't know how complex I need to make the trivial code that switches between files 0 and 1 to produce filenames that will be accepted.

Just saying that my code doesn't work doesn't help me help you. Showing us the contents of match.pl or telling us the restrictions match.pl places on -g option option-argument values would help us help you. We want to help you.