Perl: Opening a filehandle but not getting anything back from it

I have two perl functions defined, both run a set of shell commands on some somplied data and return hashs of the resulting parsed output from these shell commands.

One works, one doesn't and I can't seem to see why. It's driving me insane :mad:

The working one:

sub getcellstatus {
        my %cells;
        my @cellnames=@{$_[0]};
        my @sysmanhandles;
        foreach my $check (@allchecks) {
                my $sysmancommand="$config{'su'} - $config{'user'} -c \'$config{'sysman'} ${check}Check -l all\'";
                local *SYSMAN;
                open(SYSMAN,"$sysmancommand|") || return;
                push(@sysmanhandles, *SYSMAN);
        }
        foreach my $check (@allchecks) {
                my $cellcount=0;
                $sysmanhandle=shift @sysmanhandles;
                while(<$sysmanhandle>) {
                        if (/^<([^>]*)> <([^>]*)> (.*)$/) {
                                $ip=$1;
                                $result=$2;
                                $note=$3;
                                $cells{$cellnames[$cellcount]}{$check}="${result}:${note}";
                                $cellcount++;
                        }
                }
                close($sysmanhandle);
        }
        %cells;
}

The not-working one:

sub getndstatus {
        my %results;
        my @nds=@{$_[0]};
        my @sysmanhandles;
        foreach my $check("EnableND","StartND") {
                foreach my $nd(@nds) {
                        my $sysman="$config{'su'} - $config{'user'} -c \'$config{'sysman'} ${check}Check -m $nd\'";
                        local *SYSMAN;
                        open(SYSMAN, "$sysman|") || return;
                        push(@sysmanhandles, *SYSMAN);
                }
        }
        foreach my $check("EnableND","StartND") {
                foreach my $nd(@nds) {
                        $sysmanhandle=shift @sysmanhandles;
                        while(<$sysmanhandle>) {
                                if (/^<([^>]*)> <([^>]*)> (.*)$/) {
                                        $ip=$1;
                                        $result=$2;
                                        $note=$3;
                                        $results{$nd}{$check}="${result}:${note}";
                                }
                        close($sysmanhandle);
                        }
                }
        }
        %results;
}

As you can see, each subroutine opens a set of filehandles, one per combination of inputs, gets them all running in the background (as they can take a few seconds to run) then parses the output from them in the order they were opened.

The non-working example above looks like it's working perfectly (I can see the shell commands executing via ps and the sulog) but nothing comes back via STDOUT when I go to read from the handle.

I've even tried adding:

> /tmp/test$check$$

to the end of the commandline and it does indeed write sensible output to the temp files that creates, but I'll be damned if I can get the same data to read off the filehandle instead.

I've also tried rewriting the subroutines to do each command one by one on a single handle, rather than use the array of handles and I get the same effect, the first one works (although more slowly), the second does not.

I've added print statements all throughout and the command looks good (if I copy-paste it into a prompt it runs just fine), but a print $_; in the while loops shows good data on the working one and nothing at all on the failing one.

I've even gone so far as to rewrite the troblesome routine from scratch to see if I just had some typo in there but the problem comes back even then.

I've bolded the filehandle open, close and commandline string creation to make it easier to read.

Any bright ideas? (And yes, I know the getcellstatus one is a bit ugly with all that $cellcount indexing but I'm afraid to touch it as it's the only one that works! :))

I didn't know you were allowed to do stuff like that with file handles. Does it help if you upgrade to File::IO objects instead? They're supposed to make stuff like this doable without resorting to deep voodoo and hoping the symbol table stays sane.

You localized a typeglob named SYSMAN in the loop where you build up
the filehandle array.
I am still also used to localizing filehandles like that.
But anyway, as current Perl releases allow to use lexically scoped variables as filehandles (see POD for open, or perlopentut)
have you tried if using them would make a difference?

May I ask why you are using two separate foreach loops in your subs?
If you put it in one loop there was also no need for the @sysmanhandles array.

Also I would assume that the backslashes that quote the single quotes in your su command are redundant here

I'll give that a try but I suspect I'll hit trouble as I'm obliged to use a fairly old perl (old system that's not to be fiddled with more than absolutely necessary).

Ah, this is my cunning way to improve the speed, the sysman commands are quite long-running so I'm starting all of them at once rather than running them one by one. I got the same problem with it dropping all stdout (it is actually executing, I can see the results of the command run) back when it was just a single filehandle and one foreach loop.

That's to make sure the entire sysman command gets passed to su properly, I don't want it to try and do something silly with the parameter groupings. That said, su is fairly smart so I'll try taking them out and see if it helps (simpler commandlines are always better IMO :slight_smile: )

I had the same trouble before I started doing voodoo though so I don't think it's a handle getting muddled up :slight_smile:

I got the idea from my own insane musings then found an example on PerlMonks - The Monastery Gates

Interesting news on this:
Once again, I rewrote it (feeling sure it had to be a typo someplace as such a similar function elsewhere was working) and it's now working!
The working code:

sub getndstatus {
        my %results;
        my @nds=@{$_[0]};
        my @sysmanhandles;
        foreach my $check("EnableND","StartND") {
                foreach my $nd(@nds) {
                        my $sysman="$config{'su'} - $config{'user'} -c \'$config{'sysman'} ${check}Check -m $nd\'";
                        local *SYSMAN;
                        open(SYSMAN, "$sysman|");
                        push(@sysmanhandles, *SYSMAN);
                }
        }
        foreach my $check("EnableND","StartND") {
                foreach my $nd(@nds) {
                        my $sysmanhandle=shift @sysmanhandles;
                        while(<$sysmanhandle>) {
                                if (/^<([^>]*)> <([^>]*)> (.*)$/) {
                                        $ip=$1;
                                        $result=$2;
                                        $note=$3;
                                        $results{$nd}{$check}="${result}:${note}";
                                }
                        }
                        close($sysmanhandle);
                }
        }
        %results;
}

It turned out the escaped single quotes in the command where required, otherwise su took the remaining parameters to sysman and assumed they where for itself instead.

I can't actually see any differences in this version but I made a special effort to write it 'blind' - that is, I avoided looking at the old code when I wrote this.

Hi Smiling Dragon,

glad you could fix that.

Because my eyes got too distracted I copied and pasted your buggy and fixed versions of the second sub and made a diff

$ diff bla?
9c9
<                         open(SYSMAN, "$sysman|") || return;
---
>                         open(SYSMAN, "$sysman|");
15c15
<                         $sysmanhandle=shift @sysmanhandles;
---
>                         my $sysmanhandle=shift @sysmanhandles;
23d22
<                         close($sysmanhandle);
24a24
>                         close($sysmanhandle);

I think the removal of return on a failed open didn't make the difference
(though I would assume it to be better style after all if you couldn't reconcile with a die on that condition).
However, what I think made the difference was the explicit lexical redeclaration of $sysmanhandle.

I also would have advised to set the autoflush before the reading loop by squeezing in a

$|++;

hadn't you come up with the solution.

Sorry, for ill-advising you on presumed redundancy of quotes.
Even if I were right it is always better to be explicit.
I just was thinking about a talk I once attended at a Perl conference or workshop held by
Mark Jason Dominus (author of Higher Order Perl)
where he ranted about cave men dwellers' useless quoting voodoo.
He there brought onto my radar the wonderful Deparse module
to check what Perl really makes out of your expressions.
Have a look at

$ perldoc B::Deparse

in case you haven't come about it yet.

The lexical vars as valid file handles to an open call became available as of Perl 5.6.0
if I remember correctly.
If your Perl is that old or even older then you should really consider an update :wink:

I should have thought of that - good idea! :slight_smile:

Tried adding || return back in, still works (so I've kept it in).

Tried adding and removing the 'my' flag and it made no difference (so I've left it in).

I'm assuming this is just a false hit caused by a different line number.

So that leaves me still wondering what I did wrong the first time but not the second... It feels like an unprintable character or something but I've pulled out the old broken version and stupidly deleted it so I can't go and find out.

Tha's a good point, I've not been paying much attention to my buffers. I might go back through the whole thing and make a concious decision in each case. The close should clear most things but best to be safe :slight_smile:

I see some loopy quoting myself and have been a culpret too from time to time :slight_smile:

Will do so, that sounds really useful for optimising some of my more complex strings.

5.005_03 - like I said, pretty old :slight_smile: I could potentially upgrade but I'm trying to make this thing work on an existing production system without having to install much else.
I'll avoid going to a sensible version of perl as long as I can.

That's really legacy stuff but you can be glad it's not even 5.004.
However, I can see your constraints.
I also have many systems where I am not supposed to install anything
even if it was for the better.

I've just come back from YAPC::EU 2008 where I got many inspirations,
foremost of course to move to Perl 5.10 which unanimously was considered the best Perl ever and as Damian put it, already feels like 10% of Perl 6.
Until we get Perl 6 which should be out by Xmas - we can't yet tell which Xmas.
But then every day will be like Xmas.

I think that's the best you can do because at Perl Monks you will find many of the world's best Perl hackers.