Getting piped input from a program that's buffering it's stdout

The old buffering problem again, in a very specific case. On FreeBSD this time, but it's the generic line-buffered vs fully-buffered problem.

I'm trying to pick an available bluetooth speaker (all named audio_N), by pinging all of them en mass and taking the first to respond.
The pinger-utility holds it's output, when it is going into a pipe, so there's a delay before I get the first response.

Can anyone think of a generic solution to make things snappy? I'm not so irritated by the delay, that it would warrant a custom C-program or a patch into l2ping. I have heard of something called expect , but have not used it ever.

Juha

Edit: I didn't think it through. I'll just launch a player to each of them, and one or zero of those succeeds. Meh...

#!/bin/sh
#
# send 2 pings, 1 second interval, to each given device.
# print out the quickest to respond, if any.
#
# processes might be left running in the background for a while,
# if there are devices that don't respond.
#
# all output from pings goes into a single pipe, it's okay
# as/while l2ping writes line-by-line or everythings at once during exit,
# less than PIPE_BUF per write.
# l2ping does not fflush stdout. takes 1 second always, or more :(

for i in `sed -n "s/.* \($1\).*/\1/p" /etc/bluetooth/hosts`
do
        l2ping -c2 -i1 -a"$i" &
done | sed -n '/.* bytes from \(.*\) seq_no=.* time=.* ms result=0 *$/ {
        s//\1/p
        q
}
'

$1 for the script is audio_. in my case.

Nothing outside the program can induce it to print its output early. Some languages like python respond to a particular environment variable which forces them to not print buffered.

I had a peek into the expect , and it does look like there would a mechanism to run the children in a pty, to seduce line-buffering. Very long manpage though.

Juha

Some programs are smart enough to turn off buffering when talking to a terminal or pipe but nothing forces them to. Its a choice the program makes - "I am talking to a terminal, therefore, turn off the internal buffer". It's not something chosen from the outside. It's not even anything system at all. It's a choice between calling write() once, to write blocks of several thousand bytes at a time, or calling write() many times, one for each line. It's a difference in a program's logic.

In short, still entirely up to the program and can't be forced from the outside by any means if the program doesn't already support it somehow.

Yes, that's just what I was after, to run the pings inside a pty, so that they would choose line-buffering.

---------- Post updated at 01:02 AM ---------- Previous update was at 12:57 AM ----------

Hmm.. one-shots and explicit sleeps

ping -c1 ... &
(sleep 1; ping -c1 ...) &

Gets ugly quickly...

expect won't help. I've looked through l2ping.c and it prints line-buffered anyway, it's not what's holding you up.

I don't think this construct:

for X in whatever
do
   something &
done | command

...necessarily does what you think. Background commands are liable to freeze themselves when they try to write to the terminal. Make sure they have absolutely no connection to the terminal:

for X in whatever
do
   something 2>/dev/null </dev/null &
done | command

In particular, sed may be making you wait, too. Make sure you run sed -u.

They write to the pipe, no?

Doing the explicit sleep actually works really snappy. Immediate response!
Argument has to be -fc1, else the program makes an extra sleep after each ping.

They may also write to stderr.

I stealth edited my post, read again.

Nice catch. If there's a problem, the error message would be blocked, thanks.

---------- Post updated at 01:20 AM ---------- Previous update was at 01:19 AM ----------

sed has the quit on first matching line, it should not cause trouble.

Does this mean you have or haven't tried sed -u?

I didn't, it's

 $ time btpick audio_.
audio_1
    0.01s real     0.00s user     0.00s system

as-is

---------- Post updated at 01:25 AM ---------- Previous update was at 01:24 AM ----------

not very pretty, but must do, this time

ping="l2ping -fc1 -a "

for i in `sed -n "s/.* \($1\).*/\1/p" /etc/bluetooth/hosts`
do
        exec 2>&1
        $ping "$i" & (sleep 1; exec $ping "$i") &

done | sed -n '/.* from \(.*\) seq.* result=0 $/ {
        s//\1/p
        q
}
'

---------- Post updated at 01:41 AM ---------- Previous update was at 01:25 AM ----------

Tried it quickly, setting $i in the loop into something nonexisting, to force an error message. It all works okay, after all, without the 2>.

The interactive shell is just waiting for the script to end, and the shell running the script is non-interactive. It's children, the pingers, can freely write to stderr even when it's a terminal, they are not in background, job control-wise. Removing the redirection of the stderr, nice to not be hiding error messages.

Treppenwitz:

expect -c 'spawn prog; interact' seems to do what I was after.