UNIX fifo concurrent read from a named pipe

Shell Programming and Scripting
1

I have created a fifo named pipe in solaris, which writes the content of a file, line by line, into pipe as below:

$ mkfifo namepipe

$ cat books.txt
"how to write unix code"
"how to write oracle code"

$ cat books.txt >> namepipe &

I have a readpipe.sh script which reads the named pipe in parallel like this:

# readpipe.sh

while IFS=',' read var
do
  echo var >> log.txt

done < namepipe

I am calling the readpipe.sh like

readpipe.sh &
sleep 2
readpipe.sh &

I have introduced a sleep 2 to avoid a race condition to occur, i.e. two processes get parts of values from each line

The problem I am facing is when the all the contents of namepipe is completed, the first background process gets exited, while the second keeps on running without completing.

The logic in script is declared simple in here for clear understanding. Actual readpipd.sh does lots of activities.

Kindly help me with knowledge

2
  • You need a wait statement after the second readpipe.sh
  • echo var >> log.txt . The $ sign is left out of the variable expansion and you need to quote the variable.
  • Why are you appending instead of writing to the pipe
  • The first readpipe will read all there is in the pipe. The second one gets nothing
3

Probably best to use a mutex/semaphore to ensure only 1 process only reads from the pipe at a time.

I don't know what OS you are using so not sure if lockfile(1) is available to you.

This solution uses a lock directory as mkdir is supposed to be atomic:

readpipe.sh

lockdir=lock.dir

get_book() {
   local DONE=0
   while ! mkdir $lockdir 2> /dev/null
   do
      sleep 0.02
   done

   IFS="," read -u7 var || DONE=1

   rmdir $lockdir
   return $DONE
}

exec 7< namepipe
while get_book
do
   echo "$var" >> log$$.txt
done
exec 7<&-

Here is my test:

$ cat naveen 
mkfifo namepipe
./readpipe.sh &
./readpipe.sh &
exec 3<> namepipe
cat books.txt >&3
# Give time for readers to get established
sleep 1
# Close the write end
exec 3>&-
# wait for readers to complete
wait

$ ./naveen 

$ wc -l books.txt log*
 200 books.txt
 106 log2838.txt
  94 log2839.txt
 400 total

$ diff <(sort books.txt) <(cat log* | sort)

Update: below is an example using flock(1):

lockfile=lock.file

get_book() {
  var=$( (
      if flock -w .007 -n 200
      then
        IFS="," read -u7 var || exit 1
        echo "$var"
      fi) 200>$lockfile )
}

exec 7< namepipe

while get_book
do
  if [ -n "$var" ]
  then
          # Do any required work with the book record here...
          echo "$var" >> log$$.txt
  fi
done
exec 7<&-