Get status of dd running in background job

Hello everyone

While working on TUI for scripts, there there came the idea to' add a command' for dd too.
That was, after 'wrapping' tar and wget either, to display their growing size and return the exit code with a textual-visual-feedback to the user.

Now displaying the filesize of a tarball or a file that is downoaded during a loop is quite easy.

What i'd like to get is the value of how many times BS (as in bytesize) has been written.

So it would look similar to (the console output):
Linux : [ Text User Interface ] : tui-download & tui-tar - YouTube
At around 0:40 of the video.

The full script is:
Note that tui-printf & tui-status can be replaced by 'printf' or 'echo'
tui-indi prints on each call the following of its previous -/|\
source tui loads in the configuration files/default variables not defined in this script.

#!/bin/sh
#    Author:     Simon A. Erat (sea)
#    Contact:    erat.simon@gmail.com
#
#    Variable defaults
#
    script_version=0.3
    ME="$(basename $0)"
    help_text="\r$ME ($script_version)
        \rUsage: $ME SOURCE TARGET
        \rWrites SOURCE to TARGET
        \rRequires root rights.
        \r"
    source tui
    FILE_CMD="$TUI_TEMP_DIR/bgjob"
    SOURCE="$1"
    TARGET="$2"
#
#    Variable handling
#
    if [[ "$1" = "-h" ]] || [[ -z $2 ]];then
    echo -e "$help_text" && exit $RET_HELP;fi
#
#    Display
#
    CMD="dd if=$SOURCE of=$TARGET bs=$DD_BS status=none > /dev/zero"
    echo "$CMD" > $FILE_CMD ; chmod +x $FILE_CMD ; $FILE_CMD &
    sleep 0.3
    
    while [[ ! "" = "$(ps|grep -v tui|grep dd)" ]]
    do    sleep 0.7
        tui-printf "Writing $(basename $SOURCE) to $TARGET"  " [  $(tui-indi)   ]"
    done
    
    tui-status $? "Written $(basename $SOURCE) to $TARGET"

Which works like it should.

According to the manpage, one's supposed to get job's info by:

So i've changed accordingly:
(removed because outdated)

Problem now is:

# | Writing sea-AwesomeWM-F19.iso to /dev/sdb )                                                                                                                                            [  /   ] | #63+0 records in
63+0 records out
264241152 bytes (264 MB) copied, 4.22988 s, 62.5 MB/s
# | Writing sea-AwesomeWM-F19.iso to /dev/sdb )                                                                                                                                            [  -   ] | #65+0 records in
65+0 records out
272629760 bytes (273 MB) copied, 5.19806 s, 52.4 MB/s
# | Writing sea-AwesomeWM-F19.iso to /dev/sdb )                                                                                                                                            [  \   ] | #66+0 records in
66+0 records out
276824064 bytes (277 MB) copied, 6.16454 s, 44.9 MB/s
# | Writing sea-AwesomeWM-F19.iso to /dev/sdb )                                                                                                                                            [  |   ] | #
^C130 ~ $

I hoped redirecting the stderr to a variable would keep DD quite...
How could i achieve this?

...
CMD="dd if=$SOURCE of=$TARGET bs=$DD_BS"
...
while [[ ! "" = "$(ps|grep -v tui|grep dd)" ]]
    do    sleep 0.7
        pid=$(ps|grep -v tui|grep dd|awk '{print $1}')
        kill -USR1 $pid 2&> $TUI_TEMP_FILE
        tmp=$(grep "," $TUI_TEMP_FILE|awk '{print $3" "$4}')
        tui-printf "Writing $(basename $SOURCE) to $TARGET $tmp)"  " [  $(tui-indi)   ]"
    done

---------- Post updated at 16:39 ---------- Previous update was at 15:14 ----------

According to the manpages, this is supposed to work:

However, i have tried:

  1. kill -USR1 $pid 1&>2 > $TUI_TEMP_FILE
  2. kill -USR1 $pid 2&>1 > $TUI_TEMP_FILE
  3. kill -USR1 $pid > $TUI_TEMP_FILE > 2&>1
  4. CMD="dd if=$SOURCE of=$TARGET bs=$DD_BS > $TUI_TEMP_FILE > 2&>1" # status=none

In all caes $TUI_TEMP_FILE remained empty, while - if there were output - it was only shown to user but not written to tempfile.

It seems dd is either 'spamy' or dead-silent...
Any hints welcome. (or what am i understanding wrong?)

I would suspect more kill than dd... ( Last time I tried a kill fancy option I finished going in the white room to restart to box I lost control on...)

You cannot redirect a process after you start it. If you want dd's stderr to write into a file, you have to redirect it there in the first place.

Guess i tried that with:

    CMD="dd if=$SOURCE of=$TARGET bs=$DD_BS "
    $CMD &  > $TUI_TEMP_FILE > 2&>1

But doesnt work -- keep getting all info on each loop to stout, but not to the file

You are blanking the output file every time you kill, too. Remove that pile of redirections from kill.

Pure silence (fail)

dd if=/dev/random of=/dev/zero & 2&>1 > /tmp/tmp
pid=$(ps|grep dd|awk '{print $1}')
while [[ ! "" = "$(ps|grep dd)" ]]
do     #kill -USR1 $pid
    grep "," /tmp/tmp | grep -v "+"|awk '{$2" "$3}'
done

Full spam (fail):

dd if=/dev/random of=/dev/zero & 2&>1 > /tmp/tmp
pid=$(ps|grep dd|awk '{print $1}')
while [[ ! "" = "$(ps|grep dd)" ]]
do     kill -USR1 $pid
    grep "," /tmp/tmp | grep -v "+"|awk '{$2" "$3}'
done

Meaning:

0+5 records in
0+0 records out
0 bytes (0 B) copied0 bytes (0 B) copied, 20.7462 s, 0.0 kB/s
, 2.36932 s, 0.0 kB/s
0+5 records in
0+0 records out
0+0 records in
0+0 records out
0 bytes (0 B) copied, 2.38147 s, 0.0 kB/s
0 bytes (0 B) copied, 20.7584 s, 0.0 kB/s
0+20 records in
0+0 records out
0 bytes (0 B) copied, 49.2783 s, 0.0 kB/s

All i want to achieve is "NO 'plain-dd' output" as above, but only this part: "0 bytes (0 B) copied" so i can print it on 1 line, 'staying' that very 1 line to the user, without the console spammed.

As in $tmp contains only "20mb" -- "25mb" -- "50mb" ...
Once i could 'mute' the DD output to the user, but still retrieve the values, even with the 0+0 records, i could 'cut' it down to the value needed, but this redirection stuff drives me crazy!! :mad::confused:

One problem: dd if=/dev/random of=/dev/zero quits instantly on my system. /dev/zero isn't something you can write to.

Also: 2&>1 is garbled, you want 2>&1 .

1 Like
( dd if=/dev/zero of=/dev/null 2>&1 |
        awk -F"[()]" '/copied/ {print $2}' ) &

PID=$(pgrep -u $USER dd) # Get the PID, since we don't get it in $!
trap "kill $PID" EXIT # Stop the background proc on accidental quit

while ps $PID >/dev/null 2>/dev/null
do
        sleep 1
        kill -USR1 $PID
done

Sadly i get this on 'copy-paste':

...
> done
kill: usage: kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or kill -l [sigspec]
kill: usage: kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or kill -l [sigspec]
^C
130 /home/simon # 

It cant find the $PID the way you did, and even when applying my pid retrieval, the output is hidden :frowning:

EDIT: /* removed 3 appended posts as its working now.. */

This seems to be working: :slight_smile:

    dd if=$SOURCE of=$TARGET 2> $TUI_TEMP_FILE &
    PID=$(ps|grep -v tui|grep dd|awk '{print $1}')
    while [[ ! "" = "$(ps|grep -v tui|grep dd|awk '{print $1}')" ]]
    do     kill -USR1 $PID 2>&1
        SIZE=($(grep "," $TUI_TEMP_FILE|awk '{print $3$4}'))
        [[ ${#SIZE[@]} -eq 1 ]] && max=0 || max=$[ ${#SIZE[@]} - 1 ]
        tui-printf "Writing $SOURCE to $TARGET" "${SIZE[$max]}" "[  $(tui-indi)   ]"
        sleep 0.7
    done

Thanks for your help!

Just because i'm happy and bit proud of it, it looks now like this:
(had pressed enter to 'show progress')

Would actualy look like:

 ~ # ISO=/home/simon/net/dls/iso/sea-Awesome-WM.iso
+ ~ # tui-dd $ISO $(tui-str-usb)
# | Written sea-Awesome-WM.iso to /dev/sdb                                             [     ] | #
:) ~ # 

Thanks again :slight_smile:

/solved

You can use $! to get the PID of the last started background process.

You can also use jobs %1 to get the status of the job (testing it with $?), and kill %1 to kill it.

1 Like

I droped the $! as the example line failed yesterday, but Corona's script did run successfully (but wrote a new line each loop).
Today its vice-versa (still copy-pasted), weird things happen on/to my laptop...

Oh, sorry I missed that :slight_smile:

You guys probably knew already, but me just figured out how to get the return value of job running/ran in background.
For some weird reason it didnt want to work with an exported variable.

This one will write the exit code to a temp file, so it can be easy retrieved after the loop is done.

	(sh "$JOB" && \
			echo 0 > /dev/stdout > $TEMP_FILE || \
			echo 1 > /dev/stdout > $TEMP_FILE
	) & PID=$!

Where $JOB is a scriptfile containg the command to execute and $TEMP_FILE is a pre-set var (like: /tmp/tmp.ret)

Now to use that retvar in a variable again is easy:

JOB_RET=$(cat $TEMP_FILE)

Et voila :smiley: