For loop without checking file exists

Shell Programming and Scripting
1

In several scripts that process files matched by name pattern I needed to add a check for file existence. Just to illustrate let's say I need to process all N??? files:

/tmp$ touch N100 N101
/tmp$ l ?10[01]
-rw-rw-r--   1 moss     group          0 Apr 19 11:22 N100
-rw-rw-r--   1 moss     group          0 Apr 19 11:22 N101
/tmp$ for fn in N???; do echo $fn; ls -l $fn; done
N100
-rw-rw-r--   1 moss     group          0 Apr 19 11:22 N100
N101
-rw-rw-r--   1 moss     group          0 Apr 19 11:22 N101

If there is no N files, this for loop would try to execute ls with non-existent pattern:

/tmp$ rm N???
/tmp$ for fn in N???; do echo $fn; ls -l $fn; done              
N???
ls: N??? not found: No such file or directory (error 2)

To avoid running a process against non-existent file I add if file exists check:

/tmp$ for fn in N???; do if [ -f $fn ]; then ls -l $fn; fi; done
/tmp$

Is there any way in shell to not use those file exists checks and still run process only for an exisant file?

2

Hello migurus,

Could you please try following and let me know this helps you.

for fn in N???; do if [[ -f $fn ]]; then echo $fn; ls -l $fn; fi; done

Above will not print anything errors on standard output. Although you could simply use following too.

for fn in N???; do echo $fn; ls -l $fn; done 2> /dev/null

But only thing above will print value N??? even it is not present but no error will be printed on standard output, so in case you don't want to print filename then you could directly use as follows.

for fn in N???; do ls -l $fn; done 2> /dev/null

I hope this helps you.

Thanks,
R. Singh

3

How about

ls N??? 2>/dev/null | while read fn; do echo "$fn"; ls -l "$fn"; done
4

I guess I don't understand why you need both echo "$fn" and ls -l "$fn" ???
Doesn't just using:

ls -l N??? 2>/dev/null

convey the same information?

1 Like
5

I added the echo in the loop to show that when nothing matches the N??? pattern the fn variable assumes value of N??? and that is the root of the question - when using

for fname in some_pattern

construct I thought the pattern evaluation happens before do and shell would simply not enter the do loop part when nothing matches.

1 Like
6

Some, shells have an option (as an extension to the standards) that does remove globbing patterns that do not match any existing files. For instance, if you are using a recent bash , the code:

shopt -s nullglob
for i in N???
do	printf '%s\n' "$i"
done

would not run the loop at all if N??? does not match any files in the current working directory, while the other options work with any shell based on Bourne shell syntax:

for i in N???
do	if [ "$i" = "N???" ]
	then	break
	fi
	printf '%s\n' "$i"
done

which works fine as long as you know that there will never be a file actually named N??? . (Note that the break could be replaced by a continue and get the same results.) Or, you can use:

for i in N???
do	if [ -e "$i" ]
	then	continue
	fi
	printf '%s\n' "$i"
done

(Note that you need to use continue in this form (not break ) to protect yourself in case a file is removed while the loop is running (you just want to skip processing the file that was removed; not skip processing other remaining files with names that sort after the file that was removed).

2 Likes
7

Thank you, good to know about nullglob, although the target system uses ksh and shopt is not an option.

8

A general test without file exists checks to see if there are still glob characters in the filename, which should also work, but also only if there are no file names with actual glob characters:

for i in N???
do 
  if [ "${i%[?*]*}" = "$i" ]; then
    printf "%\n" "$i"
  done
done

or

for i in N???
do 
    case $i in (*[?*]*)
      break
    esac
    printf "%\n" "$i"
done
9

I think in ksh it is

set -o noglob

The disadvantage of this (and of bash's nullglob) is that the option is turned on until you turn it off explicitly. I suggest that you switch to zsh, where you can define this behaviour for each pattern separately. In zsh, the loop would be written as

    for fn in N???(N)
    do
        ....
    done

and the body would be skipped if there is no file matching the pattern; no error message here. While zsh differs in several ways from ksh/bash/dash, I find it much more convenient for shell programming than the other ones.

2 Likes
10

Just checked on zsh, thanks. I will keep this shell in mind

11

No. set -o noglob is a synonym for set -f in ksh . Both of which disable globing; they are not equivalent to the bash shopt -s nullglob .

If a directory contains files named x.txt and y.txt , the commands:

set +f	# default initial state
for i in *.txt
do	printf '%s\n' "$i"
done

produces the output:

x.txt
y.txt

and the commands:

set -f	# turn off globing
for i in *.txt
do	printf '%s\n' "$i"
done

produces the output:

*.txt

The set -f and set +f commands are required by the standards and are available in bash , ksh , and zsh .

3 Likes