Problem with shell script while spaces encountered in directory names

UNIX for Advanced & Expert Users
1

Hi,
I am having issues with the jar -tf command when I put in the shell script.

The command runs fine from the command line as shown below.

[root@quickstart Documents]# jar -tf "./VirtualBox Dropped Files/2016-04-17T20:58:49.129139000Z/hive-exec-0.8.1.jar"

But when I put in a shell script(shown below) and the directory name has spaces.In this case "Virtual Box Dropped" has spaces.It gives an error as shown below.

----test1.sh code------

#!/bin/bash
for jar in `find . -iname '*.jar'`;do
 jar -tf "$jar" > cls.dat
done

[root@quickstart Documents]# ./test1.sh localrunner

Error:

java.io.FileNotFoundException: ./VirtualBox (No such file or directory)
    at java.util.zip.ZipFile.open(Native Method)
    at java.util.zip.ZipFile.<init>(ZipFile.java:215)
    at java.util.zip.ZipFile.<init>(ZipFile.java:145)
    at java.util.zip.ZipFile.<init>(ZipFile.java:116)
    at sun.tools.jar.Main.list(Main.java:1004)
    at sun.tools.jar.Main.run(Main.java:245)
    at sun.tools.jar.Main.main(Main.java:1177)
java.io.FileNotFoundException: Dropped (No such file or directory)
    at java.util.zip.ZipFile.open(Native Method)
    at java.util.zip.ZipFile.<init>(ZipFile.java:215)
    at java.util.zip.ZipFile.<init>(ZipFile.java:145)
    at java.util.zip.ZipFile.<init>(ZipFile.java:116)
    at sun.tools.jar.Main.list(Main.java:1004)
    at sun.tools.jar.Main.run(Main.java:245)
    at sun.tools.jar.Main.main(Main.java:1177)
java.io.FileNotFoundException: Files/2016-04-17T20:58:49.129139000Z/hive-exec-0.8.1.jar (No such file or directory)
    at java.util.zip.ZipFile.open(Native Method)
    at java.util.zip.ZipFile.<init>(ZipFile.java:215)
    at java.util.zip.ZipFile.<init>(ZipFile.java:145)
    at java.util.zip.ZipFile.<init>(ZipFile.java:116)
    at sun.tools.jar.Main.list(Main.java:1004)
    at sun.tools.jar.Main.run(Main.java:245)
    at sun.tools.jar.Main.main(Main.java:1177)

I tried to put the double quote but it doesnt work?

Any ideas?

Thanks

2

Try enclosing the "command substitution" in double quotes.

3

Hello vinoo128,

Could you please try changing your code to as follows and let me know if this helps you.

 #!/bin/bash
for jar in *.jar
 do
   jar -tf "$jar" > cls.dat
done

because anyways find is looking for the current directories files only. Let me know how it goes for you.

Thanks,
R. Singh

4

This will not help with the quoted file having spaces in its name ("VirtualBox Dropped Files"). Enclose *.jar in double quotes.

You have to take extra measures to confine find to the uppermost directory.

1 Like
5

Why use a shell loop? Try with find's -exec

find . -iname '*.jar' -exec jar -tf {} \; > cls.dat

---

Actually it does, because Ravinder used double quotes around the variable expansion "$jar" , so spaces will be preserved and *.jar must not be put in double quotes, because the glob expansion will not work then.
Ravinder's approach is correct, but it only works for the current directory...

It even works with multi-line file names:

$ for i in ./a\ * ; do printf "file:%s:end\n" "$i"; done
file:./a  t:end
file:./a  y:end
file:./a b:end
file:./a multiline file
name.out.txt:end
file:./a z:end

compare with find:

$ find . -name "a *" -exec printf "file:%s:end\n" {} \;  
file:./a  t:end
file:./a  y:end
file:./a b:end
file:./a multiline file
name.out.txt:end
file:./a z:end
file:./fromdir/a  hg hg:end
2 Likes
6

Rats, yes. Apologies, RavinderSingh13!

7

First: `man find` shows there are options for "UNUSUAL FILENAMES" (e.g. filenames with spaces).

      -print0
              True;  print the full file name on the standard output, followed by a null character (instead of the newline char
              acter that -print uses).  This allows file names that contain newlines or other types of white space  to  be  cor
              rectly interpreted by programs that process the find output.  This option corresponds to the -0 option of xargs.

Second: using a "for loop" may be dangerous, there can be more results than a Unix line can handle. I would prefer a while loop.

find . -iname '*.jar' -print0 |while read jar
do
  jar -tf "$jar" >> cls.dat
done

Third: Scrutinizer already mentioned to use the -exec option of the `find` command. To my opinion this is the best method.

8

This working depends on the find -command you have supporting it. Actually -print0 is a GNU-extension and, for instance, the find on AIX doesn't know it.

To the rest of your sentiments, though, i can wholeheartedly agree.

I hope this helps.

bakunin

9

Hi Ivo, this is not correct, there are no line length limitations to a for loop, which is part of the shell syntax, so the limitations of passing parameters to a subprocess do not apply.

The problem with the construct in post #1 is that the result of the command substitution ( `...` ) is vulnerable to interpretation by the shell, e.g. field splitting, so for example files with spaces will be split in two or more arguments and these arguments typically do not exist and so this will fail, hence the (No such file or directory) messages..

10

Yes - and No: the maximum number of passed arguments will not apply but the maximum line length will. Suppose you have a line like:

for x in * [; do ....]

and you execute that in a directory with really many files, it will first be expanded by the shell to:

for x in fileA fileB fileC [...]

and if the really many filenames make this line too long for the shells input parser to fathom you will be in deep kimchi. I give you that: some shells don't care for system constants like "LINE_MAX", but as far as i know a POSIXly correct utility has only to be able to digest lines up to that length. Above that all bets are off. So, if you want to stay on the safe side you better do not require something that is optional in the standards.

From /usr/include/sys/limits.h on a rather up-to-date AIX 7.1-system:

/* The system supports the following utility limit minimum values */

#define _POSIX2_BC_BASE_MAX     99   /* max ibase and obase values for bc     */
#define _POSIX2_BC_DIM_MAX      2048 /* max num elements in array for bc      */
#define _POSIX2_BC_SCALE_MAX    99   /* max scale value allowed by bc         */
#define _POSIX2_BC_STRING_MAX   1000 /* max len of string constant by bc      */
#define _POSIX2_EQUIV_CLASS_MAX 2    /* this define is obsolete               */
#define _POSIX2_COLL_WEIGHTS_MAX 2   /* max num weight for LC_COLLATE order   */
#define _POSIX2_EXPR_NEST_MAX   32   /* max num expression nested for expr    */
#define _POSIX2_LINE_MAX        2048 /* max len of utility input line         */
#define _POSIX2_RE_DUP_MAX      255  /* max num repeated reg for interval not */

I hope this helps.

bakunin

11

Yes, the shell expands the wild card, but it does not somehow pass the expanded result as an input line to itself, it does not pass anything. It would only do that if it would pass it to another process, but it does not do that here.

A for loop is not a utility! It is part of the shell syntax, it is not even a builtin utility. So LINE_MAX does not play a role either!

--
Also check this post, by alister on this subject: