However, if I try to run the command from a shell script under Ubuntu, it fails and outputs to stdout instead of the specified ramdisk-recovery.cpio file.
Thanks,
Well, I can do it easily using find . | cpio ...
However, I was a bit confused to not be able to use mkbootfs in a shell script while it works in cmd line
By the way, after further looking, mkbootfs is just a cpio variant, so should not be piped to cpio anyway
So, let's see it as a general question: is there any workaround to such binaries that once in a shell script, write to stdout? And why does the piping fails in this case?
Well, you weren't doing the same thing in terminal, you were doing command > file
In the script, you change it to command | cpio > file
Is cpio really needed here when it wasn't needed in the terminal? cpio reads filenames, but if mkbootfs doesn't print filenames, piping it into cpio makes no sense.
Well, stdout is the file, by definition, so it can't be writing to stdout... Try running nohup mkbootfs ... and seeing if the output ends up in nohup.out.
Without knowing the code of mkbootfs it's hard to say for sure -- it's not a standard command, I can't test it myself here. But it's possible for a program to tell if it has a terminal or not and act different accordingly.
That is very odd. If that really is the code, there's no reason at all for it to do that... I suspect your version ended up a little different somehow, but would need to use strace or something to tell what it's really doing.
The trick with nohup may also help narrow it down.
It's also possible there's more than one mkbootfs, and due to PATH differences your shell runs one and your script runs the other!
Show me the exact line that didn't work please -- unabridged, no substitutions, exactly as you had it. The entire script, even. There may be context missing, something before that line which changed the meaning of your redirection.
So, it seems the cause is putting the command line into a variable and trying to run it that way, which causes output to stdout
Should I have used exec or similar to do it that way?
Yes, "set -x" can be pretty useful. It can create a lot of output, which can be difficult to wade through. But for the most part "set -x" shows what is going on, what the shell "sees".
Shell scripting does not work this way. Once a string is inside a variable, it doesn't get reparsed for quotes, escapes, or substitutions anymore.
You'd have to use eval, which is a very bad idea for many reasons... One, it makes your code needlessly convoluted; adding substitution to your substitution is nesting layers of doublethink that are extremely difficult for anyone to program and understand. Two, it's very insecure and a horrible habit; if someone manages to put `rm -Rf ~/` into one of your variables, the shell will execute that!
As others have said, there's almost no good reasons to do this. It's a typical beginner mistake from not knowing the right kinds of substitution yet. You can kludge all gaps with eval since it can do anything at all and build footbridges out of toothpicks, but it'd be simpler and safer and faster to use the thoroughfares that are already there.
If I understood why you were putting it into a variable, I could probably show you how to do what you wanted without this problem.
And no, I don't mean how to store a complete shell statement in a variable. I mean, what you wanted to accomplish by storing it in a variable.
So, it seems the > was quoted into ' ' by the shell
I tried to escape the > using \> in my $CMD_LINE1, but it did not fix it
Just want to understand things. Is there any way to escape
the > and have the shell stop quoting it?
I don't know the answer. It's a good question you ask. However, I would suggest "just don't do it". And as pointed out by the other poster, there is the complex "eval" command that might work. Anyway, you observed:
$ cat test.sh
set -x
CMD='date > date.txt'
$CMD
$ ./test.sh
++ CMD='date > date.txt'
++ date '>' date.txt
date: extra operand `date.txt'
Try `date --help' for more information.
It's good you're curious. I am too. I tried several ways to escape the redirection. The shell always insisted on quoting it, as you observe.
It is quoting the redirection for a reason. I cannot fathom the reason. But someone a lot smarter than me wrote the shell and decided it was needed to quote that redirection.
The shell is a very complex beast. It's trying to do many things in a hostile environment. It's not perfect, has many design compromises, has evolved over many years. You have to work within it's limits, not overtax it.
If it were really helpful in this case to put the entire command line (including redirection) in a variable, and then run the variable, this would be worth trying to figure out. You're right to think perhaps the syntax "should" work in some way. But it seemingly doesn't.
@Corona688: no, I am not going to use eval of course, but like hanson44 said, I am very curious.
Why the command is in a variable?
Well, the script sample is part of a long complex script that automates compiling custom android recovery, extracting stock android recovery images, repacking compiled image with ramdisk from a different previously extracted stock image, do incremental backups of all the output, make and sign flashable zips and tar.md5 files if needed...
Basically, I enter something like:
philz_repack.sh 4.93.6 i9300
and it processes everything up to the final step
The script handles many different devices. Each device has a different hardware that needs to be handled...
The shell sample I gave was just an example of part of the repacking process for just one device.
In fact, every part of the line is assigned to variables for paths, device_id, kernel cmdline for the device... So, I populated the variables to make it easier to understand.
As I said, I worked around it using combination of cpio and gzip
Now, thanks to hanson44, I have another workaround by splitting that command line in 3 and putting the redirection > outside the variable
However, it would be great if some one can explain why the bash is quoting the redirection symbol '>' when inside a string like hanson44 showed it in his example!
I would also like to know. My best guess is that shell quotes the redirection in this context because the redirection symbol can also be used as a literal character, and it was decided the literal meaning should take precedence. They (the creators of shell) had to make a decision, and chose the path most generally useful. They perhaps ruled out putting an entire command line, including redirection, within a variable as being not needed. Look at the following which I think shows the advantage of quoting the redirection:
$ cat input
111>222>333
$ cat test.sh
set -x
cut -d ">" -f 2 input
CMD='cut -d > -f 2 input'; $CMD # works because of quoting
CMD='cut -d ">" -f 2 input'; $CMD # fails, but we have previous way
$ ./test.sh
++ cut -d '>' -f 2 input
222
++ CMD='cut -d > -f 2 input'
++ cut -d '>' -f 2 input
222
++ CMD='cut -d ">" -f 2 input'
++ cut -d '">"' -f 2 input
cut: the delimiter must be a single character
Try `cut --help' for more information.
yes, potential good explanation
It is really the first time I use such a way
In any case it was my initial debugging script. The final script goes the clean ways like you did, by splitting things in a command variable
Despite my issue was fixed since page 1, I leave this thread open if someone can add an input / explanation about the shell quoting redirection inside a variable
