Shell quoting rules for other languages programmers

I've seen so many times that programmers were confused about shell quoting and white spaces interpretation, that I decided to investigate that problem deeper. And my conclusion is that quoting in shells is very different from other programming languages. Programmers who have bigger experience in C/Python/Java/PHP than in Shell should remember about two unintuitive rules:

  • everything in shell is a string, and one need to quote everything properly. Also when reading variables: "$X", not only storing X="hi there"
  • shell will parse lines of code just one time, leaving unresolved nested variables and commands. In this case programmer need to explicitly use "eval" function.

I've written short article about this if anyone will be more curious you can find it under "white shell cofoh"

1 Like

I favor the use of the single quote for fastest and most certain processing, as it expands nothing. You can switch to double wuotes where you want expansion or a literal single quote, and then go back to single quoting. Since there is no expansion, there is less processing. For file wild card expansion, or globbing, you need to be outside quotes. I rather like a pipe: "echo '....' | command" over a here file: "command <<EOD
....
EOD" as I can predict and view expansions easily.

The eval I save for odd situations, like constructing a variable name from a varible and then fetching it. It is a shell built-in, so probably faster than `sh -c 'echo $x'"$m"`

You are right, globbing is not expanded in any quoting. And single quotes
suppose to be faster than double quotes. But is it big difference
in performance?

Before I've investigated that problem I was thinking that eval is for odd
situation, like you. But now it looks to me like we should use it quite often!
For example, can anybody tell me how to do last line without eval:

USER_A=tom
USER_B=geoff
USER_C=anna
CURRENT='$USER_B' # without single quotes it would be just value copy
eval echo $CURRENT

The first rule about using eval is "don't do that". If you find yourself using eval, you've probably taken a wrong turn somewhere -- your program likely could've been solved in ways which don't need it, usually by representing data in a different way. eval just seems the most obvious since it's a big enough hammer to drive the most stubborn nails, even those ones with strange sharp spirals down their sides (screws, better driven with a screwdriver).

So, how would I do that in shell? I wouldn't. I'd use double-quotes and store the variable contents in the first place -- eval is almost certainly slower than double quotes!

When I have just few screws (few lines of code) and my bed is falling apart without it (software crushes) and those screws have that strange head shape that non of my screwdriver matches (pure shell syntax) I will use hammer if I want to go sleep :wink:

But seriously using eval cannot be so harmful...It's just re-evaluation of strings.

Yes, I know what you mean. I remember when I was starting with Linux, I didn't understand shell very well, and tried to write most of everything in Perl. This didn't make bash stupid, or eval necessary, I simply didn't know what I was doing.

Yes. Yes, it can.

Imagine that you're feeding variables containing filenames into eval. Did you know that `rm -Rf ~/` is a valid filename?

Using eval lets external things inject whatever code they want into your program unless done very very carefully. If you don't know enough to avoid eval, you probably don't know how to use it safely either.

Well, avoiding injection attacks and massive mistakes, with or without eval, often revolves around using quotes corrrectly and expecting all sorts of poorly behaved input data.

Rather than storing data in multiple variables with intelligence in the name, use the array. I read that in bash, simple arrays are actually implemented in a linked list, so adjust your performance expectation for big sets downward accordingly. If numeric keys and simple arrays are inconvenient, use the array-like hash container types that take string keys, in bash called associtive arrays. Hash (random) access scales much better. You have to go to PERL and up to get real arrays, other sorts of linked lists and trees as well as hash mapped containters.