Bash IFS

I am using bash and resetting IFS as below when reading the command line arguments. I do this so I can call my script as in Ex1.

Ex1: ./synt2d-ray3dmod.bash --xsrc=12/20/30
 

This allows me to split both sides so that when I do "shift"
I can get 12/20/30

What I do not understand is how calling Ex2 works also.

Ex2: ./synt2d-ray3dmod.bash --xsrc 12/20/30

OLDIFS="$IFS"
IFS="|="                # IFS controls splitting. Split on "|" and "="
set -- $*               # Set the positional parameters to the command line arguments.
IFS="$OLDIFS"

narg="$#"

while [ "$#" -gt 0 ]
do

  case "$1" in

  # name of output segy file
  "--ofl")
    shift
    val_ofl="${1}"
    hasArg_ofl="true"
  ;;

  # number ot time samples
  "--nt")
    shift
    val_nt="${1}"
    hasArg_nt="true"
  ;;

  # x-positions of sources
  "--xsrc")
    shift
    val_xSrc="${1}"
    hasArg_xSrc="true"
  ;;

  *)
    value_errLst="$arg_errLst ${1}"
    hasArg_errLst="true"
  ;;

  esac

  shift                 # Skip ahead to the next argument

done

IFS works both ways. The shell splits strings apart on IFS, and when you put together a string with $*, it also puts them back together with IFS.

So, when you do ./myscript --xsrc 1234 , $* actually turns it into --xsrc|1234 . You just don't see it because you're letting the shell split it right back apart afterwards.

To prevent the shell from splitting something, you put it in quotes, so try this:

OLDIFS="$IFS"
IFS="|="                # IFS controls splitting. Split on "|" and "="
echo "$*"
set -- $*               # Set the positional parameters to the command line arg$
IFS="$OLDIFS"
$ ./script --arg=1234 # One argument

--arg=1234

$ ./script --arg 1234 # Two arguments

--arg|1234

$

So you see, the first time it comes in as 1 argument and nothing gets inserted by $*.

The second time it comes as two arguments, and gets stuck together into one string with | between.

IFS splits on both | and =, so set -- $* splits it apart where appropriate either way.

This is not a "BASH thing", by the way. Any Bourne shell does this.

There is another thing I am unsure about. It is when I am resetting IFS. My dilemma is what happens when I use the code I attached and the following

OLDIFS="$IFS"
IFS="|="                # IFS controls splitting. Split on "|" and "="
set -- $*               # Set the positional parameters to the command line arguments.

narg="$#"

while [ "$#" -gt 0 ]
do

  case "$1" in

  # name of output segy file
  "--ofl")
    shift
    val_ofl="${1}"
    hasArg_ofl="true"
  ;;

  # number ot time samples
  "--nt")
    shift
    val_nt="${1}"
    hasArg_nt="true"
  ;;

  # x-positions of sources
  "--xsrc")
    shift
    val_xSrc="${1}"
    hasArg_xSrc="true"
  ;;

  *)
    value_errLst="$arg_errLst ${1}"
    hasArg_errLst="true"
  ;;

  esac

  shift                 # Skip ahead to the next argument

done

IFS="$OLDIFS"

---------- Post updated at 02:06 PM ---------- Previous update was at 02:01 PM ----------

Originally my thinking was to reset IFS after the while (after processing all arguments). However, doing

OLDIFS="$IFS"
IFS="|="                # IFS controls splitting. Split on "|" and "="
set -- $*               # Set the positional parameters to the command line arg$
IFS="$OLDIFS"

while [ "$#" -gt 0 ]
do

...

does not seem to pose problems. What's happening?

Changing IFS won't change splitting that's already happened. All the splitting for arguments happens in the third line, and gets saved into $1 $2 etc. The content and order of the $1 $2 ... variables won't change unless you do another set --.

I'd put the fourth line right below the first three just so you don't forget and try and split something later, and get weird results.

1 Like

Also, doing echo $* after set removes all = and replaces them with |

echo $*
set -- $*
echo $*

results is

./synt2d-ray3dmod.bash --xsrc=12/20/30 --verbose=3 --ofl=test.sgy

--xsrc=12/20/30|--verbose=3|--ofl=test.sgy
--xsrc|12/20/30|--verbose|3|--ofl|test.sgy

That's not the code you ran. You must have been running echo "$*" quotes and all, otherwise you'd have never seen the pipes.

$* after the set didn't remove the equals, it was already gone. If you ran ./script a b --arg=c , $* on line 3 wopuld be the string a|b|--arg=c and split apart on "|=" into the arguments "a" , "b" , "--arg" , and "c" The equals sign is already gone, deleted.

So when you do $* after that, it just smashes $1 $2 ... all together with | inbetween: a|b|--arg|c

Now remember, $* always, always does this, even if you don't quote it. :slight_smile: You just don't see it without quotes, because the shell splits it. It'd see a|b|--arg|c, give "a" as echo's first argument, "b" as echo's second argument, "--arg" as echo's third argument, and "c" as echo's fourth argument. (echo doesn't split the arguments; that's the shell's job.) echo is not controlled by IFS, and will just put spaces inbetween...

Yes I ran echo "$*". Quite complicated this IFS!

---------- Post updated at 03:03 PM ---------- Previous update was at 02:47 PM ----------

So that means that the delimiter is always | after splitting? Splitting just splits according to either | or =, then arguments are delimited by |. Or I am messing things up at this point, and the only thing that matters is that arguments are now in $1, $2, ...? Why does it smash everything with |. Is that standard or is it because I used | as the first character in IFS? I am sure that if I use IFS="F=", instead of having |, I will have 'F' being displayed when the arguments are smashed together.

It's actually quite simple, but works very differently from how people first imagine it :slight_smile:

Try this.

$ VAR="a b c d"
$ printf "%s\n" 1 2 3 # Each argument on a different line

1
2
3

$ printf "%s\n" "$VAR" # IFS does not split here

a b c d

$ printf "%s\n" $VAR # IFS splits here

a
b
c
d

$

You see how the shell splits $VAR into 4 different arguments, but keeps "$VAR" as one argument.

Now try this:

$ set -- a b c d
$ printf "%s\n" $*

a
b
c
d

$ printf "%s\n" "$*" # IFS is space, so $1 $2 ... get spaces between

a b c d

$ IFS="|"
$ printf "%s\n" "$*" # With IFS="|", it puts pipes instead

a|b|c|d

$ printf "%s\n" $* # ...but if a|b|c|d isn't quoted, will split it into arguments.

a
b
c
d

$

Yes, exactly. IFS does not control what $1 $2 ... do -- it controls what an unquoted $ does. So we split once, setting the $1 $2 ... variables to exactly what we want, and can expect them to remain that way no matter what we do to IFS afterwards.

It's because it's the first character in IFS.

By default, that is space, so if you didn't change IFS, $* would give you "arg1 arg2 arg3 arg4" instead of "arg1|arg2|arg3|arg4".

See what I mean about it not being what you expected? You expected that $* gives you the same arguments you passed into the script. It never did, it always squashes $1 $2 ... together -- it just looked like your arguments because IFS defaults to space. (Well, space and tab and newline, but you get the idea.)

Yes, exactly. I picked | because it's unlikely to be used in your arguments, but could have as easily used 'e', '@', '^', or any other ASCII character. Don't use * or ?, since the shell will try to glob on those. (like what happens when you do ls *.jpg )

It's important that it not be in your input, because IFS="F=" would split "--arg=Fast" into $1="--arg", and $2="ast" !

1 Like

I understand now and making sense of it all. Thanks again for all the information and examples. :slight_smile:

---------- Post updated at 05:16 PM ---------- Previous update was at 03:25 PM ----------

I am now trying to update my script to use default values if the user does not supply values as command line arguments. Getting some problems when splitting for the individual values.

Here is the bash sequence

echo ""
IFS=$'/'
${val_xSrc:='-1.0/0/1.0'} # If val_xSrc is null or unset then it will be set to "-1.0/0/1.0".
unset IFS
echo "Using new method"
echo "val_xSrc = $val_xSrc"
xSrc1=${val_xSrc[$1]}  # default value 
xSrc2=${val_xSrc[$2]}  # default value
dxSrc=${val_xSrc[$3]}  # default value
echo "xsrc1 = $xSrc1"
echo "xsrc2 = $xSrc2"
echo "dxsrc = $dxSrc"
xSrcLen=${#val_xSrc[@]}
for (( i=0; i<${xSrcLen}; i++ ));
do
  echo "$i = ${val_xSrc[$i]}"
done
echo ""

Using arrays in shell often makes a problem harder. It looks like you've been struggling, but it's simple enough the traditional way... Just read into the variables you want and prefix it with IFS to let it split on what you want. Three things in one shot and no arrays. And it'll work in any shell, not just BASH.

# If variable is blank, set it
[ -z "$val_xSrc" ] && val_xSrc="-1.0/0/1.0"

# Prefixing a variable to a command sets that variable for only that line
IFS="/" read xSrc1 xSrc2 dxSrc <<EOF
${val_xSrc}
EOF

echo "xSrc1 = $xSrc1"
echo "xSrc2 = $xSrc2"
echo "dxSrc = $dxSrc"

I managed to do as follows. The problem was that -1.0/0/1.0 should be unquoted.

IFS="/"; ary=(${val_xSrc:=-1.0/0/1.0}); unset IFS
xSrc1=${ary[0]}; xSrc2=${ary[1]}; dxSrc=${ary[2]}

You shouldn't be unsetting IFS, that could cause weird things later.

I'd reccomend doing things the way I showed you, instead.

However I would keep the one described below, yes?

IFS="|="         
set -- $*
unset IFS

---------- Post updated at 06:51 PM ---------- Previous update was at 06:43 PM ----------

I always thought that if I set a different IFS, I would need to unset it, so it keeps its shell default. Is that not so?

---------- Post updated at 06:56 PM ---------- Previous update was at 06:51 PM ----------

I can't quite understand what you mean by your second comment

# Prefixing a variable to a command sets that variable for only that line

Do you mean that the change in IFS is only temporary when filling

xSrc1 xSrc2 dxSrc

but continues to be the bash default for the rest of the script?

What about the original code you had, where you did OLDIFS="$IFS" to put it back later? You want IFS to be default. You don't want it to actually be blank.

You want IFS to be default. You don't want it to actually be blank.

Yes. Putting X=Y before a command sets that variable only for that line, and leaves it unchanged in the rest. It doesn't work everywhere, but it works for any external commands and also for the read builtin.

I thought reset IFS puts the default and not blank.

It does not.

This might work better.

IFS="/" read xSrc1 xSrc2 dxSrc <<< "${val_xSrc}"

If IFS is unset, or its value is exactly <space><tab><newline>, the default, then sequences of <space>, <tab>, and <newline> at the beginning and end of the results of the previous expansions are ignored, and any sequence of IFS characters not at the beginning or end serves to delimit words.


mute@flo-rida:~$ echo $'[ space\t tab\nnewline]' | { IFS=/ read -rd '' lb s t n rb; echo "$lb:$s:$t:$n:$rb"; }
[ space  tab
newline]
::::
mute@flo-rida:~$ echo $'[ space\t tab\nnewline]' | {  read -rd '' lb s t n rb; echo "$lb:$s:$t:$n:$rb"; }
[:space:tab:newline]:
mute@flo-rida:~$ unset IFS
mute@flo-rida:~$ echo $'[ space\t tab\nnewline]' | {  read -rd '' lb s t n rb; echo "$lb:$s:$t:$n:$rb"; }
[:space:tab:newline]: