I had a problem in a bash backup script caused by the following behavior that I don't understand.

First, we create a parameter holding a string with 2 consecutive spaces (right before '<'):

me@mysystem:~$ TEST="a string with 2 spaces right <--HERE!"
me@mysystem:~$ echo $TEST
a string with 2 spaces right <--HERE!

I can grep a space:

me@mysystem:~$ echo ${TEST} | grep " "
a string with 2 spaces right <--HERE!

I can grep a space followed by another letter:

me@mysystem:~$ echo ${TEST} | grep " s"
a string with 2 spaces right <--HERE!

I can grep the part after the 2 spaces:

me@mysystem:~$ echo ${TEST} | grep "HERE"
a string with 2 spaces right <--HERE!

I can do a variable expansion replacing both spaces with an underscore:

me@mysystem:~$ echo ${TEST// /_}
a_string_with_2_spaces_right__<--HERE!

But grepping the double space requires quoting the variable:

me@mysystem:~$ echo $TEST | grep " "; echo "Done."
Done.
me@mysystem:~$ echo ${TEST} | grep " "; echo "Done."
Done.
me@mysystem:~$ echo "$TEST" | grep " "; echo "Done."
a string with 2 spaces right <--HERE!
Done.
me@mysystem:~$ echo "${TEST}" | grep " "; echo "Done."
a string with 2 spaces right <--HERE!
Done.

I don't understand why this is the case.

Long story short: I had an if statement in the script checking whether ${FILE} represents an existing file, and that apparently also stops interpreting the variable when encountering a space unless quoted. So I'm not sure anymore what ${} does or doesn't actually take care off.

It's best to always quote the variable I guess?

1 Answer

This is a classic case of word splitting, done by shell, on unquoted variable expansion.

Basically when you do, $TEST (without quotes ("$TEST")), the shell:

• Performs word splitting, based on the value of IFS environment variable (default: space, tab, newline), and splits the whole expansion into separate words

• so as you have two consecutive spaces, the word splitting (and also pathname (glob) expansion if any of *, ?, [] is present) is done, and when the output string is rebuilt the words are joined by space, as a result you have only one space between right and <--HERE!. This would be true for any combination of IFS characters.

In a nutshell, quote your variable expansions:

"${variable}"

Your Answer

Sign up or log in

Sign up using Google Sign up using Facebook Sign up using Email and Password Submit

Post as a guest

Post Your Answer Discard

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy