In bash: trailing space capture in variable substitution

Question

I see this in BASH 4.3.48 (SLES12 SP4) and BASH 4.4.23 (OpenSUSE Leap 15.1) when trying to remove multiple trailing spaces from a variable's value:

~> xxx="-O -Wall  "
~> echo "X${xxx%% }X"    # (1)
X-O -Wall X
~> echo "X${xxx%% *}X"
X-OX
~> echo "X${xxx% }X"
X-O -Wall X
~> echo "X${xxx% *}X"    # (2)
X-O -Wall X
~> echo "X${xxx%% \*}X"
X-O -Wall  X

I feel that either (1) or (2) should do the job.

The manual states for ${parameter%%word}:

Remove matching suffix pattern. The word is expanded to produce a pattern just as in pathname expansion. If the pattern matches a trailing portion of the expanded value of parameter, then the result of the expansion is the expanded value of parameter with the shortest matching pattern (the ``%'' case) or the longest matching pattern (the ``%%'' case) deleted.

As it doesn't work as documented (or as I understand the documentation), I suspect this is a bug (non matching suffix ("-Wall") is being removed in case of "%% *") in BASH. Am I right?

Answer 1

In echo "X${xxx%% }X", the pattern is a single space: . The longest matching portion for that is just that: a single space. The shortest matching portion is also just that: a single space.

For anything more, you need the globbing operator *. But that will match anything, removing the -Wall. Bash globbing doesn't support directly have an equivalent of the regular expression a*. You'd need extended globbing:

$ shopt -s extglob
$ echo "X${xxx%%+( )}X"
X-O -WallX

Answer 2

Use a removal of a prefix within a suffix removal:

$ xxx="-O -Wall  "
$ echo "X${xxx%"${xxx##*[! ]}"}X"
X-O -WallX

• Remove everything up to the last not-space character - leaving only trailing spaces
• Use those spaces as the pattern for suffix removal
• The inner parameter expansion should be quoted to prevent it from being interpreted as a pattern (not necessary above, but may be useful in other cases):

$ bash -c 'xxx="-O -Wall*   "; echo "X${xxx%%"${xxx##*[! *]}"}X"'
X-O -WallX
$ bash -c 'xxx="-O -Wall*   "; echo "X${xxx%%${xxx##*[! *]}}X"'
XX

A contrived example, but if the inner expansion is not quoted, the asterisk it includes will be treated as a shell pattern by the outer expansion. Quoted, it becomes a literal asterisk.

---

The behaviour you observed is not a bug, it's just how simple shell patterns work:

${xxx%% }

• a single space is a single space
• longest occurence of a single space is a single space

${xxx%% *}

• longest occurence of a single space followed by anything/nothing
• anything/nothing will include -Wall

${xxx% }

• shortest occurence of a single space is a single space

${xxx% *}

• shortest occurence of a single space followed by anything/nothing is a single space

${xxx%% \*}

• \* is a backslash escaped asterisk and will be interpreted as a literal asterisk
• there is no space followed by asterisk in the variable, no suffix is removed

Answer 3

read may also work (assuming IFS contains "space"):

xxx="-O -Wall  "
read -r xxx <<EOF
$xxx
EOF
echo "X${xxx}X"

Output:

X-O -WallX

---

• read splits input into fields according to IFS
• IFS by default is space/tab/newline, so this will remove any leading & trailing spaces
• Works on the first line of the variable (may not be suitable for multiline vars, bash could use read -d '')

Answer 4

It can be done with regular expression matching like this:

~> xxx="-O -Wall  "
~> echo "X${xxx}X"
X-O -Wall  X
~> [[ "$xxx" =~ (\ +$) ]]; echo $?,"X${BASH_REMATCH[0]}X"
0,X  X
~> echo "X${xxx%%${BASH_REMATCH[0]}}X"
X-O -WallX

~/src/Perl> xxx="-O -Wall"
~/src/Perl> echo "X${xxx}X"
X-O -WallX
~/src/Perl> [[ "$xxx" =~ (\ +$) ]]; echo $?,"X${BASH_REMATCH[0]}X"
1,XX
~/src/Perl> echo "X${xxx%%${BASH_REMATCH[0]}}X"
X-O -WallX

Answer 5

A simple parameter expansion is quite limited in the patterns it could match and remove. To remove several (repeated) characters form the end of an string, the usual solution is to actually first remove everything that is not the character in question ${xxx##*[! ]} (all the trailing spaces). Then, as a second step, removing everything that results from that expansion (all the trailing spaces) from the end will give you what you want (remove trailing spaces).

$ xxx="-O -Wall  "
$ echo "<${xxx%"${xxx##*[! ]}"}>"
<-O -Wall>

As an alternative, in bash, you could use extended globbing:

$ shopt -s extglob
$ echo "<${xxx%%+( )}>"
<-O -Wall>

Or, also, as a higher level alternative, you can match what you want with a regex:

$ regex='(.*[^ ]) +$';
$ [[ $xxx =~ $regex ]] && echo "<${BASH_REMATCH[1]}>" || echo "<$xxx>"
<-O -Wall>

Or, as an script:

#!/bin/bash

xxx=${1:-"-O -Wall  "}

regex='(.*[^ ]) +$'

if    [[ $xxx =~ $regex ]]          # if there are trailing spaces
then 
      echo "<${BASH_REMATCH[1]}>"   # Print the string without spaces
else
      echo "<$xxx>"                 # if there are no trailing spaces.
fi