Parameter expansion `${parameter#word}` not working as expected

Question

I am currently trying to understand parameter expansion, especially the different forms which can remove parts of a parameter's value by pattern matching. For the sake of this question, let's focus on the ${parameter#word} expansion.

The two relevant sections from the manual (man bash) (emphasis mine):

${parameter#word}
${parameter##word}
Remove matching prefix pattern. The word is expanded to produce a pattern just as in pathname expansion, and matched against the expanded value of parameter using the rules described under Pattern Matching below. If the pattern matches the beginning of the value of parameter, then the result of the expansion is the expanded value of parameter with the shortest matching pattern (the '#' case) or the longest matching pattern (the '##' case) deleted. [...]

Pathname Expansion
After word splitting, unless the -f option has been set, bash scans each word for the characters *, ?, and [. If one of these characters appears, then the word is regarded as a pattern, and replaced with an alphabetically sorted list of filenames matching the pattern (see Pattern Matching below). [...]

However, please consider the following part of a terminal session:

root@cerberus ~/scripts # mkdir test
root@cerberus ~/scripts # cd test
root@cerberus ~/scripts/test # touch foo
root@cerberus ~/scripts/test # String=foobar
root@cerberus ~/scripts/test # printf '%s\n' ${String#foo}
bar
root@cerberus ~/scripts/test # printf '%s\n' ${String#*}
foobar
root@cerberus ~/scripts/test #

I am not able to understand the output after the last command. According to the manual, the * should be expanded to foo, because foo is the only file in the current directory; at least, this is my notion of "produce a pattern just as in pathname expansion".

Therefore, in this case, printf '%s\n' ${String#*} should give the same output as printf '%s\n' ${String#foo}.

Obviously, this is not the case. Where is my misunderstanding?

Answer 1

Regarding pattern matching:

* Matches any string, including the null string.

And, in parameter expansion for substring removal, # does the shortest match, while ## the longest.

This is removing the null string (shortest match) from the beginning:

$ printf '%s\n' "${String#*}"
foobar

And this is removing foobar (longest match) from the beginning:

$ printf '%s\n' "${String##*}"

$

---

Regarding the last paragraph of your description: Substring removal with parameter expansion has nothing to do with what files are in your current directory. The pattern is matching on the parameter value, not any files, think of it as a text processing operation on the parameter's value.

Answer 2

It take the "word" part as a pattern like the ones used for filename generation, but it doesn't use it for filename generation. Instead, the pattern there is used to match against the value of the given parameter.

E.g. if you have str=foobar, then ${str#*o} would try the pattern *o at the start of foobar, find fo as the shortest match (the * matches any number of any character), remove that and expand to obar. Similarly ${str##*o} would find the longest match, foo, remove it at expand bar.

$ str=foobar
$ echo ${str#*o}
obar
$ echo ${str##*o}
bar

The reason it says "word is expanded", is that you can use expansions in the "word" part, like so:

$ str=foobar
$ char=o
$ echo ${str#*$char}
obar

or

$ pat="*o"
$ echo ${str#$pat}
obar

Then again, with the expansion unquoted as above, the result after the prefix removal would go through word splitting and globbing. You should probably quote the whole expansion to prevent that, i.e. "${str#*o}" etc.