Shell Expansion

Commands and arguments

This chapter introduces you to shell expansion by taking a close look at commands and arguments. Knowing shell expansion is important because many commands on your Linux system are processed and most likely changed by the shell before they are executed.

The command line interface or shell used on most Linux systems is called bash/bin/bash, which stands for Bourne again shellBourne again shell.

This chapter frequently uses the echoecho(1) command to demonstrate shell features. The echo command is very simple: it echoes the input that it receives.

paul@laika:~$ echo Burtonville
Burtonville
paul@laika:~$ echo Smurfs are blue
Smurfs are blue

Arguments

One of the primary features of a shell is to perform a command line scancommand line scan. When you enter a command at the shells command prompt and press the enter key, then the shell will start scanning that line, cutting it up in arguments. While scanning the line, the shell may make many changes to the arguments you typed.

This process is called shell expansionshell expansion. When the shell has finished scanning and modifying that line, then it will be executed.

White space removal

Parts that are separated by one or more consecutive white spaceswhite space(shell) (or tabs) are considered separate arguments, any white space is removed. The first argumentarguments(shell) is the command to be executed, the other arguments are given to the command. The shell effectively cuts your command into one or more arguments.

This explains why the following four different command lines are the same after shell expansionshell expansion.

[paul@RHELv4u3 ~]$ echo Hello World
Hello World
[paul@RHELv4u3 ~]$ echo Hello   World
Hello World
[paul@RHELv4u3 ~]$ echo   Hello   World
Hello World
[paul@RHELv4u3 ~]$    echo      Hello      World
Hello World

The echoecho command will display each argument it receives from the shell. The echo command will also add a new white space between the arguments it received.

Single Quotes

You can prevent the removal of white spaces by quoting the spaces. The contents of the quoted string are considered as one argument. In the screenshot below the echo receives only one argument.

[paul@RHEL4b ~]$ echo 'A line with      single    quotes'
A line with      single    quotes
[paul@RHEL4b ~]$

Double Quotes

You can also prevent the removal of white spaces by double quoting \" (double quotes) the spaces. Same as above, echo only receives one argument.

[paul@RHEL4b ~]$ echo "A line with      double    quotes"
A line with      double    quotes
[paul@RHEL4b ~]$

Later in this book, when discussing variables we will see important differences between single and double quotes.

echo and quotes

Quoted lines can include special escaped characters recognised by the echo command (when using echo -eecho(1)). The screenshot below shows how to use \n for a newline and \t for a tab (usually eight white spaces).

[paul@RHEL4b ~]$ echo -e "A line with \na newline"
A line with 
a newline
[paul@RHEL4b ~]$ echo -e 'A line with \na newline'
A line with 
a newline
[paul@RHEL4b ~]$ echo -e "A line with \ta tab"
A line with     a tab
[paul@RHEL4b ~]$ echo -e 'A line with \ta tab'
A line with     a tab
[paul@RHEL4b ~]$

The echo command can generate more than white spaces, tabs and newlines. Look in the man page for a list of options.

commands

external or builtin commands ?

Not all commands are external to the shell, some are builtin. External commands are programs that have their own binary and reside somewhere in the file system. Many external commands are located in /bin/bin or /sbin/sbin. Builtin commands are an integral part of the shell program itself.

type

To find out whether a command given to the shell will be executed as an external command or as a builtin command, use the typetype(shell) command.

paul@laika:~$ type cd
cd is a shell builtin
paul@laika:~$ type cat
cat is /bin/cat

As you can see, the cd command is builtin and the cat command is external.

You can also use this command to show you whether the command is aliased or not.

paul@laika:~$ type ls
ls is aliased to `ls --color=auto'

running external commands

Some commands have both builtin and external versions. When one of these commands is executed, the builtin version takes priority. To run the external version, you must enter the full path to the command.

paul@laika:~$ type -a echo
echo is a shell builtin
echo is /bin/echo
paul@laika:~$ /bin/echo Running the external echo command... 
Running the external echo command...

which

The whichwhich(1) command will search for binaries in the $PATH$PATH environment variable (variables will be explained later). In the screenshot below, it is determined that cd is builtin, and ls, cp, rm, mv, mkdir, pwd, and which are external commands.

[root@RHEL4b ~]# which cp ls cd mkdir pwd 
/bin/cp
/bin/ls
/usr/bin/which: no cd in (/usr/kerberos/sbin:/usr/kerberos/bin:...
/bin/mkdir
/bin/pwd

aliases

create an alias

The shell allows you to create aliasesalias(bash). Aliases are often used to create an easier to remember name for an existing command or to easily supply parameters.

[paul@RHELv4u3 ~]$ cat count.txt 
one
two
three
[paul@RHELv4u3 ~]$ alias dog=tac
[paul@RHELv4u3 ~]$ dog count.txt 
three
two
one

abbreviate commands

An aliasalias(shell) can also be useful to abbreviate an existing command.

paul@laika:~$ alias ll='ls -lh --color=auto'
paul@laika:~$ alias c='clear'
paul@laika:~$

default options

Aliases can be used to supply commands with default options. The example below shows how to set the -i option default when typing rm/bin/rm.

[paul@RHELv4u3 ~]$ rm -i winter.txt 
rm: remove regular file `winter.txt'? no
[paul@RHELv4u3 ~]$ rm winter.txt 
[paul@RHELv4u3 ~]$ ls winter.txt
ls: winter.txt: No such file or directory
[paul@RHELv4u3 ~]$ touch winter.txt
[paul@RHELv4u3 ~]$ alias rm='rm -i'
[paul@RHELv4u3 ~]$ rm winter.txt 
rm: remove regular empty file `winter.txt'? no
[paul@RHELv4u3 ~]$

Some distributions enable default aliases to protect users from accidentally erasing files (rm -i, mv -i, cp -i)

viewing aliases

You can provide one or more aliases as arguments to the alias command to get their definitions. Providing no arguments gives a complete list of current aliases.

paul@laika:~$ alias c ll
alias c='clear'
alias ll='ls -lh --color=auto'

unalias

You can undo an alias with the unaliasunalias(bash) command.

[paul@RHEL4b ~]$ which rm
/bin/rm
[paul@RHEL4b ~]$ alias rm='rm -i'
[paul@RHEL4b ~]$ which rm
alias rm='rm -i'
        /bin/rm
[paul@RHEL4b ~]$ unalias rm
[paul@RHEL4b ~]$ which rm
/bin/rm
[paul@RHEL4b ~]$

displaying shell expansion

You can display shell expansion with set -xset -x, and stop displaying it with set +xset +x. You might want to use this further on in this course, or when in doubt about exactly what the shell is doing with your command.

[paul@RHELv4u3 ~]$ set -x
++ echo -ne '\033]0;paul@RHELv4u3:~\007'
[paul@RHELv4u3 ~]$ echo $USER
+ echo paul
paul
++ echo -ne '\033]0;paul@RHELv4u3:~\007'
[paul@RHELv4u3 ~]$ echo \$USER
+ echo '$USER'
$USER
++ echo -ne '\033]0;paul@RHELv4u3:~\007'
[paul@RHELv4u3 ~]$ set +x
+ set +x
[paul@RHELv4u3 ~]$ echo $USER
paul

Practice

How many arguments are in this line (not counting the command itself).

touch '/etc/cron/cron.allow' 'file 42.txt' "file 33.txt"
Is tac a shell builtin command ?
Is there an existing alias for rm ?
Read the man page of rm, make sure you understand the -i option of rm. Create and remove a file to test the -i option.
Execute: alias rm='rm -i' . Test your alias with a test file. Does this work as expected ?
List all current aliases.

7a. Create an alias called city that echoes your hometown.

7b. Use your alias to test that it works.

Execute set -x to display shell expansion for every command.
Test the functionality of set -x by executing your city and rm aliases.

10 Execute set +x to stop displaying shell expansion.

Remove your city alias.
What is the location of the cat and the passwd commands ?
Explain the difference between the following commands:

echo

/bin/echo
Explain the difference between the following commands:

echo Hello

echo -n Hello
Display A B C with two spaces between B and C.

(optional)16. Complete the following command (do not use spaces) to display exactly the following output:

4+4     =8
10+14   =24

Use echo to display the following exactly:

??\

Find two solutions with single quotes, two with double quotes and one without quotes (and say thank you to René and Darioush from Google for this extra).

Use one echo command to display three words on three lines.

solution: commands and arguments

How many arguments are in this line (not counting the command itself).

touch '/etc/cron/cron.allow' 'file 42.txt' "file 33.txt"

answer: three
Is tac a shell builtin command ?

type tac
Is there an existing alias for rm ?

alias rm
Read the man page of rm, make sure you understand the -i option of rm. Create and remove a file to test the -i option.

man rm

touch testfile

rm -i testfile
Execute: alias rm='rm -i' . Test your alias with a test file. Does this work as expected ?

touch testfile

rm testfile (should ask for confirmation)
List all current aliases.

alias

7a. Create an alias called city that echoes your hometown.

alias city='echo Antwerp'

7b. Use your alias to test that it works.

city (it should display Antwerp)

Execute set -x to display shell expansion for every command.

set -x
Test the functionality of set -x by executing your city and rm aliases.

shell should display the resolved aliases and then execute the command: paul@deb503:~$ set -x paul@deb503:~$ city + echo antwerp antwerp

10 Execute set +x to stop displaying shell expansion.

set +x

Remove your city alias.

unalias city
What is the location of the cat and the passwd commands ?

which cat (probably /bin/cat)

which passwd (probably /usr/bin/passwd)
Explain the difference between the following commands:

echo

/bin/echo

The echo command will be interpreted by the shell as the built-in echo command. The /bin/echo command will make the shell execute the echo binary located in the /bin directory.

Explain the difference between the following commands:

echo Hello

echo -n Hello

The -n option of the echo command will prevent echo from echoing a trailing newline. echo Hello will echo six characters in total, echo -n hello only echoes five characters.

(The -n option might not work in the Korn shell.)

Display A B C with two spaces between B and C.

echo "A B C"
Complete the following command (do not use spaces) to display exactly the following output:

4+4 =8 10+14 =24

The solution is to use tabs with \t.

echo -e "4+4\t=8" ; echo -e "10+14\t=24"

Use echo to display the following exactly:

??\ echo '??\ echo -e '??\\ echo "??\\" echo -e "??\\\" echo ??\\

Find two solutions with single quotes, two with double quotes and one without quotes (and say thank you to René and Darioush from Google for this extra).

Use one echo command to display three words on three lines.

echo -e "one \ntwo \nthree"

Shell Control Operators

In this chapter we put more than one command on the command line using control operators. We also briefly discuss related parameters ($?) and similar special characters(&).

; semicolon

You can put two or more commands on the same line separated by a semicolon ;; (shell) . The shell will scan the line until it reaches the semicolon. All the arguments before this semicolon will be considered a separate command from all the arguments after the semicolon. Both series will be executed sequentially with the shell waiting for each command to finish before starting the next one.

[paul@RHELv4u3 ~]$ echo Hello
Hello
[paul@RHELv4u3 ~]$ echo World
World
[paul@RHELv4u3 ~]$ echo Hello ; echo World
Hello
World
[paul@RHELv4u3 ~]$

& ampersand

When a line ends with an ampersand &&, the shell will not wait for the command to finish. You will get your shell prompt back, and the command is executed in background. You will get a message when this command has finished executing in background.

[paul@RHELv4u3 ~]$ sleep 20 &
[1] 7925
[paul@RHELv4u3 ~]$ 
...wait 20 seconds...
[paul@RHELv4u3 ~]$ 
[1]+  Done                     sleep 20

The technical explanation of what happens in this case is explained in the chapter about processes.

$? dollar question mark

The exit code of the previous command is stored in the shell variable $?. Actually $?$? (shell variables) is a shell parameter and not a variable, since you cannot assign a value to $?.

paul@debian5:~/test$ touch file1
paul@debian5:~/test$ echo $?
0
paul@debian5:~/test$ rm file1
paul@debian5:~/test$ echo $?
0
paul@debian5:~/test$ rm file1
rm: cannot remove `file1': No such file or directory
paul@debian5:~/test$ echo $?
1
paul@debian5:~/test$

&& double ampersand

The shell will interpret &&&& as a logical AND. When using && the second command is executed only if the first one succeeds (returns a zero exit status).

paul@barry:~$ echo first && echo second
first
second
paul@barry:~$ zecho first && echo second
-bash: zecho: command not found

Another example of the same logical ANDlogical AND principle. This example starts with a working cd followed by ls, then a non-working cd which is not followed by ls.

[paul@RHELv4u3 ~]$ cd gen && ls
file1  file3  File55  fileab  FileAB   fileabc
file2  File4  FileA   Fileab  fileab2
[paul@RHELv4u3 gen]$ cd gen && ls
-bash: cd: gen: No such file or directory

| | double vertical bar

The || represents a logical OR. The second command is executed only when the first command fails (returns a non-zero exit status).

paul@barry:~$ echo first || echo second ; echo third
first
third
paul@barry:~$ zecho first || echo second ; echo third
-bash: zecho: command not found
second
third
paul@barry:~$

Another example of the same logical ORlogical OR principle.

[paul@RHELv4u3 ~]$ cd gen || ls
[paul@RHELv4u3 gen]$ cd gen || ls
-bash: cd: gen: No such file or directory
file1  file3  File55  fileab  FileAB   fileabc
file2  File4  FileA   Fileab  fileab2

combining && and ||

You can use this logical AND and logical OR to write an if-then-else structure on the command line. This example uses echo to display whether the rm command was successful.

paul@laika:~/test$ rm file1 && echo It worked! || echo It failed!
It worked!
paul@laika:~/test$ rm file1 && echo It worked! || echo It failed!
rm: cannot remove `file1': No such file or directory
It failed!
paul@laika:~/test$

# pound sign

Everything written after a pound sign# (pound sign) (#) is ignored by the shell. This is useful to write a shell commentshell comment, but has no influence on the command execution or shell expansion.

paul@debian4:~$ mkdir test    # we create a directory
paul@debian4:~$ cd test       #### we enter the directory
paul@debian4:~/test$ ls       # is it empty ?
paul@debian4:~/test$

\ escaping special characters

The backslash \\ (backslash) character enables the use of control characters, but without the shell interpreting it, this is called escapingshell escaping characters.

[paul@RHELv4u3 ~]$ echo hello \; world
hello ; world
[paul@RHELv4u3 ~]$ echo hello\ \ \ world
hello   world
[paul@RHELv4u3 ~]$ echo escaping \\\ \#\ \&\ \"\ 
escaping \ # & " '
[paul@RHELv4u3 ~]$ echo escaping \\\?\*\"
escaping \?*"'

end of line backslash

Lines ending in a backslash are continued on the next line. The shell does not interpret the newline character and will wait on shell expansion and execution of the command line until a newline without backslash is encountered.

[paul@RHEL4b ~]$ echo This command line \
> is split in three \
> parts
This command line is split in three parts
[paul@RHEL4b ~]$

practice: control operators

Each question can be answered by one command line!
When you type passwd, which file is executed ?
What kind of file is that ?
Execute the pwd command twice. (remember 0.)
Execute ls after cd /etc, but only if cd /etc did not error.
Execute cd /etc after cd etc, but only if cd etc fails.
Echo it worked when touch test42 works, and echo it failed when the touch failed. All on one command line as a normal user (not root). Test this line in your home directory and in /bin/ .
Execute sleep 6, what is this command doing ?
Execute sleep 200 in background (do not wait for it to finish).
Write a command line that executes rm file55. Your command line should print success if file55 is removed, and print failed if there was a problem.

(optional)10. Use echo to display \"Hello World with strange characters \ * [ } ~ \\ .\" (including all quotes) solution: control operators ===========================

Each question can be answered by one command line!
When you type passwd, which file is executed ?

which passwd
What kind of file is that ?

file /usr/bin/passwd
Execute the pwd command twice. (remember 0.)

pwd ; pwd
Execute ls after cd /etc, but only if cd /etc did not error.

cd /etc && ls
Execute cd /etc after cd etc, but only if cd etc fails.

cd etc || cd /etc
Echo it worked when touch test42 works, and echo it failed when the touch failed. All on one command line as a normal user (not root). Test this line in your home directory and in /bin/ .

paul@deb503:~$ cd ; touch test42 && echo it worked || echo it failed it worked paul@deb503:~$ cd /bin; touch test42 && echo it worked || echo it failed touch: cannot touch `test42': Permission denied it failed
Execute sleep 6, what is this command doing ?

pausing for six seconds
Execute sleep 200 in background (do not wait for it to finish).

sleep 200 &
Write a command line that executes rm file55. Your command line should print success if file55 is removed, and print failed if there was a problem.

rm file55 && echo success || echo failed

(optional)10. Use echo to display \"Hello World with strange characters \ * [ } ~ \\ .\" (including all quotes)

echo \"Hello World with strange characters \\ \* \[ \} \~ \\\\ .  \"

or

echo \""Hello World with strange' characters \ * [ } ~ \\ . "\"

Shell Variables

In this chapter we learn to manage environment variables in the shell. These variables are often needed by applications. $ dollar sign ==============

Another important character interpreted by the shell is the dollar sign $$ (shell variables). The shell will look for an environment variableenvironment variable named like the string following the dollar sign and replace it with the value of the variable (or with nothing if the variable does not exist).

These are some examples using $HOSTNAME, $USER, $UID, $SHELL, and $HOME.

[paul@RHELv4u3 ~]$ echo This is the $SHELL shell
This is the /bin/bash shell
[paul@RHELv4u3 ~]$ echo This is $SHELL on computer $HOSTNAME
This is /bin/bash on computer RHELv4u3.localdomain
[paul@RHELv4u3 ~]$ echo The userid of $USER is $UID
The userid of paul is 500
[paul@RHELv4u3 ~]$ echo My homedir is $HOME
My homedir is /home/paul

case sensitive

This example shows that shell variables are case sensitive!

[paul@RHELv4u3 ~]$ echo Hello $USER
Hello paul
[paul@RHELv4u3 ~]$ echo Hello $user
Hello

creating variables

This example creates the variable $MyVar and sets its value. It then uses echo to verify the value.

[paul@RHELv4u3 gen]$ MyVar=555
[paul@RHELv4u3 gen]$ echo $MyVar
555
[paul@RHELv4u3 gen]$

quotes

Notice that double quotes still allow the parsing of variables, whereas single quotes prevent this.

[paul@RHELv4u3 ~]$ MyVar=555
[paul@RHELv4u3 ~]$ echo $MyVar
555
[paul@RHELv4u3 ~]$ echo "$MyVar"
555
[paul@RHELv4u3 ~]$ echo '$MyVar'
$MyVar

The bash shell will replace variables with their value in double quoted lines, but not in single quoted lines.

paul@laika:~$ city=Burtonville
paul@laika:~$ echo "We are in $city today."
We are in Burtonville today.
paul@laika:~$ echo 'We are in $city today.'
We are in $city today.

set

You can use the setset(shell) command to display a list of environment variables. On Ubuntu and Debian systems, the set command will also list shell functions after the shell variables. Use set | more to see the variables then.

unset

Use the unsetunset(shell) command to remove a variable from your shell environment.

[paul@RHEL4b ~]$ MyVar=8472
[paul@RHEL4b ~]$ echo $MyVar
8472
[paul@RHEL4b ~]$ unset MyVar
[paul@RHEL4b ~]$ echo $MyVar

[paul@RHEL4b ~]$

$PS1

The $PS1 variable determines your shell prompt. You can use backslash escaped special characters like \u for the username or \w for the working directory. The bash manual has a complete reference.

In this example we change the value of $PS1 a couple of times.

paul@deb503:~$ PS1=prompt
prompt
promptPS1='prompt '
prompt 
prompt PS1='> '
> 
> PS1='\u@\h$ '
paul@deb503$ 
paul@deb503$ PS1='\u@\h:\W$'
paul@deb503:~$

To avoid unrecoverable mistakes, you can set normal user prompts to green and the root prompt to red. Add the following to your .bashrc for a green user prompt:

# color prompt by paul
RED='\[\033[01;31m\]'
WHITE='\[\033[01;00m\]'
GREEN='\[\033[01;32m\]'
BLUE='\[\033[01;34m\]'
export PS1="${debian_chroot:+($debian_chroot)}$GREEN\u$WHITE@$BLUE\h$WHITE\w\$ "

$PATH

The $PATH$PATH variable is determines where the shell is looking for commands to execute (unless the command is builtin or aliased). This variable contains a list of directories, separated by colons.

[[paul@RHEL4b ~]$ echo $PATH
/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin:

The shell will not look in the current directory for commands to execute! (Looking for executables in the current directory provided an easy way to hack PC-DOS computers). If you want the shell to look in the current directory, then add a . at the end of your $PATH.

[paul@RHEL4b ~]$ PATH=$PATH:.
[paul@RHEL4b ~]$ echo $PATH
/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin:.
[paul@RHEL4b ~]$

Your path might be different when using su instead of su -su - because the latter will take on the environment of the target user. The root user typically has /sbin directories added to the $PATH variable.

[paul@RHEL3 ~]$ su
Password: 
[root@RHEL3 paul]# echo $PATH
/usr/local/bin:/bin:/usr/bin:/usr/X11R6/bin
[root@RHEL3 paul]# exit
[paul@RHEL3 ~]$ su -
Password: 
[root@RHEL3 ~]# echo $PATH
/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:
[root@RHEL3 ~]#

env

The envenv(1) command without options will display a list of exported variables. The difference with set with options is that set lists all variables, including those not exported to child shells.

But env can also be used to start a clean shell (a shell without any inherited environment). The env -ienv(1) command clears the environment for the subshell.

Notice in this screenshot that bash will set the $SHELL variable on startup.

[paul@RHEL4b ~]$ bash -c 'echo $SHELL $HOME $USER'
/bin/bash /home/paul paul
[paul@RHEL4b ~]$ env -i bash -c 'echo $SHELL $HOME $USER'
/bin/bash
[paul@RHEL4b ~]$

You can use the env command to set the $LANG, or any other, variable for just one instance of bash with one command. The example below uses this to show the influence of the $LANG variable on file globbing (see the chapter on file globbing).

[paul@RHEL4b test]$ env LANG=C bash -c 'ls File[a-z]'
Filea  Fileb
[paul@RHEL4b test]$ env LANG=en_US.UTF-8 bash -c 'ls File[a-z]'
Filea  FileA  Fileb  FileB
[paul@RHEL4b test]$

export

You can export shell variables to other shells with the exportexport command. This will export the variable to child shells.

[paul@RHEL4b ~]$ var3=three
[paul@RHEL4b ~]$ var4=four
[paul@RHEL4b ~]$ export var4
[paul@RHEL4b ~]$ echo $var3 $var4
three four
[paul@RHEL4b ~]$ bash
[paul@RHEL4b ~]$ echo $var3 $var4
four

But it will not export to the parent shell (previous screenshot continued).

[paul@RHEL4b ~]$ export var5=five
[paul@RHEL4b ~]$ echo $var3 $var4 $var5
four five
[paul@RHEL4b ~]$ exit
exit
[paul@RHEL4b ~]$ echo $var3 $var4 $var5
three four
[paul@RHEL4b ~]$

delineate variables

Until now, we have seen that bash interprets a variable starting from a dollar sign, continuing until the first occurrence of a non-alphanumeric character that is not an underscore. In some situations, this can be a problem. This issue can be resolved with curly braces like in this example.

[paul@RHEL4b ~]$ prefix=Super
[paul@RHEL4b ~]$ echo Hello $prefixman and $prefixgirl
Hello  and
[paul@RHEL4b ~]$ echo Hello ${prefix}man and ${prefix}girl
Hello Superman and Supergirl
[paul@RHEL4b ~]$

unbound variables

The example below tries to display the value of the $MyVar variable, but it fails because the variable does not exist. By default the shell will display nothing when a variable is unbound (does not exist).

[paul@RHELv4u3 gen]$ echo $MyVar

[paul@RHELv4u3 gen]$

There is, however, the nounsetnounset(shell) shell option that you can use to generate an error when a variable does not exist.

paul@laika:~$ set -u
paul@laika:~$ echo $Myvar
bash: Myvar: unbound variable
paul@laika:~$ set +u
paul@laika:~$ echo $Myvar

paul@laika:~$

In the bash shell set -u is identical to set -o nounset and likewise set +u is identical to set +o nounset. practice: shell variables =========================

Use echo to display Hello followed by your username. (use a bash variable!)
Create a variable answer with a value of 42.
Copy the value of $LANG to $MyLANG.
List all current shell variables.
List all exported shell variables.
Do the env and set commands display your variable ?
Destroy your answer variable.
Create two variables, and export one of them.
Display the exported variable in an interactive child shell.
Create a variable, give it the value Dumb, create another variable with value do. Use echo and the two variables to echo Dumbledore.
Find the list of backslash escaped characters in the manual of bash. Add the time to your PS1 prompt. solution: shell variables =========================
Use echo to display Hello followed by your username. (use a bash variable!)

echo Hello $USER
Create a variable answer with a value of 42.

answer=42
Copy the value of $LANG to $MyLANG.

MyLANG=$LANG
List all current shell variables.

set

set|more on Ubuntu/Debian
List all exported shell variables.

env export declare -x
Do the env and set commands display your variable ?

env | more set | more
Destroy your answer variable.

unset answer
Create two variables, and export one of them.

var1=1; export var2=2
Display the exported variable in an interactive child shell.

bash echo $var2
Create a variable, give it the value Dumb, create another variable with value do. Use echo and the two variables to echo Dumbledore.

varx=Dumb; vary=do

echo ${varx}le$ {vary}re solution by Yves from Dexia : echo $varx'le'$ vary're' solution by Erwin from Telenet : echo " $varx"le"$ vary"re
Find the list of backslash escaped characters in the manual of bash. Add the time to your PS1 prompt.

PS1='\t \u@\h \W$ '

Shell in Shell

This chapter takes a brief look at child shells, embedded shells and shell options. shell embedding ===============

Shells can be embeddedshell embedding on the command line, or in other words, the command line scan can spawn new processes containing a fork of the current shell. You can use variables to prove that new shells are created. In the screenshot below, the variable $var1 only exists in the (temporary) sub shell.

[paul@RHELv4u3 gen]$ echo $var1

[paul@RHELv4u3 gen]$ echo $(var1=5;echo $var1)
5
[paul@RHELv4u3 gen]$ echo $var1

[paul@RHELv4u3 gen]$

You can embed a shell in an embedded shellembedding(shell), this is called nested embedding of shells.

This screenshot shows an embedded shell inside an embedded shell.

paul@deb503:~$ A=shell
paul@deb503:~$ echo $C$B$A $(B=sub;echo $C$B$A; echo $(C=sub;echo $C$B$A))
shell subshell subsubshell

backticks

Single embedding can be useful to avoid changing your current directory. The screenshot below uses backticks instead of dollar-bracket to embed.

[paul@RHELv4u3 ~]$ echo `cd /etc; ls -d * | grep pass`
passwd passwd- passwd.OLD
[paul@RHELv4u3 ~]$

You can only use the $()$() embedded shell notation to nest embedded shells, backticks cannot do this.

backticks or single quotes

Placing the embedding between backticksbackticks uses one character less than the dollar and parenthesis combo. Be careful however, backticks are often confused with single quotes. The technical difference between '(single quote) and ``(backtick) is significant!

[paul@RHELv4u3 gen]$ echo `var1=5;echo $var1`
5
[paul@RHELv4u3 gen]$ echo 'var1=5;echo $var1'
var1=5;echo $var1
[paul@RHELv4u3 gen]$

shell options

Both setset and unsetunset are builtin shell commands. They can be used to set options of the bash shell itself. The next example will clarify this. By default, the shell will treat unset variables as a variable having no value. By setting the -u option, the shell will treat any reference to unset variables as an error. See the man page of bash for more information.

[paul@RHEL4b ~]$ echo $var123

[paul@RHEL4b ~]$ set -u
[paul@RHEL4b ~]$ echo $var123
-bash: var123: unbound variable
[paul@RHEL4b ~]$ set +u
[paul@RHEL4b ~]$ echo $var123

[paul@RHEL4b ~]$

To list all the set options for your shell, use echo $-echo $-. The noclobber (or -C) option will be explained later in this book (in the I/O redirection chapter).

[paul@RHEL4b ~]$ echo $-
himBH
[paul@RHEL4b ~]$ set -C ; set -u
[paul@RHEL4b ~]$ echo $-
himuBCH
[paul@RHEL4b ~]$ set +C ; set +u
[paul@RHEL4b ~]$ echo $-
himBH
[paul@RHEL4b ~]$

When typing set without options, you get a list of all variables without function when the shell is on posix mode. You can set bash in posix mode typing set -o posix. practice: shell embedding =========================

Find the list of shell options in the man page of bash. What is the difference between set -u and set -o nounset?
Activate nounset in your shell. Test that it shows an error message when using non-existing variables.
Deactivate nounset.
Execute cd /var and ls in an embedded shell.

The echo command is only needed to show the result of the ls command. Omitting will result in the shell trying to execute the first file as a command.

Create the variable embvar in an embedded shell and echo it. Does the variable exist in your current shell now ?
Explain what \"set -x\" does. Can this be useful ?

(optional)7. Given the following screenshot, add exactly four characters to that command line so that the total output is FirstMiddleLast.

[paul@RHEL4b ~]$ echo  First; echo  Middle; echo  Last

Display a long listing (ls -l) of the passwd command using the which command inside an embedded shell. solution: shell embedding =========================
Find the list of shell options in the man page of bash. What is the difference between set -u and set -o nounset?

read the manual of bash (man bash), search for nounset -- both mean the same thing.

Activate nounset in your shell. Test that it shows an error message when using non-existing variables.

set -u OR set -o nounset

Both these lines have the same effect.

Deactivate nounset.

set +u OR set +o nounset
Execute cd /var and ls in an embedded shell.

echo $(cd /var ; ls)

The echo command is only needed to show the result of the ls command. Omitting will result in the shell trying to execute the first file as a command.

Create the variable embvar in an embedded shell and echo it. Does the variable exist in your current shell now ?

echo $(embvar=emb;echo $embvar) ; echo $embvar #the last echo fails

$embvar does not exist in your current shell
Explain what \"set -x\" does. Can this be useful ?

It displays shell expansion for troubleshooting your command.

(optional)7. Given the following screenshot, add exactly four characters to that command line so that the total output is FirstMiddleLast.

[paul@RHEL4b ~]$ echo  First; echo  Middle; echo  Last

echo -n First; echo -n Middle; echo Last

Display a long listing (ls -l) of the passwd command using the which command inside an embedded shell.

ls -l $(which passwd)

Shell History

The shell makes it easy for us to repeat commands, this chapter explains how. repeating the last command ==========================

To repeat the last command in bash, type !!!! (shell). This is pronounced as bang bang.

paul@debian5:~/test42$ echo this will be repeated > file42.txt
paul@debian5:~/test42$ !!
echo this will be repeated > file42.txt
paul@debian5:~/test42$

repeating other commands

You can repeat other commands using one bang followed by one or more characters. The shell will repeat the last command that started with those characters.

paul@debian5:~/test42$ touch file42
paul@debian5:~/test42$ cat file42
paul@debian5:~/test42$ !to
touch file42
paul@debian5:~/test42$

history

To see older commands, use historybash history to display the shell command history (or use history n to see the last n commands).

paul@debian5:~/test$ history 10
38  mkdir test
39  cd test
40  touch file1
41  echo hello > file2
42  echo It is very cold today > winter.txt
43  ls
44  ls -l
45  cp winter.txt summer.txt
46  ls -l
47  history 10

!n

When typing !! (bash history) followed by the number preceding the command you want repeated, then the shell will echo the command and execute it.

paul@debian5:~/test$ !43
ls
file1  file2  summer.txt  winter.txt

Ctrl-r

Another option is to use ctrl-rctrl-r to search in the history. In the screenshot below i only typed ctrl-r followed by four characters apti and it finds the last command containing these four consecutive characters.

paul@debian5:~$ 
(reverse-i-search)`apti': sudo aptitude install screen

$HISTSIZE

The $HISTSIZE$HISTSIZE variable determines the number of commands that will be remembered in your current environment. Most distributions default this variable to 500 or 1000.

paul@debian5:~$ echo $HISTSIZE
500

You can change it to any value you like.

paul@debian5:~$ HISTSIZE=15000
paul@debian5:~$ echo $HISTSIZE
15000

$HISTFILE

The $HISTFILE$HISTFILE variable points to the file that contains your history. The bash shell defaults this value to ~/.bash_history.bash_history.

paul@debian5:~$ echo $HISTFILE
/home/paul/.bash_history

A session history is saved to this file when you exitexit (bash) the session!

Closing a gnome-terminal with the mouse, or typing rebootreboot as root will NOT save your terminals history.

$HISTFILESIZE

The number of commands kept in your history file can be set using $HISTFILESIZE$HISTFILESIZE.

paul@debian5:~$ echo $HISTFILESIZE
15000

prevent recording a command

You can prevent a command from being recorded in history using a space prefix.

paul@debian8:~/github$ echo abc
abc
paul@debian8:~/github$  echo def
def
paul@debian8:~/github$ echo ghi
ghi
paul@debian8:~/github$ history 3
 9501  echo abc
 9502  echo ghi
 9503  history 3

(optional)regular expressions

It is possible to use regular expressionsregular expressions when using the bang to repeat commands. The screenshot below switches 1 into 2.

paul@debian5:~/test$ cat file1
paul@debian5:~/test$ !c:s/1/2
cat file2
hello
paul@debian5:~/test$

(optional) Korn shell history

Repeating a command in the Korn shellKorn shell is very similar. The Korn shell ksh also has the history command, but uses the letter r to recall lines from history.

This screenshot shows the history command. Note the different meaning of the parameter.

$ history 17
17  clear
18  echo hoi
19  history 12
20  echo world
21  history 17

Repeating with r can be combined with the line numbers given by the history command, or with the first few letters of the command.

$ r e
echo world
world
$ cd /etc
$ r
cd /etc
$

practice: shell history

Issue the command echo The answer to the meaning of life, the universe and everything is 42.
Repeat the previous command using only two characters (there are two solutions!)
Display the last 5 commands you typed.
Issue the long echo from question 1 again, using the line numbers you received from the command in question 3.
How many commands can be kept in memory for your current shell session ?
Where are these commands stored when exiting the shell ?
How many commands can be written to the history file when exiting your current shell session ?
Make sure your current bash shell remembers the next 5000 commands you type.
Open more than one console (by press Ctrl-shift-t in gnome-terminal, or by opening an extra putty.exe in MS Windows) with the same user account. When is command history written to the history file ? solution: shell history =======================
Issue the command echo The answer to the meaning of life, the universe and everything is 42.

echo The answer to the meaning of life, the universe and everything is 42
Repeat the previous command using only two characters (there are two solutions!)

!! OR !e
Display the last 5 commands you typed.

paul@ubu1010:~$ history 5 52 ls -l 53 ls 54 df -h | grep sda 55 echo The answer to the meaning of life, the universe and everything is 42 56 history 5

You will receive different line numbers.

Issue the long echo from question 1 again, using the line numbers you received from the command in question 3.

paul@ubu1010:~$ !55 echo The answer to the meaning of life, the universe and everything is 42 The answer to the meaning of life, the universe and everything is 42
How many commands can be kept in memory for your current shell session ?

echo $HISTSIZE
Where are these commands stored when exiting the shell ?

echo $HISTFILE
How many commands can be written to the history file when exiting your current shell session ?

echo $HISTFILESIZE
Make sure your current bash shell remembers the next 5000 commands you type.

HISTSIZE=5000
Open more than one console (by press Ctrl-shift-t in gnome-terminal, or by opening an extra putty.exe in MS Windows) with the same user account. When is command history written to the history file ?

when you type exit

Shell Globbing

Typing man 7 glob (on Debian) will tell you that long ago there was a program called /etc/glob that would expand wildcard patterns.

Today the shell is responsible for file globbingfile globbing (or dynamic filename generation). This chapter will explain file globbing. * asterisk ===========

The asterisk ** (file globbing) is interpreted by the shell as a sign to generate filenames, matching the asterisk to any combination of characters (even none). When no path is given, the shell will use filenames in the current directory. See the man page of glob(7)glob(7) for more information. (This is part of LPI topic 1.103.3.)

[paul@RHELv4u3 gen]$ ls
file1  file2  file3  File4  File55  FileA  fileab  Fileab  FileAB  fileabc
[paul@RHELv4u3 gen]$ ls File*
File4  File55  FileA  Fileab  FileAB
[paul@RHELv4u3 gen]$ ls file*
file1  file2  file3  fileab  fileabc
[paul@RHELv4u3 gen]$ ls *ile55
File55
[paul@RHELv4u3 gen]$ ls F*ile55
File55
[paul@RHELv4u3 gen]$ ls F*55
File55
[paul@RHELv4u3 gen]$

? question mark

Similar to the asterisk, the question mark ?? (file globbing) is interpreted by the shell as a sign to generate filenames, matching the question mark with exactly one character.

[paul@RHELv4u3 gen]$ ls
file1  file2  file3  File4  File55  FileA  fileab  Fileab  FileAB  fileabc
[paul@RHELv4u3 gen]$ ls File?
File4  FileA
[paul@RHELv4u3 gen]$ ls Fil?4
File4
[paul@RHELv4u3 gen]$ ls Fil??
File4  FileA
[paul@RHELv4u3 gen]$ ls File??
File55  Fileab  FileAB
[paul@RHELv4u3 gen]$

[] square brackets

The square bracket [[ (file globbing) is interpreted by the shell as a sign to generate filenames, matching any of the characters between [ and the first subsequent ]. The order in this list between the brackets is not important. Each pair of brackets is replaced by exactly one character.

[paul@RHELv4u3 gen]$ ls 
file1  file2  file3  File4  File55  FileA  fileab  Fileab  FileAB  fileabc
[paul@RHELv4u3 gen]$ ls File[5A]
FileA
[paul@RHELv4u3 gen]$ ls File[A5]
FileA
[paul@RHELv4u3 gen]$ ls File[A5][5b]
File55
[paul@RHELv4u3 gen]$ ls File[a5][5b]
File55  Fileab
[paul@RHELv4u3 gen]$ ls File[a5][5b][abcdefghijklm]
ls: File[a5][5b][abcdefghijklm]: No such file or directory
[paul@RHELv4u3 gen]$ ls file[a5][5b][abcdefghijklm]
fileabc
[paul@RHELv4u3 gen]$

You can also exclude characters from a list between square brackets with the exclamation mark !! (file globbing). And you are allowed to make combinations of these wild cardswild cards.

[paul@RHELv4u3 gen]$ ls 
file1  file2  file3  File4  File55  FileA  fileab  Fileab  FileAB  fileabc
[paul@RHELv4u3 gen]$ ls file[a5][!Z]
fileab
[paul@RHELv4u3 gen]$ ls file[!5]*
file1  file2  file3  fileab  fileabc
[paul@RHELv4u3 gen]$ ls file[!5]?
fileab
[paul@RHELv4u3 gen]$

a-z and 0-9 ranges

The bash shell will also understand ranges of characters between brackets.

[paul@RHELv4u3 gen]$ ls
file1  file3  File55  fileab  FileAB   fileabc
file2  File4  FileA   Fileab  fileab2
[paul@RHELv4u3 gen]$ ls file[a-z]*
fileab  fileab2  fileabc
[paul@RHELv4u3 gen]$ ls file[0-9]
file1  file2  file3
[paul@RHELv4u3 gen]$ ls file[a-z][a-z][0-9]*
fileab2
[paul@RHELv4u3 gen]$

$LANG and square brackets

But, dont forget the influence of the LANG$LANG variable. Some languages include lower case letters in an upper case range (and vice versa).

paul@RHELv4u4:~/test$ ls [A-Z]ile?
file1  file2  file3  File4
paul@RHELv4u4:~/test$ ls [a-z]ile?
file1  file2  file3  File4
paul@RHELv4u4:~/test$ echo $LANG
en_US.UTF-8
paul@RHELv4u4:~/test$ LANG=C
paul@RHELv4u4:~/test$ echo $LANG
C
paul@RHELv4u4:~/test$ ls [a-z]ile?
file1  file2  file3
paul@RHELv4u4:~/test$ ls [A-Z]ile?
File4
paul@RHELv4u4:~/test$

If $LC_ALL is set, then this will also need to be reset to prevent file globbing.

preventing file globbing

The screenshot below should be no surprise. The echo *echo * will echo a * when in an empty directory. And it will echo the names of all files when the directory is not empty.

paul@ubu1010:~$ mkdir test42
paul@ubu1010:~$ cd test42
paul@ubu1010:~/test42$ echo *
*
paul@ubu1010:~/test42$ touch file42 file33
paul@ubu1010:~/test42$ echo *
file33 file42

Globbing can be prevented using quotes or by escaping escaping (shell)the special characters, as shown in this screenshot.

paul@ubu1010:~/test42$ echo *
file33 file42
paul@ubu1010:~/test42$ echo \*
*
paul@ubu1010:~/test42$ echo '*'
*
paul@ubu1010:~/test42$ echo "*"
*

practice: shell globbing

Create a test directory and enter it.
Create the following files :

file1 file10 file11 file2 File2 File3 file33 fileAB filea fileA fileAAA file( file 2

(the last one has 6 characters including a space)

List (with ls) all files starting with file
List (with ls) all files starting with File
List (with ls) all files starting with file and ending in a number.
List (with ls) all files starting with file and ending with a letter
List (with ls) all files starting with File and having a digit as fifth character.
List (with ls) all files starting with File and having a digit as fifth character and nothing else.
List (with ls) all files starting with a letter and ending in a number.
List (with ls) all files that have exactly five characters.
List (with ls) all files that start with f or F and end with 3 or A.
List (with ls) all files that start with f have i or R as second character and end in a number.
List all files that do not start with the letter F.
Copy the value of $LANG to $MyLANG.
Show the influence of $LANG in listing A-Z or a-z ranges.
You receive information that one of your servers was cracked, the cracker probably replaced the ls command. You know that the echo command is safe to use. Can echo replace ls ? How can you list the files in the current directory with echo ?
Is there another command besides cd to change directories ? solution: shell globbing ========================
Create a test directory and enter it.

mkdir testdir; cd testdir
Create the following files :

file1 file10 file11 file2 File2 File3 file33 fileAB filea fileA fileAAA file( file 2