The Bourne Again Shell
The Bourne Again shell (bash) is the program that functions as the frontline interface for many UNIX/Linux users at UNC, interpreting their commands and launching the programs requested. This documents features of the Bourne Again shell. The Bourne Again shell is available on commercial UNIX systems and is a highly flexible and powerful command execution environment. It is developed and maintained by contributors to the GNU project. Its growing popularity is directly attributable to its versatile scripting language and its ability to execute shell scripts written for the Bourne shell (sh).
In this document we will explore some commands and features that can make the shell more effective for you and allow you to tailor your environment to your needs. This document covers the following information:
- Reusing Your Commands with the History Mechanism
- Pipes and Redirection
- Filename Metacharacters
- Variables and Options
- Initialization Files
What Are Some Reasons to Use the bash Shell?
The Bourne Again shell (referred to in this document as bash) has several enhancements that make it a better choice than its predecessors, the Bourne shell and, depending on the user’s tastes, the tcsh: including an enhanced history mechanism, user-definable functions, and a broad set of filename metacharacters. Its scripting language is generally considered superior to that of both the Bourne and C shells.
Reusing Your Commands with the History Mechanism
Most modern shells, not the Bourne shell, offer some kind of mechanism for reusing and editing previous commands–(if, for instance you misspell a long filename or want to change a long command slightly, or you want to reuse a command several times). This mechanism is typically referred to as the history mechanism. To start getting a feel for it, type the command:
This will produce a list of your last commands in the order in which they were executed. The exact number of commands displayed is controlled by a shell variable. (We will discuss the general topic of shell variables later and address how to control this number then.) The list will be in two columns like this:
324 swap -s
325 ssh -X statapps.unc.edu
327 ls -lt
328 more test.sas
The first column is the number of the command in the list. The second column is the command itself, just as you entered it.
Command Line Editing
One of bash’s most useful features is the ability it provides the user to recall old commands, edit them, and reexecute them. The bash shell provides a mechanism for the user to interactively scroll through and edit commands from the history list. There are two different command editing modes built into bash–one uses the cursor motion and control keys from the vi editor, the other control keys from emacs. If you are not already familiar with one editor or the other, either of these modes will take a little getting used to, especially if you are accustomed to the arrow key functionality of a shell like tcsh.
Choosing a Mode
Selecting whether to use vi or emacs mode is a matter of preference–both offer similar capabilities using different keystrokes. To activate one or the other, do one of two things:
1) Set the value of the VISUAL variable to either vi or emacs using a command like this:
” editorname” would be replaced by either vi or macs.
2) Activate the desired mode by using the following command:
Either one of these settings can be made permanent by entering the command into your environment initialization file (see the section on initialization files at the end of this document).
To use the command recall and editing keys under vi mode, you must first press esc (Escape). This activates what is known as “command mode”, in which keys do not have their normal values, so that pressing a j may not cause the appropriate character to appear on screen. Instead many of the keys will perform commands, allowing you to scroll through the command history, move the cursor within a given command, and perform searching and editing functions. To go back to line command mode, click on esc again.
The following table gives a brief summary of some of the more commonly used commands–see the vi man page for more detailed instructions on these and other commands.
Table 1. vi Mode Commands
|k||scrolls one command backwards through history list|
|j||scrolls one command forwards through history list|
|l||moves cursor one character to right|
|h||moves cursor one character to left|
|0 (zero)||moves cursor to beginning of line|
|$||moves cursor to end of line|
|w||moves cursor forwards one word|
|b||moves cursor backwards one word|
|r||replace current character|
|R||replace characters at current position|
|i||insert text at current cursor position|
|a||append text after current cursor position|
|/||search history list for string|
|\||complete partially-typed filename a.k.a. file name completion|
The emacs mode differs from the vi mode in one crucial respect –there is no “command mode” which needs to be activated to use the command keys. Instead, keystrokes within emacs mode are combined with the Control or Meta (often esc) keys to perform commands. Commands may be performed at any time.
Detailed descriptions and explanations for the emacs mode are available in the bash man page, and documentation for the editor is available through the info system built into emacs itself. The following table summarizes the emacs equivalents of the vi keystrokes listed above.
Table 2. emacs Mode Commands
|Ctrl -p||scrolls one command backward through history list|
|Ctrl -n||scrolls one command forwards through history list|
|Ctrl -f||moves cursor one character to right|
|Ctrl -b||moves cursor one character to left|
|Ctrl -a||moves cursor to beginning of line|
|Ctrl -e||moves cursor to end of line|
|Ctrl -f||moves cursor forwards one word|
|Ctrl -b||moves cursor backwards one word|
|Ctrl -r||search history list for string|
|ESC ESC||complete partially typed filename|
Pipes and Redirection
Let’s look at two situations that can occur fairly easily in normal UNIX/Linux usage. The first is: you already know how to get a long listing of all the files in a directory with the ls -al command. It is often the case though, that the output produced by this command is too long to fit on the screen and so scrolls right off the page. The second situation is this: users will often want to save the output of some command, (the ls -al example will work fine for this as well), to a file for later examination. Or perhaps you want to mail the output to someone. Whatever the reason, you want to save the output as a normal text file. Both of these situations can be addressed using a mechanism known generally as redirection. There are two basic types of redirection:
(1) redirecting an input or output stream to or from a file
(2) piping the output of one command to the input of another
What Does It Mean to Redirect?
Each process running under UNIX/Linux has certain “streams” associated with it. A stream is simply a flow of data (characters, numbers, …). By default, most input is supposed to come from the keyboard, so there is a stream called the standard input (stdin for short) which is a flow of data from the keyboard. Not all programs make use of this stream, but it’s there. And most output is generally presumed to go to the terminal screen, so there is a stream called the standard output (stdout for short) which is a flow of data to the terminal. Most programs make use of this stream (but again, not all). And most output generated as a result of some error is also presumed to go to the terminal screen, and there is a stream known as the standard error (stderr for short) which has the same destination as stdout (the terminal), but it is very important to remember that they are still separate streams.
Examples of Pipes and Redirection
Now, addressing the second of our two situations above, let’s suppose that we want to save the output of a command, in this case ls -al, to a file. Since we know that the ls command sends its output to the terminal, we need simply to need to tell it that stdout actually should lead elsewhere, such as a file. We do this by means of a redirection operator, of which there are several. The most basic and commonly used is the one we want in this case, which tells the shell that the output of our command should be sent to some file, rather than the terminal. We could use something like the following:
Here the greater-than symbol (>) tells the shell that the standard output of the command immediately preceding it should be redirected to the file named by ” outputfile” , instead of being sent to the screen. Note that since standard output and standard error are separate streams, (although both by default get sent to the terminal), any error messages will remain on the screen rather than being saved to the file.
Standard output is not the only stream that can be redirected; in fact, all three streams can be redirected in a variety of ways. Suppose you have the file now after executing the ls -al command, and you wish to mail it to someone. One easy way to do this involves using the UNIX/Linux mail command and redirection. By default, typing mail email@example.com will start up the mail program, which waits for you to type in the text of the message. As you may have guessed, the keystrokes you input are read by the program off the standard input. So in this case, we need to tell the shell that, instead of reading keystrokes from the keyboard, standard input should actually read from our output file. We can do this with the following:
Here the less-than symbol (<) tells the shell that standard input should be remapped to ” inputfile” instead of the terminal keyboard.
If we think back to the first situation we described, in which the output from some command is so long that it scrolls right off the screen, there is a way around this as well. Think about it this way: We know we can redirect output to a file – what if we could send the output into another command as input? We could do this with a program that takes its input and prints it to the screen one page at a time, so the user can read long files. One command that would fill this purpose is the more program, which prints a file to the screen one page at a time, which is just what we need.
The more program executes the ls -al command and then pipes its output into more, instead of printing it directly to the terminal screen. The more program pages the output to the terminal, letting us read it one section at a time. The pipe operator (|) says “take the output of the first command and pass it to the second command as input.”
Other Redirection Operators
Here are some useful redirection operators that the bash shell recognizes:
|>||cmd > file||Write output of cmd to file (overwrite file if file already exists)|
|>>||cmd >> file||Append output of cmd to end of file|
|<||cmd < file||Read input to cmd from file|
|2>||cmd 2> file||Write stderr to file, stdout still goes to screen|
|2>& 1||cmd >file 2> &1||Write both stderr and stdout to file|
||||cmd1 | cmd2||Send stdout of cmd1 to cmd2 as input|
Applying a Command to Multiple Files with Metacharacters
Often, when you are working in UNIX/Linux, you will want to apply one command to several different files at once. There are a variety of different ways to do this: one is simply to type the command followed by the name of each file on the command line – but for large numbers of files this is tiring and inefficient. There is an easier way. We can use filename metacharacters or wildcards to allow a command to be applied to all filenames of a particular shape or pattern, say all those beginning with a certain letter or ending in a particular suffix.
What are Metacharacters/Wildcards?
Both words refer to the same thing: characters that have special meaning to the shell when used as part of a filename. To get a sense of how a wildcard works, think of playing cards: a wildcard is one which may be any other card if needed – say an ace. Filename wildcards work similarly. If I want to apply a command to all the files in a directory beginning with the letter ”h”, I can pass the following to the command as an argument: h*. Here the asterisk (*) is recognized by the shell as a metacharacter, one that has special meaning — in this case, “any string of characters zero or more in length.” So ”h*” matches ”heaven”, ”hologram”, and ”h”. Now let’s suppose you wanted to apply a command (let’s use rm) to files beginning with either ”h” or ”c”. You could use the following:
There are two important things to note here. One is the new metacharacter denoted by the square brackets (). This means “any one of the enclosed characters.” So ”[hc]” matches either ”h” or ”c”, as we wanted. The other important concept is that metacharacters can be used together; here we’ve combined the bracketed list with an asterisk.
Filename Metacharacters Recognized in bash:
The bash shell has an extremely extensive list of metacharacters. Some of them resemble, but must not be confused with, the characters used in regular expressions to search text files (this topic is covered in the ITS Document: UNIX/Linux: Data Management. The following table illustrates some metacharacters recognized by bash:
Table 4. bash Filename Metacharacters
|*||any string of characters zero or more in length|
|?||any one character|
Storing Frequently-Used Commands as Aliases
Most UNIX users (indeed, most users of any operating system) find themselves using certain commands more frequently than others. This may be a small number, but they are often the commands that the user executes the vast majority of the time. If any of these commands are rather long or complicated, the user may wish for some way to simplify them and make their use easier. Or perhaps a user is migrating from another operating system (such as MS Windows) and wants to be able to use the same commands. These are both problems that the shell’s command aliasing mechanism can address.
What’s an Alias?
In a previous example, we used ls -al | more to produce a long listing of all files in a directory (including invisible files) and page it to the terminal screen. This is a relatively long and unwieldy command, and one we will probably reuse. So to make it easier to remember and enter, we use an alias. An alias is another name for a command – you can choose something short and mnemonic, type it in, and execute the actual command. To create the alias la for ls -al | more, type this on the command line:
alias la=’ls -al | more’
You can now perform the ls -al | more command simply by typing in la. The basic syntax for the alias command under bash is this:
The two flags available to alias are -t, which tells the shell to store the absolute pathname of the aliased command as part of the alias definition (useful if you have two different versions of the same command installed), and -x, which tells the shell to “export” the alias, thus making it available to shell scripts. If alias is executed without arguments, it then prints out a list of the defined aliases. If you wish to remove any aliases from the list, use:
No Control Panels Here
If you are a Macintosh or Windows user, then you are probably used to having access to your system settings through GUI-based tools (control panels). On a UNIX system, most customization of the user environment is done through manual manipulation of the settings themselves. In the remaining sections of this document we will look at the types of settings, how to adjust them, and how to store your settings permanently.
Types of Settings
There are several different types of settings which you can manipulate within the bash shell. The basic distinction is between options, which can only be turned on or off, and variables, which can be set to a range of values. Each is manipulated in a slightly different way.
Many of the settings involved in tailoring the behavior of the bash shell are like switches: they are either on or off. Such settings are referred to as options. To manipulate these options, either enabling them or disabling them, we use the set command. One useful way to use set is to get a complete listing of the available options and their current settings. To do this, type at the command prompt.
This will give you a listing much like the following:
(The above is a partial listing.)
You will notice that to the right of each option is either “on” or “off” – this shows whether the option is enabled or not. If you want to enable/disable a specific option, you will again need to use the set command:
The important flags to use when manipulating options are -o, which, when followed by the name of an option, turns the option on, for example:
and +o, which turns the following option off. Be careful: many users get confused by setting and option with -o and unsetting it with +o. A complete listing of the options, along with descriptions of what they do, is contained in the “bash man page.” The following table describes some of the more useful ones:
|Option||Effect when set|
|allexport||all variables automatically exported to subshells and scripts|
|emacs||turns on emacs command editing mode|
|ignoreeof||CTRL-D will not log out; must use exit|
|noclobber||output redirection cannot overwrite existing files|
|noglob||filename wildcards are not expanded|
|trackall||all aliases are automatically tracked aliases|
|vi||turns on vi command editing mode|
There are some settings which require more than a yes/no choice: these are shell variables. Typically, a variable refers to a label standing for some value. For example, if we type the following:
This sets the value of the variable “TERM” to “vt100.” The TERM variable is used to tell the system what kind of terminal the user has, so that output will formatted correctly. There are many shell variables, and like options, each one controls some aspect of the user environment. The basic syntax for setting a variable to a value is this:
To retrieve this value, either to use it or just to check it, we can type either of the following command:
Preceding a variable name with the dollar sign ($) causes the shell to replace the variable name with its value – this replacement is known as variable expansion. This particular command should print out the string “vt100,” which is the value of the variable. It is important to remember the following about variable assignment and expansion:
1. When assigning a value to a variable, leave no spaces around the equals sign (=).
2. Precede the variable name with a dollar sign ($) only when performing expansion.
As we have noted, there are many shell variables, which are integral to the shell and its behavior. Among these are the variables which specify your command search path, your home directory, and your default editor. Although these are not created by the user (the shell initializes them at login time), the values of some built-in variables can be altered by the user. Let’s take a look at some of these.
One of the most important variables is your PATH variable. When you type a command, how does the system know where to find it? Remember that on UNIX, although some commands are built directly into the shell, (such as the echo command we saw above), most are actually individual program files themselves. For a command to be executed successfully, the shell must know the path to the program file. The PATH variable is an array variable-variable which contains a list of values, rather than one single value-containing a list of directories. When a user types a command at the shell prompt, the shell searches each of these directories for the program, beginning with the first in the list and proceeding in order. If there are different programs, each having the same name, then the one executed will be the one found by the shell first, regardless of whether that is the one the user intended. So it can be extremely important to know your PATH setting. Since the PATH variable holds an array, not a single value, the initialization syntax is slightly different from that we used before. An example might be
Note that each value is a directory path, and the values are all separated by colons. To add one or more values to the list without having to reinitialize the entire array, use the following:
This is equivalent to saying “PATH equals the value of PATH plus….” Before the new assignment is made, variable expansion is performed, so that the former value of PATH is retained.
Note that the first value is “.” — which means “the current dir.”
The CDPATH setting works much as it did with PATH, but instead of providing the shell a list of places to look for commands, it provides the shell a list of directories in which to look when you type a cd command. For instance, suppose you have a directory under your home directory called ” documents,” in which you have several further subdirectories, each containing a type of document (e.g., papers, programs, letters, etc.). If you are working in another directory, and you need to switch back to one of the document sub-directories, wouldn’t it be convenient to be able to type just the simple directory name, rather than its pathname?
To make this possible, you would need to add the path ” ~/documents” to your CDPATH. Thus, when you try to cd to one of the directories under documents from anywhere else in the filesystem, the shell can look for the directory inside each of the directories specified in CDPATH, as it does with the PATH. This is the default setting for CDPATH:
So, when you type
the cd command looks in the current directory “.” first for the sub-directory ” letters.” If it finds ” letters”, the it cds into ” letters.” If cd does not find ” letters”, it looks in the previous directory “..” and then looks in your home directory “~/” (the tilda “~” evaluates to whatever the path is to your home directory i.e. what the environment variable $HOME is set to). To see what your CDPATH is set to, type:
To add ” ~/documents/” to your CDPATH, type:
If you have a favorite editor (such as vi or emacs), then you should specify its absolute pathname (e.g. /usr/ucb/vi) as the value of the EDITOR variable. This will mean that if you post from “tin,” for example, the editor invoked will be the one you prefer.
COLUMNS and LINES
These two variables, COLUMNS and LINES, tell the shell the dimensions of your terminal window in characters and rows. They may seem to be redundant if you’ve set your TERM variable properly, but it can’t hurt to be sure.
The PS1 variable holds your primary prompt string. Each time you execute a command at the shell prompt, variable expansion is performed upon the value of PS1, with the result being displayed as your new prompt. There are several different pieces of information which might be useful to have in the prompt, such as the current directory, username, or command number.
The prompt variables, such as PS1, are initialized much like any other variable. But there is one important fact to remember: variable expansion is performed on the prompt string every time you execute a command. It is thus possible to embed a variable name in the prompt variable and have the prompt updated after each command. Consider the following prompt initialization:
This initializes PS1 to the string “$PWD$ .” The built-in variable PWD always holds the path to your current working directory, so at each command the prompt string is expanded to show your path as the prompt, followed by the standard bash $ prompt. But why isn’t expansion performed at the time PS1 is first initialized? One would think that PS1 should simply hold the working directory path at the time of initialization and thereafter always show the same string. To understand why this is not the case, we have to briefly discuss the shell quoting conventions. When a variable is contained in double quotes (“x”), the variable will be expanded if passed as an argument to any command. But notice that in the initialization command above, the variable is inside single quotes. Inside single quotes, no variable expansion can take place — the $ is interpreted literally, rather than as denoting a variable name.
Making a Setting Permanent
When you login to your UNIX/Linux account, most of your settings are automatically established by one or more configuration files. In the C shell, there are the files .login and .cshrc files; in the Korn and bash shells, we have the .profile.
The .profile file contains a number of commands which are read in at login just as if you had typed them. While you can always adjust a setting directly from the command line, these adjustments are temporary and will be lost when you logout and login again. To make a setting automatic (and thereby effectively permanent), you will need to place the command on a line in your .profile. To do this will naturally require a basic familiarity with at least one UNIX text editor.
Word of caution: here at UNC, users of ITS machines are not allowed to modify .cshrc and .profile. Rather, they modify the files .cshrc.personal and .profile.personal in directory $HOME/public/.
Back to the UNIX/Linux Getting Started page.