Copyright © 2000 Paul Sheer - Click here for copying permissions       Source by FTP

Processes and environment variables

Add comments here

Introduction

On UNIX, when you run a program (like any of the shell commands you have been using) the actual computer instructions are read out of a file on disk out of one of the /bin/ directories and placed in RAM^12.1. The program then gets executed in memory and becomes a process. A process is some command/program/shell-script that is being run (or executed) in memory. When the process is finished running, it is removed from memory. There are usually about 50 processes running simultaneously at any one time on a system with one person logged in. The CPU^12.2 hops between each of them to give a share of its execution time^12.3.Each process is given a process number called the PID (Process ID). Besides the memory actually occupied by the process, the process itself ceases addition memory for its operations.

In the same way that a file is owned by a particular user and group, a process also has an owner -- usually the person who ran the program. Whenever a process tries to access a file, its ownerships is compared to that of the file to decide if the access is permissable. Because all devices are files, the only way a process can do anything is through a file, and hence file permission restrictions are the only kind of restrictions there need ever be on UNIX. This is how UNIX security works.

Tutorial

Login on a terminal and type the command ps. You should get some output like:

PID TTY STAT  TIME COMMAND 5995   2 S    0:00 /bin/login -- myname 5999   2 S    0:00 -bash 6030   2 R    0:00 ps

ps with no options shows 3 processes to be running. These are the only three processes visible to you as a user, although there are other system processes not belonging to you. The first process was the program that logged you in by displaying the login prompt and requesting a password. It then ran a second process call bash, the Bourne Again Shell^12.4 where you have been typing commands. Finally you ran ps, hence it must have found itself when it checked for what processed were running, but then exited immediately afterward.

Controlling jobs

The shell has many facilities for controlling and executing processes -- this is called job control. Create a small script called proc.sh:

#!/bin/sh echo "proc.sh: is running" sleep 1000

Run the script with chmod 0755 proc.sh and then ./proc.sh. The shell will block waiting for the process to exit. Now hit ^Z^12.5. This will stop the process. Now do a ps again. You will see your script listed. However it is not presently running because it is in the condition of being stopped. Type bg standing for background. The script will now be un-stopped and run in the background. You can now run other processes in the mean time. Type fg and the script will return to the foreground. You can then type ^C to interrupt the process.

Creating background processes

Create a program that does something a little more interesting:

5	#!/bin/sh echo "proc.sh: is running" while true ; do         echo -e '\a'         sleep 2 done

Now perform the ^Z, bg, fg and ^C operations from before. To put a process immediately into the background, you can use:

./proc.sh &

The JOB CONTROL section of the bash man page (bash1) looks like this:

 
Job control refers to the ability to selectively stop (suspend) the execution of processes and continue (resume) their execution at a later point. A user typically employs this facility via an interactive interface supplied jointly by the system's terminal driver and bash.

The shell associates a job with each pipeline. It keeps a table of currently executing jobs, which may be listed with the jobs command. When bash starts a job asynchronously (in the background), it prints a line that looks like:

     [1] 25647

indicating that this job is job number 1 and that the process ID of the last process in the pipeline associated with this job is 25647. All of the processes in a single pipeline are members of the same job. Bash uses the job abstraction as the basis for job control.

To facilitate the implementation of the user interface to job control, the system maintains the notion of a current terminal process group ID. Members of this process group (processes whose process group ID is equal to the current terminal process group ID) receive keyboard-generated signals such as SIGINT. These processes are said to be in the foreground. Background processes are those whose process group ID differs from the terminal's; such processes are immune to keyboard-generated signals. Only foreground processes are allowed to read from or write to the terminal. Background processes which attempt to read from (write to) the terminal are sent a SIGTTIN (SIGTTOU) signal by the terminal driver, which, unless caught, suspends the process.

If the operating system on which bash is running supports job control, bash allows you to use it. Typing the suspend character (typically ^Z, Control-Z) while a process is running causes that process to be stopped and returns you to bash. Typing the delayed suspend character (typically ^Y, Control-Y) causes the process to be stopped when it attempts to read input from the terminal, and control to be returned to bash. You may then manipulate the state of this job, using the bg command to continue it in the background, the fg command to continue it in the foreground, or the kill command to kill it. A ^Z takes effect immediately, and has the additional side effect of causing pending output and typeahead to be discarded.

There are a number of ways to refer to a job in the shell. The character % introduces a job name. Job number n may be referred to as %n. A job may also be referred to using a prefix of the name used to start it, or using a substring that appears in its command line. For example, %ce refers to a stopped ce job. If a prefix matches more than one job, bash reports an error. Using %?ce, on the other hand, refers to any job containing the string ce in its command line. If the substring matches more than one job, bash reports an error. The symbols %% and %+ refer to the shell's notion of the current job, which is the last job stopped while it was in the foreground. The previous job may be referenced using %-. In output pertaining to jobs (e.g., the output of the jobs command), the current job is always flagged with a +, and the previous job with a -.

Simply naming a job can be used to bring it into the foreground: %1 is a synonym for ``fg %1'', bringing job 1 from the background into the foreground. Similarly, ``%1 SPMamp;'' resumes job 1 in the background, equivalent to ``bg %1''.

The shell learns immediately whenever a job changes state. Normally, bash waits until it is about to print a prompt before reporting changes in a job's status so as to not interrupt any other output. If the -b option to the set builtin command is set, bash reports such changes immediately. (See also the description of notify variable under Shell Variables above.)

If you attempt to exit bash while jobs are stopped, the shell prints a message warning you. You may then use the jobs command to inspect their status. If you do this, or try to exit again immediately, you are not warned again, and the stopped jobs are terminated.

killing a process, sending a process a signal

To terminate a process, use the kill command:

kill <PID>

The kill command actually sends a signal to the process causing it to execute some function. In some cases, the developers would not have bothered to account for this signal and some default behaviour happens.

To send a signal to a process you can name the signal on the command-line or use its numerical equivalent:

kill -SIGTERM 12345

or

kill -15 12345

Which is the signal that kill normally sends: the termination signal.

To unconditionally terminate a process:

kill -SIGKILL 12345

or

kill -9 12345

Which should only be used as a last resort.

It is cumbersome to have to constantly look up the PID of a process. Hence the GNU utilities have a command killall which sends a signal to all processes of the same name:

killall -<signal> <process_name>

This is useful when you are sure that there is only one of a process running, either because there is no one else logged in on the system, or because you are not logged in as super user.

The list of signals can be gotten from signal7.

List of comman signals

SIGHUP

Hang up. If the terminal becomes disconnected from a process, this signal is sent automatically to the process. Sending a process this signal often causes it to reread its configuration files, so it is useful instead of restarting the process. Always check the man page to see if a process has this behaviour.

SIGINT

Interrupt from keyboard. Issued if you press ^C.

SIGQUIT

Quit from keyboard. Issued if you press ^D.

SIGFPE

Floating Point Exception. Issued automatically to a program performing some kind of illegal mathematical operation.

SIGKILL

Kill Signal. This is one of the signals that can never be caught by a process. If a process gets this signal it has to quit immediately and will not perform any clean-up operations (like closing files or removing temporary files). You can send a process a SIGKILL signal if there is no other means of destroying it.

SIGSEGV

Segmentation Violation. Issued automatically when a process tries to access memory outside of its allowable address space. I.e. equivalent to a Fatal Exception under Windows. Note that programs with bugs or programs in the process of being developed often get these. A program receiving a SIGSEGV however can never cause the rest of the system to be compromised. If the kernel itself were to receive such an error, it would cause the system to come down, but such is extremely rare.

SIGPIPE

Pipe died. A program was writing to a pipe, the other end of which is no longer available.

SIGTERM

Terminate. Cause the program to quit gracefully

Niceness of processes, and scheduling priority

All processes are allocated execution time by the kernel. If all processes were allocated the same amount of time, performance would obviously get worse as the number of processes increased. The kernel uses heuristics^12.6 to guess how much time each process should be allocated. The kernel tries to be fair, hence when two users are competing for CPU usage, they should both get the same.

Most processes spend their time waiting for either a key press, or some network input, or some device to send data, or some time to elapse. They hence do not consume CPU.

On the other hand, when more than one process runs flat out, it can be difficult for the kernel to decide if it should be given greater priority, than another process. What if a process is doing some more important operation than another process? How does the kernel tell? The answer is the UNIX feature of scheduling priority or niceness. Scheduling priority ranges from +20 to -20. You can set a process's niceness with the renice command.

renice <priority> <pid> renice <priority> -u <user> renice <priority> -g <group>

A typical example is the SETI^12.7 program. Set its priority to +19 with:

renice +19 <pid>

to make it disrupt your machine as little as possible.

Note that nice values have the reverse meaning that you would expect: +19 means a process that eats little CPU, while -19 is a process that eats lots. Only superuser can set processes to negative nice values.

Mostly multimedia applications and some device utilities will be the only processes that need negative renicing, and most of these will have their own command-line options to set the nice value. See for example cdrecord1 and mikmod1 -- a negative nice value will prevent skips in your playback^12.8.

Also useful is the -u and -g options which set the priority of all the processes that a user or group owns.

Further, there is the nice command which starts a program under a defined niceness relative to the current nice value of the present user. For example

nice +<priority> <pid> nice -<priority> <pid>

Finally, there is the snice command which can set, but also display the current niceness. This command doesn't seem to work.

snice -v <pid>

Listing process CPU/Memory consumption with top

The top command sorts all process by their CPU and memory consumption and displays top twenty or so in a table. top is to be used whenever you want to see whats hogging your system. top -q -d 2 is useful for scheduling the top command itself to a high priority, so that it is sure to refresh its listing without lag. top -n 1 -b > top.txt is useful for listing all process, and top -n 1 -b -p <pid> prints info on one process.

top has some useful interactive responses to key presses:

f

Shows a list of displayed fields that you can alter interactively. By default the only fields shown are USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME COMMAND which is usually what you are most interested in. (The field meanings are given below.)

The top man page describes the field meanings. Some of these are confusing and assume knowledge of the internals of C programs. The main question people ask is: How much memory is a process using? This is given by the RSS field, which stands for Resident Set Size. RSS means the amount of RAM that a process consumes alone. The following show totals for all process running on my system (which had 65536 kilobytes RAM at the time.). They represent the total of the SIZE, RSS and SHARE fields respectively.

5          10

echo `echo '0 ' ; top -q -n 1 -b | sed -e '1,/PID *USER *PRI/D' | \     awk '{print "+" $5}' | sed -e 's/M/\\*1024/'` | bc 68016 echo `echo '0 ' ; top -q -n 1 -b | sed -e '1,/PID *USER *PRI/D' | \     awk '{print "+" $6}' | sed -e 's/M/\\*1024/'` | bc 58908 echo `echo '0 ' ; top -q -n 1 -b | sed -e '1,/PID *USER *PRI/D' | \     awk '{print "+" $7}' | sed -e 's/M/\\*1024/'` | bc 30184

The SIZE represents the total memory usage of a process. RSS is the same, but excludes memory not needing actual RAM (this would be memory swapped to the swap partition). SHARE is the amount shared between processes.

Other fields are described by the top man page as:

uptime

This line displays the time the system has been up, and the three load averages for the system. The load averages are the average number of process ready to run during the last 1, 5 and 15 minutes. This line is just like the output of uptime(1). The uptime display may be toggled by the interactive l command.

Environment's of processes

Each process that runs does so with the knowledge of several var=value text pairs. All this means is that a process can look up the value of some variable that it may have inherited from its parent process. The complete list of these text pairs is called the environment of the process, and each var is called an environment variable. Each process has its own environment, which is copied from the parent processes environment.

After you have logged in and have a shell prompt, the process you are using (the shell itself) is just like any other process with an environment with environment variables. To get a complete list of these variables, just type:

set

This is useful to find the value of an environment variable whose name you are unsure of:

set | grep <regexp>

Try set | grep PATH to see the PATH environment variable discussed previously.

The purpose of an environment is just to have an alternative way of passing parameters to a program (in addition to command-line arguments). The difference is that an environment is inherited from one process to the next: i.e. a shell might have certain variable set (like the PATH) and may run a file manager which may run a word-processor. The word-processor inherited its environment from file-manager which inherited its environment from the shell.

Try

X="Hi there!" echo $X

You have set a variable. But now run

bash

You have now run a new process which is a child of the process you were just in. Type

echo $X

You will see that X is not set. This is because the variable was not exported as an environment variable, and hence was not inherited. Now type

exit

Which returns you to the parent process. Then

export X bash echo $X

You will see that the new bash now knows about X.

Above we are setting an arbitrary variable for our own use. bash (and many other programs) automatically set many of their own environment variables. The bash man page lists these (when it talks about unsetting a variable, it means using the command unset <variable>). You may not understand some of these at the moment, but they are included here as a complete reference for later:

Shell Variables

The following variables are set by the shell:

PPID

The process ID of the shell's parent.

PWD

The current working directory as set by the cd command.

OLDPWD

The previous working directory as set by the cd command.

REPLY

Set to the line of input read by the read builtin command when no arguments are supplied.

UID

Expands to the user ID of the current user, initialized at shell startup.

EUID

Expands to the effective user ID of the current user, initialized at shell startup.

BASH

Expands to the full pathname used to invoke this instance of bash.

BASH_VERSION

Expands to the version number of this instance of bash.

SHLVL

Incremented by one each time an instance of bash is started.

There are also many variables that bash uses which may be set by the user. These are:

The following variables are used by the shell. In some cases, bash assigns a default value to a variable; these cases are noted below.

IFS

The Internal Field Separator that is used for word splitting after expansion and to split lines into words with the read builtin command. The default value is ``<space><tab><newline>''.

PATH

The search path for commands. It is a colon-separated list of directories in which the shell looks for commands (see COMMAND EXECUTION below). The default path is system-dependent, and is set by the administrator who installs bash. A common value is ``/usr/gnu/bin:/usr/local/bin:/usr/ucb:/bin:/usr/bin:.''.

HOME

The home directory of the current user; the default argument for the cd builtin command.

CDPATH

The search path for the cd command. This is a colon-separated list of directories in which the shell looks for destination directories specified by the cd command. A sample value is ``.:~:/usr''.

ENV

If this parameter is set when bash is executing a shell script, its value is interpreted as a filename containing commands to initialize the shell, as in .bashrc. The value of ENV is subjected to parameter expansion, command substitution, and arithmetic expansion before being interpreted as a pathname. PATH is not used to search for the resultant pathname.

MAIL

If this parameter is set to a filename and the MAILPATH variable is not set, bash informs the user of the arrival of mail in the specified file.

MAILCHECK

Specifies how often (in seconds) bash checks for mail. The default is 60 seconds. When it is time to check for mail, the shell does so before prompting. If this variable is unset, the shell disables mail checking.

MAILPATH

A colon-separated list of pathnames to be checked for mail. The message to be printed may be specified by separating the pathname from the message with a `?'. $_ stands for the name of the current mailfile. Example:
MAILPATH='/usr/spool/mail/bfox?"You have mail":~/shell-mail?"$_ has mail!"'
Bash supplies a default value for this variable, but the location of the user mail files that it uses is system dependent (e.g., /usr/spool/mail/$USER).

MAIL_WARNING

If set, and a file that bash is checking for mail has been accessed since the last time it was checked, the message ``The mail in mailfile has been read'' is printed.

PS1

The value of this parameter is expanded (see PROMPTING below) and used as the primary prompt string. The default value is ``bash\$ ''.

PS2

The value of this parameter is expanded and used as the secondary prompt string. The default is ``> ''.

PS3

The value of this parameter is used as the prompt for the select command (see SHELL GRAMMAR above).

PS4

The value of this parameter is expanded and the value is printed before each command bash displays during an execution trace. The first character of PS4 is replicated multiple times, as necessary, to indicate multiple levels of indirection. The default is ``+ ''.

HISTSIZE

The number of commands to remember in the command history (see HISTORY below). The default value is 500.

HISTFILE

The name of the file in which command history is saved. (See HISTORY below.) The default value is ~/.bash_history. If unset, the command history is not saved when an interactive shell exits.

HISTFILESIZE

The maximum number of lines contained in the history file. When this variable is assigned a value, the history file is truncated, if necessary, to contain no more than that number of lines. The default value is 500.

OPTERR

If set to the value 1, bash displays error messages generated by the getopts builtin command (see SHELL BUILTIN COMMANDS below). OPTERR is initialized to 1 each time the shell is invoked or a shell script is executed.

PROMPT_COMMAND

If set, the value is executed as a command prior to issuing each primary prompt.

IGNOREEOF

Controls the action of the shell on receipt of an EOF character as the sole input. If set, the value is the number of consecutive EOF characters typed as the first characters on an input line before bash exits. If the variable exists but does not have a numeric value, or has no value, the default value is 10. If it does not exist, EOF signifies the end of input to the shell. This is only in effect for interactive shells.

TMOUT

If set to a value greater than zero, the value is interpreted as the number of seconds to wait for input after issuing the primary prompt. Bash terminates after waiting for that number of seconds if input does not arrive.

FCEDIT

The default editor for the fc builtin command.

FIGNORE

A colon-separated list of suffixes to ignore when performing filename completion (see READLINE below). A filename whose suffix matches one of the entries in FIGNORE is excluded from the list of matched filenames. A sample value is ``.o:~''.

INPUTRC

The filename for the readline startup file, overriding the default of ~/.inputrc (see READLINE below).

notify

If set, bash reports terminated background jobs immediately, rather than waiting until before printing the next primary prompt (see also the -b option to the set builtin command).

history_control

HISTCONTROL

If set to a value of ignorespace, lines which begin with a space character are not entered on the history list. If set to a value of ignoredups, lines matching the last history line are not entered. A value of ignoreboth combines the two options. If unset, or if set to any other value than those above, all lines read by the parser are saved on the history list.

command_oriented_history

If set, bash attempts to save all lines of a multiple-line command in the same history entry. This allows easy re-editing of multi-line commands.

glob_dot_filenames

If set, bash includes filenames beginning with a `.' in the results of pathname expansion.

allow_null_glob_expansion

If set, bash allows pathname patterns which match no files (see Pathname Expansion below) to expand to a null string, rather than themselves.

histchars

The two or three characters which control history expansion and tokenization (see HISTORY EXPANSION below). The first character is the history expansion character, that is, the character which signals the start of a history expansion, normally `!'. The second character is the quick substitution character, which is used as shorthand for re-running the previous command entered, substituting one string for another in the command. The default is `^'. The optional third character is the character which signifies that the remainder of the line is a comment, when found as the first character of a word, normally `#'. The history comment character causes history substitution to be skipped for the remaining words on the line. It does not necessarily cause the shell parser to treat the rest of the line as a comment.

nolinks

If set, the shell does not follow symbolic links when executing commands that change the current working directory. It uses the physical directory structure instead. By default, bash follows the logical chain of directories when performing commands which change the current directory, such as cd. See also the description of the -P option to the set builtin ( SHELL BUILTIN COMMANDS below).

hostname_completion_file

HOSTFILE

Contains the name of a file in the same format as /etc/hosts that should be read when the shell needs to complete a hostname. The file may be changed interactively; the next time hostname completion is attempted bash adds the contents of the new file to the already existing database.

noclobber

If set, bash does not overwrite an existing file with the >, >&, and <> redirection operators. This variable may be overridden when creating output files by using the redirection operator >| instead of > (see also the -C option to the set builtin command).

auto_resume

This variable controls how the shell interacts with the user and job control. If this variable is set, single word simple commands without redirections are treated as candidates for resumption of an existing stopped job. There is no ambiguity allowed; if there is more than one job beginning with the string typed, the job most recently accessed is selected. The name of a stopped job, in this context, is the command line used to start it. If set to the value exact, the string supplied must match the name of a stopped job exactly; if set to substring, the string supplied needs to match a substring of the name of a stopped job. The substring value provides functionality analogous to the %? job id (see JOB CONTROL below). If set to any other value, the supplied string must be a prefix of a stopped job's name; this provides functionality analogous to the % job id.

no_exit_on_failed_exec

If this variable exists, a non-interactive shell will not exit if it cannot execute the file specified in the exec builtin command. An interactive shell does not exit if exec fails.

cdable_vars

If this is set, an argument to the cd builtin command that is not a directory is assumed to be the name of a variable whose value is the directory to change to.