Project 5 - bsh: the Bowdoin Shell

This project should be completed individually or in groups of 2. Groups should only complete and turn in a single program.

This project will help you understand the inner workings of the shell, a program that you have been using all semester. You'll do this by writing your own simple shell program that supports Unix-style job control. Implementing your shell will teach you the core concepts of process control and signaling, as well as give you experience with system programming in C.

This project is more complex than past projects. Plan and budget your time accordingly!

Start by reading through the entire project description!

Project Overview

A shell is an interactive command-line interpreter that runs programs on behalf of the user. At a high level, a shell repeatedly prints a prompt, waits for a program name and command-line arguments on stdin, then carries out some action as directed by the input.

The shell program that you have been using all semester is bash (the Bourne Again Shell). Bash is only one of many shell programs, however - others include sh, tcsh, and csh. In this project, you will implement your own shell, bsh (the Bowdoin Shell).

Unix Shell Basics

A command-line string is a sequence of text words delimited by whitespace. The first word of the string is either the pathname of an executable file (i.e., a program) or a built-in command. The remaining words are command-line arguments. If the first word is a built-in command, the shell immediately executes the command within the current shell process. Otherwise, the shell forks a child process, then executes the specified program in the context of the child. The child processes created as a result of interpreting a single command are known as a job. A job can consist of multiple child processes connected by Unix pipes (denoted in a command by vertical bars, |).

By default, the job runs in the foreground, which means that the shell waits for the job to terminate before prompting for the next command string. Thus, at any point in time, at most one job can be running in the foreground. However, if the command string ends with an ampersanad (&), then the job runs in the background, which means that the shell does not wait for the job to terminate before printing another prompt and waiting for another command string. Thus, an arbitrary number of jobs can be running in the background at a given time.

Typing the following command runs the program ls (located in the directory /bin) in the foreground with command line arguments -l -d:

bsh> /bin/ls -l -d

Note that more specifically, calling the above will execute the main function of /bin/ls with the following values of argc and argv:

• argc is 3
• argv[0] is '/bin/ls'
• argv[1] is '-l'
• argv[2] is '-d'

Alternately, typing the same command with an ampersand will run ls in the background:

bsh> /bin/ls -l -d &

Job Control

Unix shells support the notion of job control, which allows users to move jobs back and forth between background and foreground, and to change the process state (running, stopped, or terminated) of all the processes in a job. Job states can be changed via signals: typing Ctrl-Z causes a SIGTSTP signal to be delivered to every process in the foreground job. The default action for SIGTSTP is to place the process in the stopped state, where it remains until it is awakened by the receipt of a SIGCONT signal. Typing Ctrl-C causes a SIGINT signal to be delivered to each process in the foreground job. The default action for SIGINT is to terminate the process.

Unix shells also provide various built-in commands that support job control. Key commands are listed below:

• jobs: List the running and stopped background jobs.
• bg <job>: Change a stopped background job to a running background job.
• fg <job>: Change a stopped or running background job to a running job in the foreground.
• kill <job>: Terminate a job (more specifically, sends a SIGTERM signal to the job, for which the default behavior is to terminate the process).

The bsh Specification

The bsh shell should have the following features:

• The prompt should be the string "bsh> ".
• As described previously, the command string typed by the user should be either a built-in command or a program name, possibly following by command-line arguments. Programs should be executed in the context of a child process forked by the shell.
• Typing Ctrl-C should cause a SIGINT signal to be sent to the current foreground job (i.e., the initial child that was forked for that job as well as any descendant processes of that child). Typing Ctrl-Z should work the same except that the signal sent is SIGTSTP.
• If the command string ends with &, the job should be run in the background. Otherwise, it should run in the foreground.
• Each job can be identified either by a process ID (PID) or by a job ID (JID). PIDs are positive integers assigned by the operating system when processes are created. JIDs are positive integers assigned by bsh to each job. On the command line, a JID is denoted by the prefix '%'. For instance, '%5' denotes JID 5, while '5' denotes PID 5.
• bsh should support the following built-in commands:
  
  
  • quit: Terminate the shell.
  • jobs: List all background jobs.
  • bg <job>: Restarts <job> by sending it a SIGCONT signal, then runs it in the background. The job argument can be either a PID or a JID.
  • fg <job>: Restarts <job> by sending it a SIGCONT signal, then runs it in the foreground. The job argument can be either a PID or a JID.
• You do not need to support pipes (|) or I/O redirection (< and >) in your shell.

Code Structure

To start, you have been provided with a functional skeleton of the shell. The starting code implements a number of less interesting functions that you should use while implementing the complete shell, allowing you to focus on the more interesting components. In particular, you are responsible for implementing each of the empty functions listed below. As a sanity check, also listed below is the number of code lines implementing each function in the reference shell (which includes comments):

• eval: Main routine that parses and interprets the command line. [70 lines]
• builtin_cmd: Recognizes and interprets the built-in commands listed above (bg and fg commands should result in calling do_bgfg as below). [25 lines]
• do_bgfg: Implements the bg and fg built-in commands. [50 lines]
• waitfg: Waits for a foreground job to complete. [20 lines]
• sigchld_handler: Handler for SIGCHILD signals. [80 lines]
• sigint_handler: Handler for SIGINT (Ctrl-C) signals. [15 lines]
• sigtstp_handler: Handler for SIGTSTP (Ctrl-Z) signals. [15 lines]

Note: While the function lengths given above are fairly modest, don't be lulled into a false sense of security! System programming involves writing dense, precise, and often error-prone code, and is likely to require significant debugging time. Be sure to give yourself plenty of time!

The file you should modify that contains the code of your shell is bsh.c. The included Makefile will compile the shell for you. To run your shell, simply execute it:

unix> ./bsh
bsh> [type commands to your shell here]

Included Files

You have also been provided with a number of tools to help you check your work. All included files are described below:

• bsh.c: The code of your shell.
• bshref: The reference shell. Run this program if you have any questions about how your shell should behave. Your shell should emit identical output to the reference solution (except for PIDs, which change from run to run).
• sdriver.pl: A shell driver program that executes the shell and feeds it commands and signals as directed by a trace file, then captures and displays the output from the shell.
• trace{01-16}.txt: 16 trace files that you will use in conjunction with the shell driver to test the correctness of your shell. The lower-numbered trace files do very simple tests, while the higher-numbered tests do more complicated tests.
• bshref.out: The output of the reference solution on all traces, for your reference. This might be more convenient than manually running the shell driver on all trace files.
• myspin.c: A test program that sleeps for a specified number of seconds.
• mysplit.c: A test program that forks a child, which then sleeps for a specified number of seconds.
• mystop.c: A test program that sleeps for a specified number of seconds, then sends a SIGTSTP signal to itself.
• myint.c: A test program that sleeps for a specified number of seconds, then sends a SIGINT signal to itself.
• Makefile: Builds the shell and all test programs. Also provides useful targets for testing the shell (see below).

Use the -h flag to see the usage string for sdriver.pl:

unix> ./sdriver.pl -h
Usage: ./sdriver.pl [-hv] -t <trace> -s <shellprog> -a <args>
Options:
  -h            Print this message
  -v            Be more verbose
  -t <trace>    Trace file
  -s <shell>    Shell program to test
  -a <args>     Shell arguments

You can run the shell driver on trace01.txt by typing the following:

unix> ./sdriver.pl -t trace01.txt -s ./bsh -a "-p"

Similarly, to compare your result with the reference shell, you can run the trace driver on the reference shell by simply substituting bsh with bshref in the command above.

More simply, you can use the included Makefile to run the driver on the trace files. To pass trace01.txt through your shell, you can just run:

unix> make test01

Similarly, to pass trace01.txt through the reference shell, you can run:

unix> make rtest01

The output of your shell from the trace files is exactly the same as the output you would have gotten from running your shell interactively, except for an initial comment that identifies the trace.

Advice

Here are some general (and specific!) tips for working on your shell:

• Chapter 8 of the textbook (Exceptional Control Flow) is an excellent reference for this project, particularly sections 8.4 and 8.5.
• Use the trace files to guide the development of your shell. Start with trace01.txt and make sure that your shell produces output that is identical to that of the reference shell. Then move onto trace02.txt, and so forth.
• Useful system calls that you will want to use include fork, execve, getpid, waitpid, kill, setpgid, and sigprocmask. The situations in which the latter two will be useful are mentioned below. The WUNTRACED and WNOHANG options to waitpid will also be useful.
• When you implement your signal handlers, be sure to send SIGINT and SIGTSTP signals to the entire foreground process group, using "-pid" instead of "pid" in the argument to the kill function. Note that sdriver.pl tests for this error.
• One of the tricky parts of the shell design is deciding on the allocation of work between waitfg and sigchld_handler. A recommended approach is the following:
  
  
  • In waitfg, loop over the sleep function as long as needed (this is called a busy-wait and normally isn't a good idea, but here is appropriate). Note that this is the only place in your code where using sleep sleep is acceptable.
  • In sigchld_handler, use exactly one call to waitpid.
  While other approaches are possible, it is simpler to do all reaping in the handler as recommended above.
• In eval, the parent must use sigprocmask to block SIGCHLD, SIGINT, and SIGTSTP signals before it forks the child, and then unblock these signals (again using sigprocmask) after it adds the child to the job list by calling addjob. If these signals are not blocked, it is possible that the child exits and is reaped by sigchld_handler (and thus removed from the job list) before the parent has actually called addjob. This type of bug is called a race condition, as it is a possible error depending on two processes "racing" during concurrent execution. Race conditions are nasty to debug because they occur nondeterministically!
  
  Also related to the above issue, note that children inherit the blocked vectors of their parents, and therefore the child must be sure to unblock its signals before it executes the new program.
• Full-window programs like more, less, nano, vi, and emacs do strange things with the terminal settings. Don't run these programs from your shell - instead, stick with simple text-based programs like /bin/ls, /bin/ps, and /bin/echo.
• When you run your shell from the standard Unix shell, your shell is running in the foreground process group. If your shell then creates a child process, that child will also be a member of the foreground process group. Since typing Ctrl-C sends a SIGINT to every process in the foreground group, doing so will send a SIGINT to your shell (which is good), but also to every process that your shell created (which is bad).
  
  To work around this, after calling fork but before the child calls execve, the child should call setpgid(0, 0), which puts the child in a new process group whose group ID is equal to the child's PID. This ensures that there will be only one process (your shell) in the foreground process group. When you type Ctrl-C, the shell should catch the resulting SIGINT and then forward it to the appropriate foreground job (or more precisely, the process group that contains the foreground job).
• As discussed in class, be careful about what code you put in signal handler functions. For example, you should not use printf in signal handlers. You can use the following as a "safe" printf for use in handlers:

  /* safe_printf – async-signal-safe wrapper for printf */
  void safe_printf(const char *format, ...) {
      char buf[MAXBUF];
      va_list args;
      sigset_t mask, prev_mask;
      sigfillset(&mask);
      sigprocmask(SIG_BLOCK, &mask, &prev_mask);
      va_start(args, format);
      vsnprintf(buf, sizeof(buf), format, args);
      va_end(args);
      write(1, buf, strlen(buf));
      sigprocmask(SIG_SETMASK, &prev_mask, NULL);
  }

• You can still (and are encouraged to) use gdb for debugging your shell. To debug a multi-process program, you will need a few extra gdb commands:

  (gdb) set detach-on-fork on/off

  The above command sets whether child processes will be detached when fork is called. The default is on (i.e., the child runs without any interruption). If you turn this option off, the child is suspended as soon as it is forked. Another useful option is the following:

  (gdb) set follow-fork-mode parent/child

  The above decides which process gdb will follow after fork is called. To switch between difference processes manually, you can use the inferior command:

  (gdb) info inferiors
  ... listing of processes ...
  (gdb) inferior 1
  [Switching to inferior 1 [process 0] (<noexec>)]
  

Logistics

You are responsible for completing the contents of the bsh.c file. You should not modify any other file. You are responsible for ensuring that your program runs on the class server, regardless of where else you may be writing code.

As usual, your final submission will consist of your committed bsh.c file at the time of the due date. Each group will make only one submission, which should be committed to the group's shared SVN folder (not the SVN folders of the individual group members).

Note for groups: If you are working in a group, you must individually send me an email giving me a brief description of your and your group member's contribution to the project after your submission. The purpose of this requirement is to encourage full group participation on the project. Your email does not need to be detailed, but your project is not considered complete until your summary is received. Your email will be kept confidential by me and will not be shared (if I have any concerns, I will reach out to you).

Evaluation

You will be evaluated both on the correctness of your shell implementation (as determined by the 16 trace files) as well as the style and overall quality of your program (as determined by me).

Particular things to watch out for:

• For full credit, your program must compile without any warnings on the class server.
• Your program must also check the return value of every system call!
• Your shell will be tested on the class server, where it should product identical output to the reference shell, with two exceptions:
  
  
  • The PIDs can (and will) be different.
  • The output of the /bin/ps commands in traces 11, 12, and 13 will be different from run to run. However, the running states of any mysplit processes in the ps output should be identical.

In addition, you should follow all standard and sensible style guidelines, such as using good variable names, commenting, using consistent indentation, etc. While you do not have to adhere to any particular set of conventions (e.g., where to put parentheses), you SHOULD be consistent with whatever conventions you choose. If you are working with a partner, agree on a single set of conventions and stick to them!

You do not need to define any functions beyond those already specified in bsh.c, but you are welcome to do so if you wish.

Please ask if you have any questions about what contitutes good style or what is expected!