CSCI 3310
Operating Systems

Bowdoin College
Fall 2022
Instructor: Sean Barker

Project 3 - Thread Library

Assigned:Wednesday, October 12.
Groups/Repos Due:Friday, October 14, 11:59 pm.
Code Due:Tuesday, November 1, 11:59 pm.
Writeup Due:Thursday, November 3, 11:59 pm.
Collaboration Policy:Level 1
Group Policy:Pair-optional (but recommended!)

In the previous project, you used a user-level thread library to implement a concurrent disk scheduler. In this project, you will implement the thread library itself! Your thread library will provide exactly the same API that you used in your disk scheduler, and will allow you to run any concurrent program (including the disk scheduler) that uses this API.

Implementing the thread library is considerably more complex than writing the disk scheduler. Additionally, this project will require you to write and submit tests along with your library. Plan appropriately and start early!

Project Overview

Your task is to implement the full thread library API as detailed in Project 2. In particular, you must write each of the thread_ operations defined in the thread.h header used by all clients of the thread library.

Note that while you will need to implement the main threading functions, you will still be provided with the interrupt library, meaning that you will not need to implement the start_preemptions function. Additionally, your thread library implementation will use two other functions provided by the interrupt library that are not part of the user-facing thread API (interrupt_enable and interrupt_disable) to facilitate atomicity by controlling interrupts.

One of the challenges of this project will be understanding the representation of threads and the infrastructure provided by Linux to support user-level thread libraries. Other challenges will include providing appropriate atomicity within the thread library and ensuring robustness against arbitrary (and potentially invalid) usage by clients of the library.

Threading Specification

In addition to implementing the publicly-defined interface of thread.h, your library should follow the specifications given below.

Scheduling Order

Your library should follow these rules when deciding how to order threads:


When there are no runnable threads in the system (e.g. all threads have finished, or all threads are deadlocked), your thread library should execute the following code to terminate the program:

    cout << "Thread library exiting.\n";

This message is the only output the thread library itself should ever produce. Do not send any other output to cout from within the thread library.

Error Handling

As defined in the thread library API, all functions should return 0 on success (except for thread_libinit) and -1 on failure. Your functions must be as robust as possible, and should handle every possible error without crashing.

This specification intentionally does not provide you with an exhaustive list of errors you should handle. OS programmers must have a healthy sense of paranoia to make their system robust, so part of this project is thinking of and handling lots of different error types. A few types of errors are not possible to handle due to the thread library existing in userspace rather than the kernel (thus, for example, a misbehaving user program could corrupt the memory used by the thread library). However, most types of errors should be gracefully caught by the library.

The following actions that might be taken by a user program are highly questionable but should not be considered errors by the thread library:

Questionable behaviors that should be considered errors (and therefore return -1) include the following:

Ask if you're unsure about whether any other specific behaviors should be considered errors. Note that errors can also arise from factors other than invalid client usage (e.g., running out of memory). Some possible sources of errors that you might overlook are highlighted in the implementation advice section.

Remember that all types of errors should be handled silently, returning -1 but not producing any output.

Memory Usage

The library should not leak memory over time as threads are created and destroyed. After a thread is finished (i.e., after it returns from the function given in thread_create), you must remember to deallocate the memory used for the thread and its stack. Deallocation does not need to happen immediately after thread termination so long as finished threads do not pile up over time without deallocation.

Test Cases

While methodical testing should be a part of any development process, testing will be a required and evaluated part of your thread library. In particular, you will submit a suite of test cases along with your thread library, and the autograder will evaluate both your library and your tests.

Each test case will be a C++ program that uses functions in the thread library (e.g. the example program from Project 2). Each test case should not expect any command-line arguments, should not read any input files, and should call exit(0) when run with a correct thread library (which normally happens when your test case's last runnable thread ends or blocks). If you wish to use your disk scheduler as a test case, you will need to specify all inputs (number of requesters, queue size, and the list of requests) statically inside the program in order to obey these rules. Finally, your test cases should not call start_preemption, as your test suite will not be evaluated on how thoroughly it exercises the interrupt_enable and interrupt_disable calls in the library.

Your test suite may contain up to 20 test cases, and each test case may generate at most 10 KB of output and may take up to 60 seconds to run (though these limits are much larger than needed).

The autograder includes a number of buggy thread libraries that misbehave in various ways. Your test suite will be autograded based on how many of the buggy thread libraries are 'exposed' by your test suite. A buggy library is considered exposed by a test case if the output produced by that test case differs when run using the buggy library versus a correct library. Your goal is to expose as many of the buggy libraries as possible using your entire test suite. Similarly to the regular test cases, the bugs within each of the buggy libraries will remain hidden.


This section contains information on implementing various parts of the thread library.

Thread Creation and Context Switching

Linux provides several library calls to help implement user-level thread libraries. The calls you will need are getcontext, makecontext, setcontext, and swapcontext. These calls interact with ucontext_t structs, which contain the information comprising a thread (stack, program counter, register values, and so forth). Shown below is an example of creating a new user-level thread using getcontext and makecontext. To actually run another thread, you can use swapcontext to save the context of the current thread and switch to the context of another thread, or setcontext to set the thread context without saving an existing context. You will want to consult the Linux manual pages for these calls (e.g., man getcontext) for further details.

    #include <ucontext.h>

     * Allocate memory for a ucontext_t struct.
    ucontext_t* ucontext_ptr = new ucontext_t;

     * Initialize the context structure by copying the current thread's context.
     * Necessary since ucontext_t objects contain machine-dependent information
     * that will be initialized by copying here.

     * Every thread needs a stack to facilitate making function calls.
     * Your thread library should allocate STACK_SIZE bytes (which is
     * defined in thread.h) for each stack.
    char* stack = new char[STACK_SIZE];          // allocate the stack
    ucontext_ptr->uc_stack.ss_sp = stack;        // store a pointer to the stack
    ucontext_ptr->uc_stack.ss_size = STACK_SIZE; // store the stack size
    ucontext_ptr->uc_stack.ss_flags = 0;         // no flags are needed

     * Depending on your library design, the uc_link field is potentially useful
     * (but designs are also possible that do not use or depend on this field,
     * in which case just set it to NULL).
    ucontext_ptr->uc_link = NULL;

     * Direct the new thread to call start(arg1, arg2), as an example.
     * Does NOT actually start executing the new thread; you need
     * to use swapcontext or setcontext for that.
    makecontext(ucontext_ptr, (void(*)()) start, 2, arg1, arg2);

Managing Context Structs

As described above, the contextual information supporting each thread is stored in a ucontext_t struct. It is important to avoid copying ucontext_t structs due to their internal structure, which you have no control over. For example, a ucontext_t struct happens to contain a pointer to itself (via one of its fields). If you copy the struct itself, you will copy the value of this pointer, and the new copy will point to the old copy's data member. If the old copy is later deallocated, the new copy will then point to garbage.

Be aware that it is rather easy to accidentally copy ucontext_t structs, such as by doing any of the following:

To avoid these problems, allocate new ucontext_t structs by calling new (as demonstrated in the example code above) and then use the resulting pointers exclusively. That way, the actual ucontext_t structs need never be copied.

Ensuring Atomicity

To ensure atomicity of multiple operations, your thread library will enable and disable interrupts. Since this is a user-level thread library, it can't manipulate the hardware interrupt mask to actually stop hardware interrupts. Instead, you will interact with the interrupt library libinterrupt.a that simulates software interrupts. While concurrent applications interact with the interrupt library solely via the start_preemptions call, the thread library itself will use several other calls, which are defined in the interrupt.h header file. This file will be included by your thread library, but is not included by application programs that use the thread library.

The relevant sections of interrupt.h are shown below:

 * interrupt_disable() and interrupt_enable() simulate the hardware's interrupt
 * mask.  These functions provide a way to make sections of the thread library
 * code atomic.
 * assert_interrupts_disabled() and assert_interrupts_enabled() can be used
 * as error checks inside the thread library.  They will assert (i.e. abort
 * the program and core dump) if the condition they test for is not met.
 * These functions/macros should only be called in the thread library code.
 * They should NOT be used by the application program that uses the thread
 * library; application code should use locks to make sections of the code
 * atomic.
extern void interrupt_disable(void);
extern void interrupt_enable(void);

#define assert_interrupts_disabled()          \
    assert_interrupts_private((char*) __FILE__, __LINE__, true)
#define assert_interrupts_enabled()         \
    assert_interrupts_private((char*) __FILE__, __LINE__, false)

Note that the interrupt_disable and interrupt_enable functions will abort the program if you try to call them when interrupts are already disabled or enabled, respectively. Also note that these functions do not allow you to test whether interrupts are currently enabled. While there is nothing stopping you from tracking the interrupt state yourself (e.g., via a boolean variable), you should not need to do so. Tracking the interrupt state explicitly is probably a sign that your interrupt handling logic isn't precise. At any specific point in your library, you should already know whether interrupts are enabled without having to track the interrupt state via a variable (and you can use the assert_interrupts_disabled and assert_interrupts_enabled calls to make sure your assumptions are correct).

Importantly, remember that interrupts should never be left disabled when returning to user code outside of the thread library itself.

Error Handling

There are three sources of errors that your library (or any OS code, for that matter) should handle. The first and most common source of errors comes from misbehaving user programs (e.g., misusing condition variables, releasing an unowned lock, etc). A second source of errors comes from resources that the OS uses, such as memory or hardware devices. Your thread library must detect if one of the lower-level functions it calls returns an error. For example, the C++ new operator may fail if the system is out of memory. By default, this operator will throw an exception if the system is out of memory, but you can also tell C++ to skip the exception and return null instead via the std::nothrow constant, which may be a simpler behavior to work with:

int* p = new (std::nothrow) int;   // allocate int or set to null on failure

For these first two sources of errors (user errors and OS resource errors), the thread function should detect the error and return -1. User programs can then detect the error and retry or exit.

A third source of errors is when the OS code itself (in this case, your thread library) has a bug. During development, the best behavior in such cases is to detect the bug quickly and abort (in the context of a real OS kernel, this is called a "kernel panic", and generally means you have to restart the system). A useful way to check for bugs in your thread library is by using assertions to verify that the library state is as you expect. In addition to the built-in interrupt assertions mentioned in the previous section, you can write your own assertion statements by including assert.h and then using the assert statement, such as the example below:

assert(lock_owner == current_thread);  // I should own the lock; panic if I don't

Use assertion statements liberally in your thread library to check for bugs in your code. However, don't confuse assertions with regular error checks; assertions are to be used for verifying conditions that should always be true (assuming that your thread library is correct) regardless of misbehaving users. Erroneous user program behavior should be handled by your regular error checks and return -1 as appropriate but shouldn't ever result in failed assertions.


As usual, starter code will be distributed via GitHub once you form your group and accept the starter repository.

Write your thread library inside and write each test case in a separate file named whatever you wish (as long as the filenames end in .cc). None of the provided header files should be modified. As with Project 2, the only supported development environment is on the class server. Your program may use any functions in the standard C++ library, including (and especially) the STL. You should not use any libraries other than the standard C++ library.

As a suggested implementation plan, start by implementing thread_libinit, thread_create, and thread_yield. Don't worry at first about disabling and enabling interrupts. After you get the basic threading system working, implement the lock and condition variable functions. Finally, add calls to interrupt_disable and interrupt_enable to ensure that your library works with arbitrary yield points. A correct concurrent program must work for any instruction interleaving. In other words, calls to thread_yield could be inserted anywhere in your code that interrupts are enabled and should not cause incorrect behavior.

The public functions in thread.h are declared extern (meaning that code outside of the thread library can call them), but all helper functions and global variables in your thread library should be declared static (meaning that code outside of the thread library will not see them) to prevent naming conflicts with programs that link with your thread library.

Test cases should be designed methodically: think of a particular behavior you would like to test, then write a test case that would distinguish a thread library demonstrating the intended (correct) behavior from a thread library demonstrating incorrect behavior. For example, you might write a test verifying that a thread is prohibited from acquiring a lock when another thread already owns it. Small tests that exercise specific behaviors will typically be more useful during initial development than large tests that do many things. Once you are able to consider larger tests, your disk scheduler is a good option (but you will need to modify it a bit to follow the testing rules, as discussed previously).

Compiling and Submitting

Compiling your project requires your thread library code in as well as a test program containing a main function that uses the thread library. The sample concurrent program from Project 2 is included as an example of this in To compile the thread library and a test program from a file named, use the included Makefile like so:

make mytest

This command will compile and and output an executable named mytest. You can substitute whatever test program name you wish (e.g., make sample, etc). The initial file contains a dummy implementation of each thread function, so you can compile the sample application immediately (but it will not execute properly when run, since the thread library functions are not implemented).

You can submit your program to the autograder as follows (the thread library must be named, but the test programs can be named whatever you wish ending in .cc):

submit3310 3 ...

To more easily submit all your source files at once ( and all of your test programs), you can use a shell wildcard, like so:

submit3310 3 *.cc


In addition to writing the program itself, you will also write a short paper (3-4 pages or so) that describes your thread library. Your paper should include the following:

  1. an introductory section that highlights the purpose of the project
  2. a design section that describes your major design choices and key data structures that you used, focusing particularly on how safe synchronization is achieved (if a figure makes your explanation more clear, use one!)
  3. an implementation section that overviews the structure of your code (at a reasonably high level - this should supplement rather than duplicate your code)
  4. an evaluation section that describes how you tested your library
  5. a conclusion that summarizes your project and reflects on the assignment in general

The same advice given for the Project 2 writeup applies here; make sure your writeup focuses on the major challenges and design decisions without becoming 'code-like'. Your writeup should also clearly state anything that does not work correctly.

To submit your group's writeup, add your writeup to your Git repository as a PDF named writeup.pdf (by the writeup deadline). Remember to commit and push and after adding your file. Typesetting your writeup in LaTeX is encouraged but not required.


Your project will be graded on program correctness, design, and style, as well as the quality of your project writeup. Remember that the autograder will only check the correctness of your program (and nothing else)!

You can (and should) consult the Coding Design & Style Guide for tips on design and style issues. Please ask if you have any questions about what constitutes good program design and/or style that are not covered by the guide.