CSCI 3325
Distributed Systems
Fall 2026
Instructor: Sean Barker
| Release Date: | Wednesday, September 9. |
| Due Date: | Tuesday, September 29, 11:59 pm. |
| Collaboration Policy: | Level 1 |
| Group Policy: | Groups of 2 or 3 |
In this project, you will implement and test a basic web server in C using low-level networking primitives. Building your server will teach you the basics of network programming, client/server architectures, and concurrency in networked applications. Testing your server will help you consider your design decisions, tradeoffs, and experimental design.
This project should be done in teams of two or three. However, remember that the objective of working in a team is to work as a team. In other words, all team members are responsible for all parts of the project, and the general expectation is that all team members will work together on all parts.
The project has two main components: (1) implementing your server, and (2) experimentally testing your server. Note that "experimentally testing" means more than simply verifying its correctness; rather, you will be designing and running a few experiments to explore the behavior of the system and the significance of your design choices.
Your task is to write a simple web server capable of servicing remote clients by sending them requested files from the local machine. Communication between a client and the server is defined by the Hypertext Transfer Protocol (HTTP). As such, your server will need to understand HTTP requests sent by clients and send HTTP-formatted responses back to clients.
You are not expected to implement the entire HTTP protocol. Instead, your server must support the following subset of functionality of both the HTTP 1.0 and the HTTP 1.1 standards:
GET method, which is the most common method used when fetching a web page. You do not need to support any other HTTP methods, such as HEAD or POST.Host and Connection. Note that for the Host header, you needn't worry about the actual contents of the header, but its presence is required by HTTP 1.1.Date, Content-Length, and Content-Type.200, 400, 403, and 404 in your server responses.HTML, TXT, JPG, PNG, and GIF.All other HTTP functionality is optional, including (for example) chunked requests, pipelining, absolute URLs, 100 Continue responses, or other headers such as If-Modified-Since and If-Unmodified-Since. If desired, however, feel free to extend your server to provide any functionality not required by the base specification.
Here is a concise tutorial on the essentials of the HTTP protocol that should provide enough detail to implement all required HTTP functionality.
Your server program must be written in C on Linux and must accept the following two command-line arguments, in arbitrary order:
-p [num] to set the port on which the server listens. If no port is specified, you may hardcode a default port, but your program must be able to specify a particular port using this flag.-r [path] to set the directory out of which all files are served, called the document root. If no document root is specified, your program should exit with an error message.For example, you should be able to start the server using a command like the following:
./server -p 8888 -r somedir/mywebpages
Note the important distinction between directory names that start with a /, known as absolute paths, and all other directory names, known as relative paths. Relative paths are interpreted relative to the current working directory, whereas absolute paths are the same regardless of the current working directory.
For example, suppose your current directory is /home/jdoe/proj1 and you launched your server as shown above, which specifies somedir/mywebpages as the document root. Since this path does not start with /, it is a relative path, and therefore files will be served out of /home/jdoe/proj1/somedir/mywebpages.
However, if you instead specified a document root that is an absolute path like /somedir/mywebpages, then files would be served out of /somedir/mywebpages, regardless of the current working directory.
IMPORTANT: Do not allow files to be accessed outside of the document root! The simplest way someone could attempt to do so is by sending a request like GET ../myprivatefile.txt, which navigates out of the document root (via ..) and then tries to access a file that the server shouldn't permit. A server that does permit such access would allow a client to potentially access any file on the machine that's readable by your user account (example: your SSH key granting access to your server account!). A simple way to prevent such access is by disallowing any file paths that include ... While a more elegant solution would permit paths including .. as long as the path remains inside the document root, it is perfectly sufficient for you to simply treat any file path containing .. as unauthorized (i.e., return HTTP code 403).
Client requests for a directory without a filename should default to fetching index.html inside the specified directory. For example, if the document root is /somedir/files and the request GET /catpictures/ is received, then the desired file is /somedir/files/catpictures/index.html. A request like GET /catpictures/mittens.jpg, which specifies a particular filename, would fetch /somedir/files/catpictures/mittens.jpg.
Your Git repository includes the following provided starter files:
server.c: the source file in which you should write your server.Makefile: a Makefile that will let you compile the server by typing make.testroot: a directory containing a simple web site that you can use as a document root during testing.The only file you must modify is server.c. You are welcome to modify the test document root or create other document roots to use during testing. Note that the provided Makefile is configured to compile with all errors turned on and warnings counted as errors; this is done intentionally to ensure that you fix compiler warnings rather than ignore them. Fixing warnings will often teach you something about programming even if the warning in question doesn't represent a bug in the program!
There are several ways you can test your server. The first is to simply access your server in a browser. For example, if your server is running on port 8888, then you could type http://hopper.bowdoin.edu:8888/something.html into your web browser to access something.html inside the document root on the server. However, testing with a browser is not recommended during early development and testing, as browsers will often simply hang or display nothing if your server isn't responding correctly. A more effective initial testing approach is to use telnet, which is a tool for sending arbitrarily-formatted text messages to any network server. For example, below is an example of connecting to google.com on port 80 and then sending an HTTP request for the file index.html:
$ telnet google.com 80 GET /index.html HTTP/1.0
Note that in the above command, there must be two carriage returns (i.e., blank lines) after the "GET" line in order to complete the command. The response to this request will be the HTTP-formatted response from the server. Using telnet will be more initially reliable than a browser, as you will be able to verify that you are getting any response back at all without also having to worry about whether the response is compliant with HTTP.
As an intermediate step between telnet and a full-blown browser, you can also use the wget or curl utilities. These utilities provide command-line HTTP clients: wget is a bit simpler to use, while curl provides some additional features but similar core functionality. Consult the man pages for details on proper usage.
A recommended testing strategy is to use telnet initially, then move to wget and/or curl, then finally move to a full-blown browser once things seem to be working. The provided sample document root will be useful in testing that HTTP 1.1 is working properly, as the pages with embedded images will reuse persistent HTTP 1.1 connections when possible.
IMPORTANT: Do not leave your server running when you are not actively testing! Whenever you are done testing, make sure to terminate your server (Control-C), especially before logging off the server. Leaving a server running for long periods will occupy port numbers and is a potential security risk.
This section contains tips on implementing various parts of the server.
You should use the getopt library function for parsing arguments. The basic idea is that getopt is given a string that specifies all of the possible command-line arguments, some of which may take associated values (which would be both -p and -r here). The string passed to getopt specifies arguments taking a value including a colon : after the associated character (so here, you would want to use p:r:). An idiomatic usage of getopt is to wrap calls to getopt in a while loop, and inside the loop, switch on the return value to process that argument. Within the switch, the predefined global variable optarg will contain the string value passed to that particular argument, which you can use to save each argument value.
You may find it helpful to consult, e.g., this simple example of parsing arguments using getopt, or your caching lab from CSCI 2330, which also used getopt for argument parsing.
At a high level, your core server functionality should be structured something like the following:
Forever loop:
Accept new connection from incoming client
Parse HTTP request
Ensure well-formed request (return error otherwise)
Determine if target file exists and is accessible (return error otherwise)
Transmit contents of file to client (by performing reads on the file and writes on the socket)
Close the connection (if HTTP/1.0)
You have a choice in how you handle multiple clients within the above loop structure. In particular, recall that we discussed three basic approaches to supporting multiple concurrent client connections:
pthreads thread library (e.g., pthread_create),
as demonstrated in class. Creating a new thread for each new connection is a bit less scalable than using a thread pool but is also simpler to design and perfectly sufficient for this project.
pipe system call; you can see an example usage of pipe together with fork in the manpages (run man 2 pipe).
While any of the above options are possible, a multi-threaded approach will generally be the most straightforward option, as coordination among threads is relatively simple via shared global variables.
Remember that HTTP requests will specify relative filenames (such as index.html)
which are translated by the server into absolute local filenames. For example, if your
document root is in ~username/cs3325/proj1/mydocroot, then when a request
is received for foo.txt, the file that you should read is actually
~username/cs3325/proj1/mydocroot/foo.txt.
The translated filename may exist and be readable, or it may exist but be
unreadable (e.g., due to file permissions), or it may not exist at all. A missing
file should result in HTTP error code 404, while an inaccessible file should
result in HTTP error code 403. You can test trying to access an inaccessible file
by changing file permissions using chmod. For example, chmod a-r foo.txt
will leave foo.txt intact but render it unreadable by your server, while chmod a+r foo.txt
will make it readable again.
Remember that the default filename (i.e., if just a directory is specified)
is index.html. This convention is why, for instance, the
two URLs http://www.bowdoin.edu
and http://www.bowdoin.edu/index.html
return the same page.
When you fetch an HTML web page in a browser (i.e., a file of type text/html),
the browser parses the file for embedded links (such as images) and then retrieves those
files from the server as well. For example, if a web page contains four images, then a total of
five files will be requested from the server. The primary difference between HTTP 1.0 and HTTP 1.1
is how these multiple files are requested.
Using HTTP 1.0, a separate connection is used for each requested file.
While simple, this approach is not the most efficient. HTTP 1.1 attempts
to address this inefficiency by keeping connections to clients open,
allowing
for "persistent" connections. That is, after the results of a single request are returned
(e.g., index.html), if using HTTP 1.1, your server should leave the connection open for
some period of time, allowing the client to reuse that connection to make subsequent requests.
One design decision here is determining how long to keep the connection open.
This timeout needs to be configured in the server and should be dynamic based
on the number of active connections the server is currently supporting. If the server
is idle, it can afford to leave the connection open for a relatively long period of time, but
if it is busy servicing several clients at once, it may not wish to
have an idle connection sitting around and consuming thread resources for very long.
You should decide how to dynamically determine this timeout in your server.
There are a few ways to implement socket timeouts. The simplest option is using the setsockopt system call, like so:
struct timeval tv; // timeout value struct tv.tv_sec = 5; // seconds tv.tv_usec = 0; // microseconds // set timeout for receiving data (SO_RCVTIMEO) setsockopt(socket_fd, SOL_SOCKET, SO_RCVTIMEO, &tv, sizeof(tv));
Nearly all system calls, including socket, accept, send, recv, and pretty much every other library function you're using in this project, share the same error conventions: a negative return value from a system call means an error occurred, and when this happens, the predefined global variable errno is set to some value indicating what happened. Socket timeouts piggyback on this standard convention, like shown below:
int bytes_received = recv(...); // read from a socket
if (bytes_received < 0) { // anomalous system call return
if (errno == EAGAIN || errno == EWOULDBLOCK) {
// either of these error values indicate just a timeout
} else {
// an actual socket error occurred;
// print a description of the error
printf("recv error: %s\n", strerror(errno));
}
}
As usual, all details on system calls (arguments, return values, etc.) are accessible via the man pages, such as man recv, which includes a list of various error values that could be generated by recv.
Your server may run into situations in which a client closes one end of the connection and then your server tries to send more data. When this happens, your server process receives a SIGPIPE signal, and the default action upon receipt of this signal is to terminate the process. Since you probably don't want this, an easy solution if you're running into this problem is telling your server to ignore SIGPIPE, as follows:
// ignore SIGPIPE to avoid crashes when using a closed client connection signal(SIGPIPE, SIG_IGN);
When you send or receive data over a network socket, what you are really doing is reading or copying data to a lower-level
network data buffer in the OS. Since these data buffers are limited in size, you may not be able to read or send all desired data at once.
In other words, when receiving data, you have no guarantee of receiving the entire request at
once, and when sending data, you have no guarantee of sending the entire response at once. As a
result, you may need to call send or recv multiple times in the course
of handling a single request.
As discussed in class, an alternative to using the low-level send and recv functions
is to use streams, which you have likely used before in the context of file I/O. Using streams
allows you to employ higher-level reading and writing functions like fgets (to read an entire line of data)
and fprintf (to write formatted data). To construct a stream from a socket descriptor, just use
the fdopen function. You can then use the resulting stream with all of the higher-level I/O functions like
fgets and fprintf to receive and send data, rather than the lower-level send
and recv calls. Doing so is likely to simplify your string processing code.
Since your server will need to do some string manipulation both when sending and receiving messages, you may want to use some of C's string processing routines, such as strcat,
strncpy, strstr, etc.
Any program involving concurrency (e.g., multiple processes or threads) needs to worry about the issue of synchronization, which refers to ensuring a consistent view of shared data across multiple threads of execution. Remember the general principle that shared data (such as any global variable) should not be modified concurrently by more than one thread to avoid potential data corruption. For example, it is unsafe to have two threads simultaneously incrementing a shared counter. One specific example in this project where you might want such a counter is if you want to track the active number of client connections.
To safely handle a situation like this, you should use a lock (also known as a mutex), which allows ensuring that only a single thread has access to a piece of code. The pthread library includes the pthread_mutex_t type for this situation. For example, if lock is a pthread_mutex_t, then you could safely increment a counter shared across multiple threads as shown below:
pthread_mutex_lock(&lock); // current thread acquires the lock global_counter++; // safe modification; only one thread can be holding the lock at once pthread_mutex_unlock(&lock); // release the lock to another thread
In addition to implementing the functionality of your server, you will experimentally evaluate a few aspects of your server's design. Your goal is not to demonstrate that your server is necessarily fast or scalable by any absolute metric, but rather to understand how particular design choices affect the behavior of the system.
Before you actually embark on a particular experiment, you should be able to clearly describe the following:
When you are actually running your experiments, remember that empirical results may vary from run to run, so make sure to repeat each configuration at least three times to make sure your results are reasonably consistent (a good rule of thumb is to graph the average). Record your quantitative results and produce a graph clearly demonstrating the results from each experiment. You should be able to explain what the results are showing and what they suggest about the design decision in question (whether or not they agreed with your initial hypothesis). Importantly, always be skeptical of empirical results! Results that don't make sense are often a result of a faulty experimental design that can be remedied.
For some of these experiments, you may need to generate a test workload that is impractical to produce manually. For example, if you wanted to test the behavior of the system with 50 simultaneous clients, it would not be advisable to do so by opening 50 browser or terminal windows. In this type of situation, you generally need some kind of testing harness code that will automatically generate a workload. For example, such a program might create a large number of concurrent threads and have each thread repeatedly request files from the server without requiring any manual human input. To make things a bit easier for this project, I will provide you with some boilerplate testing code to illustrate how you might implement such a test harness. Feel free to adapt this code to your own purposes if helpful.
Investigate the impact of the server's buffer size when transferring a large file to a client (i.e., the size of a single 'block' of data that you transmit). The assumption here is that the size of the file is larger than the buffer size and hence will be transferred in multiple blocks. Have a client request a large file in multiple trials, varying the server's buffer size for each trial. Consider both very small buffers (e.g., 64 bytes) as well as large buffers (e.g., a megabyte). Measure the time needed to complete each request. Remember to graph your results and be sure you can explain what your results are demonstrating about the impact of the buffer size.
Investigate the impact of the server's concurrency design as the server workload varies. Consider two possible server designs:
Test your server with a varying number of simultaneous clients under both concurrency designs. It should not be difficult to switch your server code between its concurrent design and the simpler baseline for experimental purposes. Consider 1, 5, 10, 25, and 50 simultaneous clients (don't go too high or you risk overwhelming the machine generating the workload). As the primary metric, consider the system throughput, defined as total requests / total elapsed time. Seconadilry, you can also consider the average duration of each request (i.e., the system latency).
In addition to the code of your server, you must submit a README document (in plain text format) that contains your names and the following sections:
telnet, you should not have very high confidence that your server is fully functional! Most typically, the "hardest" test that you should aim to pass is a sequence of browser requests for pages containing embedded images (which will be requested over HTTP 1.1).
Your writeup should be committed to your repository as a plain text file named README and is due at the same time as your server code. Remember to include your graphs as well!
I will initialize and share your group's GitHub repo after your group has been formed. Once your repository is initialized, clone it to hopper and work there. As a general rule of thumb when working on a group project through GitHub, always pull at the start of a work session and always commit and push at the end of a work session to minimize the chance of a merge conflict. Make sure that your final work (including your writeup) is committed to the repository by the deadline.
To avoid accidentally interfering with the servers of other groups, each group will be assigned a specific (non-standard) port number to use while testing on hopper. Stick to using your assigned port only to avoid conflicting with other groups. However, make sure that you are still able to specify any arbitrary port number via the -p command-line argument. Port assignments will be coordinated over Slack.
In addition to evaluating your submitted project, I will evaluate your project in an oral review session (aka viva voce). In this session, conducted with only myself and your group, we will (briefly) review your code, your experiments, and your results. You should expect me to ask clarifying or followup questions to make sure that you understand both the technical details and the conceptual ideas covered by the project. Also remember that even though you are working as a group, you are equally and individually responsible for understanding all parts of the group's submission. As such, I may direct questions to individual group members, particularly if a group member is minimally engaged during the session. Project grades for individual group members may be adjusted up or down based on the oral review if there are clearly unequal contributions or understanding.
Since an oral review session is evaluative and all sessions will necessarily occur at different times, you may not discuss any aspect of your review session with other groups until all reviews are complete. Although each group's session will be unique, maintaining confidentiality during the review period will ensure that no groups have an unfair advantage in the review session.
Your project will be graded on (1) correctly implementing the server specification, (2) the design and style of your program, (3) your experimental design and results presentation, and (4) your oral review session. For guidance on what constitutes good coding design and style, see the Coding Design & Style Guide, which lists many common things to look for. Please ask if you have any other questions about design or style issues. Also don't forget to submit your individual group reports prior to the deadline.
Here is a list of resources that may be helpful in completing your server:
man pages (e.g., man socket) should be your first stop for details on library functions.getopt: read and write are essentially equivalent to recv and send when applied to sockets): pthreads (thread creation) tutorial: