General Project Guide

This document describes the general project framework for CS 261. It does not tell you how to use basic Linux commands or how to program in C. For help with those topics, see the other links on the resources page. Refer to the stu page on the wiki for instructions about how to log into stu if you've never done that before.

Getting Started

All of the starter files for CS 261 projects will be located in the following directory on stu: 

/cs/students/cs261/f23

We suggest creating a new folder in your home directory for CS 261 projects. To get started with a project, copy the corresponding tarball to the directory you want to work in, then extract it. For example, the first time you work on project 0, you might execute the following commands: 

$ mkdir cs261
$ cd cs261
$ cp /cs/students/cs261/f23/p0-intro.tar.gz .
$ tar -zxf p0-intro.tar.gz
$ cd p0-intro
$ ls
main.c  Makefile  p0-intro.c  p0-intro.h  tests

Here are brief descriptions of each file you will find in the starter tarball:

  • main.c - Application driver; you will implement integration test functionality here.
  • p0-intro.c - Project source file; you will implement unit test functionality here.
  • p0-intro.h - Project header file. This contains the interface for functionality that you must implement in the corresponding .c file. DO NOT MODIFY THIS OR ANY OTHER HEADER FILE!
  • Makefile - Project build instructions. This contains the project-specific compilation instructions. Do not modify this file unless you know what you're doing.
  • tests - This folder contains the testing files, described below. You may add your own tests if you wish, but be careful not to break the original tests.

For later projects, you will have multiple code modules, but they will follow the same pattern as the p0-intro.c and p0-intro.h files.

You may also be given .o files with pre-compiled solutions to previous projects. If you wish, you can replace these with your source code from previous projects and modify the Makefile accordingly; if your solutions were complete then this may help you with debugging.

To build the project, simply run "make" in the project folder.

Note: Occasionally, the timestamps on your files may get modified such that "make" does not recompile a component even though it needs to. If you suspect that this may be the case (e.g., you're seeing a segfault where it makes no sense at all), you should try running "make clean" to clear out ALL output files (the final executable as well as any .o files). You'll then need to re-run "make" to rebuild from scratch. This is certainly not guaranteed to fix the problem but it will ensure that you're working with an executable compiled from all of the latest source files.

Project Specifications

Every project has its own specification page in addition to this general guide. For example, here is a link to the project 0 page. Reading and understanding these specifications is crucially important in CS 261. You will be doing programming at a lower level (in terms of abstraction) than ever before in this course, and often students find that understanding exactly what the specification is asking for is a non-trivial first step to completing the projects in this course.

In fact, being able to read and understand specifications for complicated, systems-level programs is a learning outcome of this course, and a skill that will be required in CS 361 and your advanced systems elective(s). Thus, we recommend re-reading the project specifications multiple times as you work on the project. Some students also find it helpful to print out the project description pages to highlight important parts and make notes in the margin.

General Testing

When testing your code, we WILL try to break it with invalid input. Your code must detect this invalid input and react in an appropriate manner. For all projects, our test cases will include the following conditions:

  • invalid command-line flags
  • missing required command-line arguments
  • passing non-existent or empty files
  • repeating command-line flags
  • invalid arguments to flags
  • invalid combinations of flags
  • very large numbers and files
  • passing invalid parameters to functions
  • passing NULL values and empty strings as arguments

To run all automated tests, simply run "make test" from your project folder. The output should look something like this when you are finished with your project (this example is from P1): 

$ make test
gcc -c -g -O0 -Wall --std=c99 -pedantic main.c
gcc -c -g -O0 -Wall --std=c99 -pedantic p1-check.c
gcc -g -O0 -o y86 main.o p1-check.o
make -C tests test
make[1]: Entering directory '/cs/home/stu-f/uid/cs261/p1-check/tests'
gcc -c -g -O0 -Wall --std=c99 -pedantic -I/opt/local/include testsuite.c
gcc -c -g -O0 -Wall --std=c99 -pedantic -I/opt/local/include public.c
gcc -g -O0 -L/opt/local/lib -o testsuite testsuite.o public.o ../p1-check.o private.o \
    -lcheck -lm -lpthread -lrt -lsubunit
========================================
             UNIT TESTS
100%: Checks: 9, Failures: 0, Errors: 0
========================================
          INTEGRATION TESTS
C_simple_hex                   pass
B_help                         pass
B_simple_full                  pass
B_multisegment                 pass
B_stripped                     pass
A_invalid_param                pass
A_missing_filename             pass
A_nonexistent_file             pass
A_no_elf                       pass
A_short_header                 pass
No memory leak found.
No uninitialized value found.
========================================
make[1]: Leaving directory '/cs/home/stu-f/uid/cs261/p1-check/tests'

Below you will find descriptions of the unit and integration test suites, respectively. You are strongly encouraged to explore these tests and make sure you understand them, because you will be doing a LOT of debugging in this class. Note also that this "make test" automated command is only intended to help you get a sense of where you are and whether any recent changes have broken previously-implemented functionality. For debugging your program's behavior on individual test cases, you will want to reproduce the failure outside the automation of the "make test" command.

In addition, you are encouraged to write and run your own tests, especially because you will not be given the source code for some of the tests.

Unit Testing

Unit test cases are intended to test whether or not one particular function or piece of code is working correctly.

Here are brief descriptions of the unit testing files you will find in the tests folder after you extract the starter tarballs:

  • Makefile - Test suite build instructions.
  • testsuite.c - Unit test suite driver.
  • public.c - Source code for public tests.
  • private.o - Compiled object file for private tests.

Some tests are public and the source code for these tests are in public.c. Other tests are private and are compiled into private.o.

To run only the unit tests, run the command "make utest". (For this command to work, make sure you're inside the tests folder.) The results will show all of the commands run by make to build your program and test suite, followed by the testing results. The most important line of output is the test suite summary, which looks like this: 

0%: Checks: 2, Failures: 2, Errors: 0

This line tells you how many unit test checks were run and how many of them failed. If there were any failures, the specific test names will appear on the following lines, along with the specific assertion that failed.

Every test case is prefixed by a capital letter (e.g., "A_" or "B_" or "C_"). In order to earn a particular letter grade, you must pass ALL of the tests that are tagged with that grade (and any lower grade), both in the unit tests and the integration tests (see below).

If you wish to run a public test outside of the testing framework (useful for debugging), you can usually copy/paste the test code into your main() and temporarily disable your normal implementation using comments or #if 0 / #endif block. You will be unable to run private tests outside of the testing framework; for these you are encouraged to look at the test name as a hint regarding the problem and to create your own testing code to help you debug your solution.

If you are seeing a segfault in a unit test but cannot reproduce it outside the test suite, it may be helpful to run the test suite in a debugger. Unfortunately, the multi-threaded nature of the test suite makes it difficult to debug by default. However, it is possible to disable that aspect of the tests by using the CK_FORK option. Run this command from the tests folder: 

CK_FORK=no gdb ./testsuite

You should then be able to place a breakpoint on a particular test using its name, and if there is a segfault the debugger should pause at the right place. Note that if it is a private test you will not be able to see the source code.

Integration Testing

While unit tests focus on individual components of your code, integration testing will evaluate the complete functionality of your project. These tests are based purely on the output produced by your code in response to various inputs (files and combinations of parameters). It must match the contents of the expected/*.txt files verbatim. If there is even a single extra space anywhere, the test case fails.

Every integration test has a tag that is prefixed with a capital letter indicating the corresponding letter grade, just as with unit tests, and in order to earn a particular letter grade you must pass ALL of the tests that are tagged with that grade (and any lower grades).

For every integration test, there is a corresponding line in itests.include, which shows the command line parameters used for the test, as well as a corresponding text file in the tests/expected subfolder that shows the "correct" output. In the inputs folder, you will find several input files; however, each input file may be used by multiple individual tests (and some integration tests use no input files).

To run only the integration tests, run the command "make itest". (For this command to work, make sure you're inside the tests folder.) Unlike unit tests, which only print individual test names if they fail, integration tests print a line of output for every test case along with a status message reading either "pass" or "FAIL". Your program outputs will be placed in a text file in the tests/outputs folder, and a unified diff against the expected output will be placed in that same folder.

The test suite will also run each integration test using the Valgrind Memcheck tool to check for memory leaks and uninitialized values. If you are leaking memory or if you have an uninitialized value, you can view the reports stored in the tests/valgrind subfolder for some hints about where to look to fix the problem. Points will be deducted from your score for memory leaks and uninitialized values.

If you wish to run an integration test outside of the testing framework (useful for debugging), look inside itests.include in the testing folder to see the command line parameters corresponding to that test and run them yourself. If you are running the program from the project root instead of from the tests folder (which is most likely the case), you'll need to add "tests/" as a prefix to any filename parameter(s).

Submitting

Discussion and collaboration is both welcomed and encouraged in this class. As such, you are encouraged to post questions and discuss the project on the course Piazza page, which is available through Canvas. However, you may not post any non-trivial code, nor may you copy another another person's code. Your final submission must be your individual work. By turning in your submission, you are affirming your compliance with these guidelines and the JMU Honor Code.

To submit your code, run the following script from your project directory (the one with your .c files), giving it the assignment key ("pX" where X is the project number): 

/cs/students/cs261/f23/submit.sh p0

You may wish to create a link to this script in a directory that is in your PATH. DO NOT copy the script, as it may change during the semester and you need to make sure you are always using the most recent version.

The output should look like this: 

Submitting p0 for user 'YOUR-EID' ...
Destination: /cs/students/cs261/f23/submissions/YOUR-EID/
Files submitted:
  * main.c [Sep 1 9:07 AM]
  * p0-intro.c [Sep 1 9:07 AM]
Done.

This script copies your files to the /cs/students/cs261/f23/submissions/YOUR-EID/ folder. Only you and your instructor have access to that folder.

We will only assign grades based on files properly copied to your submit folder as described above. Please check the folder yourself to verify that the submit script worked. If you wish, you can manually copy the files yourself using cp command. However, if you accidentally copy the files to the wrong folder, you will not receive credit, so make sure you check your files carefully. Additionally, the submit script does NOT copy header files--your submission must compile with the original provided headers.

IMPORTANT: We will use the system file timestamps to verify on-time submissions. If you copy or modify the files after the due date, you will receive the late penalty, regardless of when you claim the work was actually done because we have no way to verify your claims.

Grading

Your submission must compile and run on stu using only the standard C libraries (excluding functions considered unsafe as enumerated in the C functions reference) and explicitly allowed third-party libraries (such as the Check library). Code that does not compile or that uses unauthorized or unsafe function calls will receive a grade of zero regardless of other merit. Code that compiles with warnings will receive a grade penalty. If you aren't sure about which functions are unsafe, please consult the C function reference.

Your base grade is determined by the test cases that you pass or fail. You must pass all tests tagged with a particular grade in order to qualify for that grade. For instance, if you are failing a test and its name begins with "A_", the highest score you can receive is a "B". If you have a memory leak, we will deduct half a letter grade from the base grade.

Further, we will be examining your submission manually. If you did not make a good-faith effort to complete the project, you will receive a zero. In particular, if you attempt to hard-code the solution (i.e., passing tests simply by detecting the test and manually outputting the "correct" answer), you will receive a zero. If you are found to have copied code or allowed someone else to copy your code, you will recieve a zero on the assignment and may be reported to the JMU honor council.

We will also be checking for a readable, clean, and consistent code style and good coding practices. If we find minor issues, we will deduct half a letter grade from your base grade. If we find major issues, we will deduct a full letter grade from your base grade. If you have questions about what constitutes acceptable coding practices in this class, please consult the class coding standards.

As per the late policy from the syllabus, we will also be deducting a letter grade for each 24-hour period that it is late. For example, a submission that would have earned an "A" in an on-time submission will earn a "B" if submitted up to 24 hours late, or a "C" if submitted up to 48 hours late.