Experiments on CS 412

Benchmark Suites

Mon, 01 Jan 0001 00:00:00 +0000

The course provides benchmark collections and experiment-running infrastructure so that the Final Project can focus on algorithmic interpretation, not on writing a benchmark harness.

The benchmark definitions live in the team repository under:

benchmarks/

and are controlled by problem-specific manifest.json files. The manifest records which algorithms should run, time limits, fixed random seeds, structural metadata, and a known optimum when the exact value is available.

Required suites

Each fully supported problem provides:

Experimental Evaluation

Mon, 01 Jan 0001 00:00:00 +0000

This page provides guidance for the experimental work required by Checkpoint 2 and the Final Project. The checkpoint pages define which experiments and analyses are required; this page explains how to measure algorithm behavior, compare results fairly, characterize instances, organize experimental artifacts, and make the results reproducible.

The course experiment runner combines your algorithm results with benchmark metadata from the problem manifest. In addition to your returned solution and statistics, the generated result files record information such as known optima when available, published or class best-known values, bounds, instance size and structure, algorithm, seed, and timing information. This lets you focus on interpreting the experiment rather than manually reconstructing benchmark metadata.

Challenges & Leaderboards

Mon, 01 Jan 0001 00:00:00 +0000

Once your heuristic reliably returns valid solutions, the next question is not merely whether it works. It is:

How much better can you make the solution, and what can you actually say about its quality?

The project uses optional challenge benchmarks and leaderboards to give you a reason to keep improving a heuristic after you already have a feasible answer. The leaderboard is intended as a design challenge and source of experimental evidence, not as the primary basis for your grade.