HW4: Accessibility Assessment

Extra Credit Opportunity

This assignment is paired with an honors thesis research project. You have the opportunity to participate as a research subject by filling out a quick survey before and after the completion of the assignment.

There is an opportunity to gain extra credit by participating in the research. There will be an alternative way to gain extra credit if you opt not to participate in the survey. You will also have the opportunity to enter a raffle for a $10 gift card.

Please click the link below to access the consent form and read it thoroughly. Once you've indicated your decision in the form, whether or not to participate in the research aspect of this project, you will be directed to the project specs.

https://jmu.co1.qualtrics.com/jfe/form/SV_cwJ3rmjR9M8TFqe

As part of their review, the James Madison University Institutional Review Board has determined that this study (protocol # IRB-FY26-109) is no more than minimal risk and exempt from on-going IRB oversight.

This short 4 minute video provides a more in depth explanation of the accessibility concepts being covered in this lab. Click here to open the video on YouTube.

Watch the video before starting the assignment!

Introduction

Today, almost everyone uses the internet—for learning, shopping, chatting with friends, and much more. When we build websites, we have to remember that all kinds of people will use them, including people with disabilities. Some of these users might use special tools like:

• Screen readers, which read the text on a page out loud.
• Speech recognition software, which lets people control the computer using their voice.

To make websites usable for everyone, developers need to follow accessibility guidelines. These are rules that help websites work well for all users. Making a website accessible isn't hard, but it's something developers sometimes forget. When accessibility is ignored, some people may not be able to use the site at all.

Luckily, there are tools that can help. These tools scan a website and find common accessibility problems, like:

• Images missing alt text (descriptions for screen readers),
• Poor color contrast (making text hard to read),
• Headings that are not structured correctly.

These tools give quick feedback and suggest ways to fix the issues.

But here's the catch: no single tool finds everything. Each tool checks in a different way. One tool might catch a mistake that another tool misses. That's why it's a good idea to use multiple accessibility checkers. When you combine results from several tools, you get a better overall picture of how accessible a website really is.

A few years ago, the UK Government Digital Service created a test to compare different accessibility checkers. They built a webpage full of accessibility mistakes and tested which tools could find them. We saved part of their results in a file called a11yCheckersResults.txt.

In this project, your task is to work with that file and help analyze the results. This will help us understand how well the tools did at finding accessibility problems.

a11yCheckersResults.txt Format

The file is organized such that each line contains the results for one assessment organized as follows:

[google result] [wave result] [sortsite result] [aslint result] [assessment description]

The first 4 parts of a line (the result of the assessment for each checker) are all a single "word". The last part of the line is the assessment description, and it is of variable length.

The result of the assessment for each checker is a single word that indicates the result of the assessment. The 6 possible results are:

1. error – the error was properly found
2. error_paid – the error was properly found in the paid version
3. warning – the checker only gave a warning
4. manual – the checker required the user to manually identify the error
5. identified – The checker noticed the error, but didn't give a warning
6. notfound – The checker failed to identify the error

Part 1 - AccessibilityAssessment

You are to design and implement a class named AccessibilityAssessment, which stores the description of the assessment and the results of the assessment for the four checkers. The constructor for the class should take all 5 pieces of information as parameters and store those values in fields. It should have accessor methods for each value, e.g., getDescription, getGoogleResult, getWaveResult, etc. It should also have a toString method which presents a readable format of the results of the assessment.

classDiagram
    class AccessibilityAssessment {
        -googleResult : String
        -waveResult : String
        -sortsiteResult : String
        -aslintResult : String
        -description : String
        +AccessibilityAssessment(googleResult : String, waveResult : String, sortsiteResult : String, aslintResult : String, description : String)
        +getDescription() String
        +getGoogleResult() String
        +getWaveResult() String
        +getSortsiteResult() String
        +getAslintResult() String
        +toString() String
        +equals(other : Object) boolean
        +foundError(checkerPartialName : String) boolean
    }

Update 10/13: Small fix to constructor (it was missing the sortsiteResult parameter).

Constructor

The constructor is required to validate its parameters. It must ensure that all parameters are not null and that the different checker results are all valid options for checkers.

The valid options for the checker results are:

• error
• error_paid
• warning
• manual
• identified
• notfound

If all parameters are valid it should store the parameters in the corresponding fields. If any parameters are invalid then the constructor should throw an IllegalArgumentException with the error message: "Invalid Constructor Parameters".

toString(): String

This method should return a (1-line) String with the following information:

1. The results of the assessment for each checker (in the order presented in the UML diagram and the constructor parameters) formatted as follows:
   1. The name of the checker (i.e. Google, WAVE, SortSite, or ASLint) followed by : (a colon and a space)
   2. The result of the assessment for the checker (from the previously enumerated list of valid result String values), followed by (a space).
2. The description of the assessment, preceded by - (a dash and a space).

For example: Google: notfound WAVE: notfound SortSite: notfound ASLint: warning - Colour/Contrast: Colour alone is used to convey content

equals(other: Object): boolean

This method should return true if the other object is an AccessibilityAssessment object with the same description and checker results (for all 4 checkers) as this object. It should return false otherwise.

Format for Equals Methods

For (almost) all equals methods you will write, you'll need to do the following things:

1. Null Check: Check if the other object is null. If it is, return false.
2. Type Check: Check if the other object is the correct type using instanceof. If it isn't, return false.
   • Why is other of type Object? So that you can compare it with any object. Unfortunately, this means that the parameter is not guaranteed to be the correct type. So we have to check!
3. Type Cast: Cast the other object to the correct type.
   • Why did we check the type first? Because if we try to cast an object to a type that it isn't, we'll get a runtime error (an exception).
   • Why do we need to cast it? Because the type Object doesn't "know" about the fields of your class. You have to cast it to your class type so that you can access those fields.
4. Comparison: Compare all of the fields of this object to the corresponding fields of the other object. If they all match, return true. If any field doesn't match, return false.
   • For primitive types (like int, double, boolean, etc.), use == to compare.
   • For objects (like String, or any class), use the .equals() method to compare, since you want to compare the objects, not their memory addresses.

For example, here is a sample equals method for a class named Person with fields name (a String) and age (an int):

@Override
public boolean equals(Object other) {
    if (other == null) {
        return false;
    }

    // instanceof will check if the type is correct
    if (!(other instanceof Person)) {
        return false;
    }

    // You must cast the object to the correct type to access its fields
    Person otherPerson = (Person) other;

    return this.name.equals(otherPerson.name)
        && this.age == otherPerson.age;
}

foundError(checkerPartialName: String): boolean

This method should return true when the checker whose (case-insensitive) name contains checkerPartialName had a result of error or error_paid. It should return false otherwise.

The formal names of the checkers are (in order of precedence in the file):

1. google
2. wave
3. sortsite
4. aslint

For example let's say this AccessibilityAssessment object is representing a line in a11yCheckersResults.txt file, which reads:

notfound error error_paid notfound Content: This is a description

In this case, we should have created an AccessibilityAssessment object with the following values:

• googleResult: notfound
• waveResult: error
• sortsiteResult: error_paid
• aslintResult: notfound
• description: Content: This is a description

Thus, foundError() should behave as follows:

• If we call foundError("goog") on this object, it should return false because the Google checker did not find an error.
• If we call foundError("wave") on this object, it should return true because the WAVE checker did find an error.
• If we call foundError("sortsite") on this object, it should return true because the SortSite checker found an error (in the paid version).
• If we call foundError("lint") on this object, it should return false because the ASLint checker did not find an error.
• If the partial string matches multiple checkers, the first matched checker (based on precedence) should be returned.
  • For example, if we call foundError("o") on this object, it would refer to the Google checker and should return false.

If the string passed cannot be found in the name of any checker, then the method should throw an IllegalArgumentException with the error message: "Invalid String Parameter".

Part 2 - AccessibilityResults

The file a11yCheckersResults.txt as been provided for you, which contains the information about assessments used by the UK Government Digital Services to evaluate accessibility checkers. Each line in the file contains the information about a single assessment organized as described previously.

You are to design and implement a class named AccessibilityResults that reads in the assessment information from a file of this format, stores them in an ArrayList of AccessibilityAssessment objects, and provides methods for accessing that information.

When you are reading in the information, make sure that you are storing the values in the most appropriate types. Your implementation of AccessibilityResults should conform to the following UML diagram as well as the detailed specifications that follow.

classDiagram
    class AccessibilityResults {
        -assessments : ArrayList<AccessibilityAssessment>
        +AccessibilityResults(filename : String)
        +numAssessments() int
        +writeAssessmentsToFile(filename : String, format : String, results : ArrayList<AccessibilityAssessment>)
        +showAssessmentResults(details : String) ArrayList<AccessibilityAssessment>
        +showAllMissedErrors() ArrayList<AccessibilityAssessment>
        +showErrorsFound(details : String) ArrayList<AccessibilityAssessment>
        +numErrorsFound(details : String) int
        +getAll() ArrayList<AccessibilityAssessment>
    }

Note: for all methods, you must always check to ensure your parameters are not null. If a parameter is null the method should throw an IllegalArgumentException with the error message: Null Parameter.

You should also avoid repeating code. If you find yourself writing the same code that you've already written in another method, maybe call that method instead! If needed, you should create a private helper method to avoid redundancy.

This class should conform to the following specifications:

Constructor

This method should take the filename as a parameter and should read in the file of accessibility assessments. The constructor should parse each line in the file to create AccessibilityAssessment instances and store them in the assessments ArrayList.

The constructor should use try-catch exception handling to print an error message if an invalid filename is found.

The error message should be formatted as follows: File not found: <filename> (where <filename> is the name of the file that was passed in as a parameter).

numAssessments(): int

This method takes no parameters and returns the number of assessments that are stored in the ArrayList.

getAll(): ArrayList<AccessibilityAssessment>

This method must return a copy of the assessments list. Do not just return the assessments reference, as that would allow the caller to modify the list.

writeAssessmentsToFile(filename : String, format : String, results : ArrayList<AccessibilityAssessment>)

This method is a helper method that writes the contents of an ArrayList of AccessibilityAssessment objects to a file. You should use this method in other methods that need to write results to a file!

This method expects 3 parameters:

1. filename: The path to the file to which the results should be written.
2. format: The format string for the final line of the output.
3. results: the ArrayList of AccessibilityAssessment objects to be written.

This method should write each of the AccessibilityAssessment objects in the results ArrayList to its own line of the file specified by filename. These lines should be formatted according to the AccessibilityAssessment's toString.

The final 2 lines of the file should be:

1. an empty line
2. a line formatted according to the format parameter.
   • For example, the format string might be Total tests matching: %d or Total tests failed: %d.
   • This line should be passed the results.size() as a parameter to the format string.
   • This supports messages like, Total tests matching: 10 or Total tests failed: 51.

Use try-catch exception handling to print an error message if an invalid filename is found.

The error message should be formatted as follows: File not found: filename (where filename is the name of the file that was passed in as a parameter).

Writing to a File?

The easiest way to write to a file is to use the PrintWriter class. It lets you write to a file just like you would write to the console using System.out.

Here's a short example:

import java.io.FileNotFoundException;
import java.io.File;
import java.io.PrintWriter;

public class Example {
    public static void main(String[] args) {
        String filename = "output.txt";

        try {
            PrintWriter writer = new PrintWriter(filename);

            writer.println("Hello, World!");
            writer.printf("How many cookies: %d\n", 10);
        } catch (FileNotFoundException e) {
            // Handle the exception here
        }
    }
}

showAssessmentResults(details : String): ArrayList<AccessibilityAssessment>

This method takes assessment details (or a portion of the details) as a parameter, and both returns an ArrayList of AccessibilityAssessments that match (or contain) that detail in the description (case insensitive) AND writes the results to a file.

The file should have each matching result on its own line, then a blank line at the end, followed by a summary. The summary should read as follows: Total tests matching: <# of tests>.

The name of the file should be showAssessmentResults-<details>.txt where <details> is the value of the details parameter.

For example, if the information from a11yCheckersResults.txt was read in and stored in a AccessibilityResults object named a11y, then a11y.showAssessmentResults("Colour") should write the following to the file named showAssessmentResults-Colour.txt:

Google: notfound WAVE: notfound SortSite: notfound ASLint: warning - Colour/Contrast: Colour alone is used to convey content Google: error WAVE: error SortSite: error_paid ASLint: notfound - Colour/Contrast: Small text does not have a contrast ratio of at least 4.5:1 so does not meet AA
Google: error WAVE: error SortSite: error_paid ASLint: notfound - Colour/Contrast: Large text does not have a contrast ratio of at least 3:1 so does not meet AA
Google: error WAVE: error SortSite: error_paid ASLint: notfound - Colour/Contrast: Small text does not have a contrast ratio of at least 7:1 so does not meet AAA
Google: error WAVE: error SortSite: error_paid ASLint: notfound - Colour/Contrast: Large text does not have a contrast ratio of at least 4.5:1 so does not meet AAA
Google: notfound WAVE: notfound SortSite: notfound ASLint: notfound - Colour/Contrast: Focus not visible
Google: notfound WAVE: notfound SortSite: notfound ASLint: notfound - Links: Identifying links by colour alone
Google: notfound WAVE: notfound SortSite: notfound ASLint: warning - Forms: Errors identified by colour only
Google: warning WAVE: error SortSite: error ASLint: error - Forms: Errors identified with a poor colour contrast
Google: notfound WAVE: notfound SortSite: notfound ASLint: notfound -  HTML: Inline style adds colour

Total tests matching: 10

showAllMissedErrors(): ArrayList<AccessibilityAssessment>

This method takes no parameters, and returns an ArrayList containing the AccessibilityAssessments that all checkers failed (i.e. an assessment is only shown if Google, WAVE, ASLint, and SortSite all failed to find the error, and if even one of the checkers found the error, it is not shown).

The method will also write these AccessibilityAssessments to a file named showAllMissedErrors.txt followed by a summarizing statement in the format specified below.

Remember!

As previously described, failing to find the error is the result notfound.

For example, if the information from a11yCheckersResults.txt was read in and stored in a AccessibilityResults object named a11y, then a11y.showAllMissedErrors() should write the following to showAllMissedErrors.txt:

Google: notfound WAVE: notfound SortSite: notfound ASLint: notfound - Content: Content identified by location
...
Google: notfound WAVE: notfound SortSite: notfound ASLint: notfound - HTML: Inline style adds colour
Google: notfound WAVE: notfound SortSite: notfound ASLint: notfound - HTML: PRE element without CODE element inside it

Total tests failed: 51

showErrorsFound(details : String): ArrayList<AccessibilityAssessment>

This method takes assessment details (or a portion of the details) as a parameter. It returns an ArrayList of assessments that had at least one result of either error or error_paid.

Note: this method should work with a partial assessment description and should be case insensitive.

The method will also write these AccessibilityAssessments to a file named ShowErrorsFound-<details>.txt followed by a summarizing statement in the format specified below.

For example, if the information from a11yCheckersResults.txt was read in and stored in an AccessibilityResults object named a11y, then a11y.showErrorsFound("language") should write something like the following excerpt to ShowErrorsFound-language.txt:

...
Google: error WAVE: notfound SortSite: error ASLint: error - Language: html element has an empty lang attribute
...

Total tests matching: 5

numErrorsFound(details : String): int

This method takes assessment details (or a portion of the details) as a parameter, and returns the number assessments that had at least one result of either error or error_paid. However, this method should NOT write anything to a file.

Note: this method should work with a partial assessment description and should be case insensitive.

For example, if the information from a11yCheckersResults.txt was read in and stored in a AccessibilityResults object named a11y, then a11y.numErrorsFound("language") should return 5.

Avoid Redundancy

The behavior of this method is very similar to another method, with one key difference: this method should NOT write anything to a file.

You may want to call the other method to avoid repeating code, but that will result in writing to a file, which you don't want.

We recommend that you create a private helper method that does the common work, and then have both of these methods call that helper method.

Unit Testing

You must write JUnit tests for both the AccessibilityAssessment and AccessibilityResults classes. Your JUnit test suite should achieve at least 95-98% coverage (we will give you 10 grace lines/branches). Your test files should be named AccessibilityResultsTest.java and AccessibilityAssessmentTest.java.

Required Files

Download the following starter files:

• 📥 a11yCheckersResults.txt - Sample accessibility assessment data
• 📥 AccessibilityAssessmentTest.java - Starter test file with examples

Right-click the links above and select "Save link as…" to download the files to your project directory.

Recommended Process

1. Create a hw4 folder in cs159/src/hws/.
2. Stub out both classes and their methods.
3. Make an early submission to Gradescope to ensure that all classes and methods have the correct names and signatures.
4. Write tests for AccessibilityAssessment.
5. Implement AccessibilityAssessment.
6. Write tests for AccessibilityResults.
7. Implement AccessibilityResults.

AccessibilityAssessment

1. Define the class and instance variables.
2. Write the constructor.
3. Write the accessors ("getters").
4. Test the constructors and accessors.
   • For this, you may wish to start with the provided AccessibilityAssessmentStudentTest.java
5. Write the equals method.
6. Test the equals method.
7. Before writing toString, write a test for it. (you should be failing this test right now)
8. Write the toString method. If you wrote your test well, you should be passing the test when you're done with this method.

AccessibilityResults

1. Define the class and instance variable.
2. Write the constructor.
   1. Probably begin by having your constructor manually create a few AccessibilityAssessment objects and add them to the ArrayList.
   2. Once you have that working, you can start working on reading the AccessibilityAssessments from a file. Perhaps as you move into this step, you should create a smaller version of the data file, (maybe just make a copy of a11yCheckersResults.txt and delete all but the first 6 or so lines) until you're happy with how it's working, and then move on to the full file.
      • In reading the file, observe that while it may be helpful to know that (e.g.) your Scanner hasNextLine(), it might be helpful to extract less than a full line at a time, at least at first.
3. Write writeAssessmentsToFile.
   • note that this method is early in the instructions, especially that it's before other methods that should… write… results… to a file.
4. Write the rest of the methods.

Additional Resources

JDK Scanner class

The Scanner class in the Java Development Kit (JDK) is useful for reading in data. It can help parse data from the command line as well as from a file. Take a look at the Scanner javadoc and our previous File I/O lab for examples of how to use the Scanner class.

JDK String methods

You may find several of the Java Development Kit's String methods useful in this assignment. Here are some links to the documentation for a few of these methods:

• String.contains
• String.format
• String.toLowerCase

Submission

You must submit (using Gradescope):

1. AccessibilityAssessment.java
2. AccessibilityResults.java
3. AccessibilityAssessmentTest.java
4. AccessibilityResultsTest.java
5. a11yCheckersResults.txt (the data file provided to you)
6. Any other .txt test data files you created for your tests

Submission Limit: You will have 20 free submissions for this assignment. After that, each submission will cost you 2 points.

We recommend that you still make early and frequent submissions to ensure that you are making correct progress. Do not wait until the last minute to submit your assignment!

If your code does not compile, you will receive a score of 0 for the assignment. Thus, don't avoid submitting, but also don't waste all your submissions using Gradescope to test your code for you!

Grading

Your code will first be graded by Gradescope and then by the Professor. The grade you receive from Gradescope is the maximum grade that you can receive on the assignment. The Professor may deduct points for style issues, poor documentation, or other issues.

Styling

Your code must pass checkstyle style checks in order to receive any credit. However, you do NOT need to pass checkstyle for any test files.

Gradescope Grading

Your code must compile (in Gradescope, this will be indicated in the section on "Does your code compile?") and all class names and method signatures must comply with the specifications (in Gradescope, this will be indicated in the section on "Do your class names, method signatures, etc. comply with the specifications?") for you to receive any points on this assignment. Gradescope will then grade your submission as follows:

Criterion	Points	Details
Conformance to the Style Guide (Style)	0 points	(All or Nothing; Success Required)
Passing Your Tests (SelfTests)	10 points	(Partial Credit Possible)
Coverage Check (Coverage)	30 points	(Partial Credit Possible; points deducted for missing lines/branches)
Correctness (Official Tests)	60 points	(Partial Credit Possible)

Gradescope will provide you with hints, but may not completely identify the defects in your submission. Rely on your own tests to ensure that your code is working correctly.

Manual Grading

After the due date, the Professor may manually review your code. At this time, points may be deducted for inelegant code, inappropriate variable names, bad comments, etc.

You must ensure that you are avoiding redundancy in your code and that you are calling methods where appropriate. You will not receive full credit if you are repeating code unnecessarily. If needed, you can always create a private helper method to avoid redundancy.