Skip to content

HW 5: User Evaluations

The goal of this homework is to practice evaluating VR apps with users and gathering useful feedback. You will learn how to conduct a user study and analyze its results, which is essential for real VR development.

Instructions

You will test out your HW 4 prototype with a real participant. This can be a friend, family member, or even stranger. The only requirements are that they should be at least 18 years of age and are not in the VR class.

You will have them come in, participate in the study, and then sign a form signifying that they participated.

Carefully read through the instructions below:

Before the study

  1. Download and print out the user questionnaire and verification form.
  2. Build your app and deploy it onto one of the headsets. Make sure to set up a boundary.
  3. Make sure that you are set up to stream from the headset onto your computer.
  4. Prepare software to record the stream on your computer.
  5. Get a notepad or note-taking app ready to take notes during the study.

During the study

  1. Welcome the participant and introduce yourself.
  2. Describe the purpose of the study.
    • "I developed an application for a VR class and would like to get some feedback on it. If you would like to participate, I will ask you to wear the headset and use the application. While you are wearing the headset, it would be helpful if you talked aloud about your thoughts and give a commentary on what you are doing and why. Once you are finished, I will ask you to complete a questionnaire and answer some short interview questions. I will be audio and video recording the study. You may choose to stop participating at any time."
  3. Ask the participant if they have any questions.
    • "Do you have any questions?"

  4. Ask if they would like to participate (it's important that we give them this choice!).

    • "Would you still like to participate?"

    Read More: Informed Consent Process

    Why do we do all this explaining? Why do we say that the participant can stop participating at any time? Why do we ask them again if they want to participate?

    It's all related to one thing: Informed Consent

    In the real world, a study like this would also require following a strict protocol approved by an ethics review board. Participants would typically read and sign a "consent form" before starting. While class projects don't need approval, it's important that we still follow these steps and wording!

  5. If they agree, you can start setting up the headset and recording.

  6. Help the participant put on the headset and hold the controllers. You may give a brief description of the controls.
    • e.g., "You see this thumbstick? Press it forward to aim and then release to teleport."
  7. Tell them the goal and remind them again to "think aloud".
    • e.g., "Your goal is to play through the experience until you see a game over screen. Please think aloud about everything you are doing and why. It's super helpful to help me understand what you are doing!"
  8. Observe the participant and take notes. If they ask for help, its okay to help them. Don't forget to ask questions too!
    • Note anytime they had trouble, anything surprising, things that went well, and anything else that happens! More detail is better than less.

After the study

  1. Have the participant take off the headset and put the controllers aside.
  2. Present them with the questionnaire and ask them to fill it out.
    • "Please fill out this questionnaire. You may skip any questions you do not wish to answer. Please let me know once you've finished it or if you have any questions."
  3. Once they are done, ask the following interview questions and take notes (required). You may also expand upon the questions from the questionnaire or other observations you made.
    • Health: How do you feel right now, physically? Any discomfort?
    • Liked: What did you like most about the experience? Why?
    • Disliked: What did you like least about the experience? Why?
    • Features: Is there anything you wanted to experience that you weren't able to?
    • Other: Do you have any other comments or suggestions to make the experience better?
  4. Thank them for participating in your study.
  5. Have the participant sign the verification form.

Analysis

Afterwards, take a look through your notes. What went well? What difficulties did the participant have? While you don't need to reflect for this assignment, you will need to do so for the group project!

Submission

Please submit scans of the notes you took, the completed questionnaire, and the verification form for credit.

Grading

I will grade based on submission of the three components. As long as you made an honest attempt on all three components, you will receive full credit.