Activity 6: Spatial Audio

Objectives:

Compare and contrast different types of audio sources (2D, 3D, Spatial)
Identify the limitations of simple stereo 3D audio
Understand how perception of audio changes based on location
Utilize spatial audio sources to locate objects

This activity will have you take turns putting on the VR headset and reporting your findings to the rest of your team. The rest of the team will ask questions and guide you through the activity. Ideally, you should have two laptops ready, one with this activity, and another for viewing the headset's screen.

Team Members:

Getting Started

Have one person wear the headset and hold the controllers (you should stand for this activity). Make sure you recenter the view by holding the Oculus () button on the right controller (the one that looks like a horizontal oval).

Tip

Every time someone else puts on the headset, you should reset the view by holding the Oculus () button.

Cast the headset view

Warning

As of 11/14/2023, casting (and recording) is missing from the class headsets. You should take turns using the headset and trying the demos.

Launch the App

Open the app menu, then at the top, click the dropdown next to the search bar and find "Unknown Sources". Launch the SpatialAudioDemo app.

Read the instructions to the person wearing the headset (quietly) and answer the questions:

Part 1: Simple Tests

You should see an orange area with three sets of buttons. You can use the left thumbstick for direct movement, or the right thumbstick to teleport. Stand in front of the Stationary Test.

One at a time, press each of the buttons (use your finger to poke it or point and use the grip button) to play the audio sources. Start with 2D Audio, then 3D Audio, then Spatial Audio. As you listen to the sample, rotate your head back and forth. Play each sound multiple times if you need.

How is 2D Audio different from 3D and Spatial Audio?

Between 3D and Spatial Audio, which one more accurately sounds like it is coming from its respective audio source in the world?

Which one is more accurate?

Let's do another test. Turn to the left and repeat the steps with the Verticality Test.

Which one accurately sounds like it is rising upwards?

If you need to, try closing your eyes and listen to where the sound is coming from.

One more test. Turn around and repeat the steps with the Orbiting Test.

For the 3D Audio, can you hear a difference between when the orb is in front vs. behind you?
What about the Spatial Audio? Any difference here?
How do you think this illusion is achieved?

That concludes the first series of tests! The differences between the three audio sources is as follows:

2D Audio: Traditional audio source that plays through both speakers and does not change based on head position and rotation.
3D Audio: Changes volume of each ear's audio based on which ear is facing towards the source.
Spatial Audio: Advanced processing of the audio source that takes into account the user's head position, rotation, movement, and the audio's location in the world.

Please pass the headset and have your teammates try out all three tests.

Any other comments?

Part 2: Spatial Audio

Now, move to the Distance Test to the right of the Orbiting Test.

Just like before, press each of the buttons to play the sounds.

Which audio source(s) sound like they're getting further?
How do you think this is achieved?

Next, head back and enter the purple room. Stand in front of the Wall Reflection test. Press each of the buttons to play the sound and rotate your head back and forth to hear where it is coming from.

What is the difference with/without reflections?

Reflections are when audio bounces off a wall and propagates through a room. The reflections should have made it sound like the speech was coming from an entire area, not just a single point.

Now, turn around and try the Occlusion Test. Press the "Play Spatial Audio" button and listen closely.

What happens when the audio source goes behind the blue wall?

With the audio still playing, go ahead and move closer to the blue wall.

When it goes behind the wall, where does the audio sound like its coming from?

Hopefully, you noticed that the audio was slightly muffled when behind the wall, but you could hear it reflect off the wall on the left. Spatial audio allows us to not only simulate head direction, but also simulate the acoustics of an entire 3D space.

Press the "Stop Audio" button and then pass the headset and have your teammates try out all three tests.

Any other comments?

Part 3: Localization Game

Now it's time for some fun! Exit the purple room and move to the Locate the Sound game on the left.

When you press one of the buttons, a sound will start playing from a random location. Your goal is to determine where the sound is coming from and walk up to it.

Start with the 3D Audio button.

Were you able to find the source?
How did you do so? Was it easy?

After that, press the "3D Audio (Maze)" button. A maze should appear. Now, try and find the sound in the maze. You may want to play a couple of times to challenge yourself.

Were you able to find the 3D Audio source in the maze?
How did you do so? Was it easy?

Simple 3D Audio doesn't feature any reflections nor any occlusions with walls. It only uses the user's distance and their head rotation to determine how to play the sound in each ear.

Last, press the "Spatial Audio (Maze)" button. Try and find the sound now. It may take a few tries!

Were you able to find the Spatial Audio source in the maze?
How did you do so? Was it easy?
Were you able to tell if the sound was behind a wall or coming down a corridor?

Spatial Audio uses ray tracing to propagate down a hallway and transmit through walls. This can be very compute-intensive, so its best if only a few audio sources are active at once when you have these special effects enabled. But overall, it could be worth it for the added realism!

Pass the headset to your teammates and have them try finding the sounds.

Any last comments?

Submission

Click the Export as PDF button to create a PDF of your answers. Have one person upload it to Gradescope. After the PDF is uploaded, click the "View or edit group" link in the upper-right and add your teammates in the Gradescope assignment.