Sound fluctuates after turning off Voicing on iPad

Question

Sound fluctuates after turning off Voicing on iPad

Closed this issue 2 years ago · 9 comments

Nancy-Salpepi commented 2 years ago

Test device
iPad 9th generation

Operating System
iOS 16.3.1

Browser
safari

Problem description
For phetsims/qa#923, after turning off Voicing the Shape Sounds will fade in and out.

Steps to reproduce

Move a side/corner to begin sound
Before sound fades, turn on Play Shape Sounds Forever
Turn on Voicing
Turn off Voicing and exit Preferences

Visuals
Here is what is sounds like after the steps above:

SoundFlux.mov

Troubleshooting information: !!!!! DO NOT EDIT !!!!! Name: ‪Quadrilateral‬ URL: https://phet-dev.colorado.edu/html/quadrilateral/1.0.0-dev.76/phet/quadrilateral_all_phet.html Version: 1.0.0-dev.76 2023-03-28 17:13:45 UTC Features missing: applicationcache, applicationcache, touch Flags: pixelRatioScaling User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/16.3 Safari/605.1.15 Language: en-US Window: 1342x705 Pixel Ratio: 2/1 WebGL: WebGL 1.0 GLSL: WebGL GLSL ES 1.0 (1.0) Vendor: WebKit (WebKit WebGL) Vertex: attribs: 16 varying: 30 uniform: 1024 Texture: size: 16384 imageUnits: 16 (vertex: 16, combined: 32) Max viewport: 16384x16384 OES_texture_float: true Dependencies JSON: {}

Answer 1 · 2023-03-29T13:11:59.000Z

It also fluctuates when Voicing is on and nothing is being read.

Answer 2 · 2023-03-29T14:07:00.000Z

Thanks @Nancy-Salpepi - Just in case, I tried on Win 10 Chrome but did not see it there so Ill try on iOS. This is probably a common code problem with audio ducking + speech synthesis (phetsims/tambo#167)

Answer 3 · 2023-03-29T17:20:54.000Z

I am able to reproduce this with iOS 15.0.2 Safari. I could not reproduce this in macOS 11.7.2 Safari.

Answer 4 · 2023-03-29T17:37:29.000Z

My first thought was that the synth was trying to speak with empty strings, causing the soundManager to duck/un-duck over and over. But I put a log statement here: https://github.com/phetsims/tambo/blob/4e4aab289995d7f6e3ddb0f77116618a6aa33b43/js/soundManager.ts#L257-L267 and I do not see updateDuckingState getting called while the sounds fluctuate.
I removed this code and the problem was still there: https://github.com/phetsims/tambo/blob/4e4aab289995d7f6e3ddb0f77116618a6aa33b43/js/soundManager.ts#L279-L284. I verified caches were cleared.
With that code commented out the volume still fluctuates. Is this a problem with quadrilateral sound code? Is this a problem with Apple's built-in audio ducking???
I verified that it the gain of quadrilateral tracks are not fluctuating.

Answer 5 · 2023-03-29T19:49:00.000Z

I found it. It is caused by this workaround that was added for Chromebooks: https://github.com/phetsims/utterance-queue/blob/36fdea460883369af36712e1aefef89bf2e25e0d/js/SpeechSynthesisAnnouncer.ts#L472-L483

What must be happening is that iOS has built-in audio ducking for speech synthesis. That is why removing PhET's audio ducking didn't change the behavior. The empty string in that workaround still triggers the ducking even though there is no speech.

I am guessing this was also introduced when the workaround string became an empty space for phetsims/friction#328.

Answer 6 · 2023-03-29T20:07:40.000Z

I just noticed that this is a problem for all iOS browsers, not just Safari. So if we opt out of that workaround we need it to be for all iOS which we don't currently have a check for in platform.js.

Answer 7 · 2023-03-29T21:14:26.000Z

A fix has been proposed in phetsims/utterance-queue#110. @Nancy-Salpepi can you please test again and confirm this is fixed?

Answer 8 · 2023-03-30T12:48:58.000Z

The sound sounds fixed in master. haha 🙃

Answer 9 · 2023-03-30T12:55:40.000Z

Excellent, thanks!