What is the canonical procedure to convert hexadecimal PCM to Float32Array for processing at AudioWorkletProcessor?
guest271314 opened this issue · 1 comments
Describe the feature
Decode hexadecimal input to Float32Array capable of being copied to an existing AudioBuffer or processing with AudioWorkletProcessor. Encode Float32Array to hexadecimal.
From V1 repository
If you need help on how to use WebAudio, ask on your favorite forum or consider visiting
the WebAudio Slack Channel (register here) or
StackOverflow.
Visited web-audio.slack.com first, yet apparently an "invitation" is required
Don't have an account on this workspace yet?
Contact the workspace administrator for an invitation
If this is issue is deemed off-topic for this repository kindly post the "invitation" to the slack page so can ask the question there.
Banned from SO for 4+ more years.
Is there a prototype?
No completely. Though found this example https://babbage.cs.qc.cuny.edu/IEEE-754.old/32bit.html today.
A sampling of research efforts thus far:
- https://stackoverflow.com/questions/42699162/javascript-convert-array-of-4-bytes-into-a-float-value-from-modbustcp-read
- https://stackoverflow.com/questions/15935365/convert-float-to-bytes-in-javascript-without-float32array/16043259#16043259
- https://stackoverflow.com/questions/57803/how-to-convert-decimal-to-hexadecimal-in-javascript
- https://stackoverflow.com/questions/39275175/how-can-i-convert-javascript-float-pcm-audio-array-to-little-endian-byte-string
- https://stackoverflow.com/questions/39275175/how-can-i-convert-javascript-float-pcm-audio-array-to-little-endian-byte-string/40660469#40660469
- https://stackoverflow.com/questions/25371713/playing-raw-audio-pcm-samples-in-web-audio
- https://github.com/samirkumardas/pcm-player/blob/master/pcm-player.min.js
- https://stackoverflow.com/questions/49221045/javascript-hexadecimal-string-to-ieee-754-floating-point
Describe the feature in more detail
Provide more information how it does and how it works.
Decoding PCM from hexadecimal to Float32Array and encoding Float32Array to hexadecimal PCM (or other audio codec).
The use case is creating N discrete files, for example using a WebM container template where the values can be read from or written to the template for the purpose of "streaming" (encoding and decoding, reading, writing) those N files for "seamless" playback using AudioWorklet without using decodeAudioData(). Since Chromium is capable of playing Matroska(TM) container with PCM codec (MediaRecorder.isTypeSupported("audio/webm;codecs=pcm")) and Firefox is currently moving towards supporting WebM with FLAC encoded audio (https://bugzilla.mozilla.org/show_bug.cgi?id=1606822) this will provide a means to encode and decode Matroska and WebM files "on the fly" for various use cases.
Sample input
Header (in an XML file output by mkv2xml )
<mkv2xml>
<EBML>
<EBMLVersion>1</EBMLVersion>
<EBMLReadVersion>1</EBMLReadVersion>
<EBMLMaxIDLength>4</EBMLMaxIDLength>
<EBMLMaxSizeLength>8</EBMLMaxSizeLength>
<DocType>matroska</DocType>
<DocTypeVersion>4</DocTypeVersion>
<DocTypeReadVersion>2</DocTypeReadVersion>
</EBML>
<Segment>
<Info>
<TimecodeScale>1000000</TimecodeScale>
<MuxingApp>Chrome</MuxingApp>
<WritingApp>Chrome</WritingApp>
</Info>
<Tracks>
<TrackEntry>
<TrackNumber>1</TrackNumber>
<TrackUID>66451940004784068</TrackUID>
<TrackType>2</TrackType>
<CodecID>A_PCM/FLOAT/IEEE</CodecID>
<Audio>
<SamplingFrequency>22050.0</SamplingFrequency>
<Channels>1</Channels>
<BitDepth>32</BitDepth>
</Audio>
</TrackEntry>
</Tracks>
Hexadecimal values within a <data> tag
0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
9201c93b4201a13b4201a13b9201c93be001703be001703b00000000000072bb
004017bd009080bd1a018d3c7f813f3d8981443de201f13b00c0ddbce001703b
0601833ccf81e73c9201c93b000000006a01353ce381713c1a018d3c4001a03a
4201a13be201f13b6a01353c4201213ce201f13b19810c3c9201c93b6a01353c
e381713cba015d3c19810c3c4201213c9201c93be201f13b9201c93be201f13b
6a01353ce201f13b19810c3c4201a13b9201c93b9201c93be201f13b9201c93b
4001203b9201c93b4001203b9201c93be001703b4001203b000000004001203b
6a01353c0080c9bc00602bbd9201c93b2e21173d6041303d00805dbc004021bd
e001703ba7a1533d0181803d7e41bf3c00401cbd00208dbd2e01973c4c41263d
ba015d3c4201a13b6a41b53c6b61353d56412b3d9201c93b00c0e7bc0601833c
cf81e73c6041303df801fc3c000000001a018d3c6a01353c1a018d3c0000fcbc
00a049bd00200dbd008049bc0601833ce201f13b4001a03a19810c3c9241c93c
ba015d3c4201213ca781d33c0000f2bb008035bc4201a13b000072bb0601833c
19810c3c000022bbe381713c9201c93b0000cabb0000a4ba1a018d3ce4c1f13c
0601033d9201c93b0080bfbc4201a13ce001703b9201493c4201a13b00000dbc
0080c9bc4201213ccf81e73c00805dbccf81e73c008049bc008035bc5641ab3c
7e41bf3c1a010d3d4001a03a008035bc4001203be4c1f13cf801fc3c4001a03a
0000a4ba008035bc0000a1bb0000a4ba5641ab3c00408dbc0601833c00000000
2e01973c19810c3c0000000000000000000000009381493d4201213ccf81e73c
9201c93b008049bc75613a3d0000000038211c3d6a01353c1a018d3ca781d33c
ba015d3c2e01973c000000002e01973c008049bc00c062bd0000f2bbbb81dd3c
e4c1f13ce201f13bba015d3c4201a13be381713c00000000000000002e01973c
bb81dd3c008035bc7e41bf3c9201c93b008035bce4c1f13c0000a4bae001703b
e201f13b0040a1bc19810c3c19810c3c0000f2bb4201213c4201a13bba015d3c
ba015d3c008035bc9201c93b4001a03a5641ab3c9201c93b0000a4ba000072bb
0000a1bb1a018d3c6a01353c000000004201a13be001703be381713ce001703b
004097bc1a010d3d00000000000021bc0601833c00805dbc4201a13c0000a4ba
4201a13bba015d3c9201c93b4201a13c0080c9bcbb81dd3c1a010d3d0080b5bc
6a41b53c00c0d3bce4c1f13c9381493d00409cbdeee1763d9201c93b0000cabb
0b81853d0080c9bc4201213ce001703b4001203bcf81e73c004097bc0000a1bb
e4c1f13c00805dbc4c41263de201f13b0080c9bc75613a3d00c0e7bc1a018d3c
e201f13b0080b5bc9201493ce4c1f13c000072bc0040a1bc0601833c2e01973c
008035bce001703bc5a1623d00e06cbd0000000038211c3dba015d3c0080b5bc
0080abbcc5a1623d000022bb000000000080abbc19810c3ccf81e73c004097bc
e201f13b4201a13ba781d33c0000f2bb19810c3c00c0ddbc000072bbc5a1623d
000021bc0080b5bca781d33c4001a03a0601833c0601833c004083bc000021bc
19810c3c1a018d3c000000004201a13c000022bb5641ab3c0000a4ba0040a1bc
9241c93c4201213c4001a03a0000a4ba9241c93c0040a1bcbb81dd3c8981443d
004097bda7a1533d4001a03a0080c9bc1a118d3d0080bfbc00805dbc4001a03a
00a03fbd4c41a63d008035bd000072bbcf81e73c00401cbd0601033d002003bd
7e41bf3c00c0d3bc2e01973c9201493c002012bdf801fc3c9201c93b4201213c
4c41263d9201493c00000dbc000072bb56412b3d4001a03ae381713c0601033d
f801fc3c00000dbc7e41bf3c002012bdf8017c3d4001a03a2e01973c9241c93c
008035bd8981443d7e41bf3ce201f13b2421123d00602bbd0080abbc1011883d
19810c3c008049bcba015d3c000022bbe201f13b0601833cf801fc3c00200dbd
e4c1f13c0601033d004097bd0601833d56412b3d00e0a8bde201f13b2e21173d
4221213d4201a13b00805dbc00c0d3bc29a1943d00408dbcba015d3c2e01973c
004021bd1a118d3d4001203b00e067bd2e31973d004017bd000021bc0181803d
00a044bd1a118d3d4001a03a00c0d8bdfd81fe3d0000fcbc1a018d3c7e41bf3c
0080bfbcdac16c3d9201c93b0080bfbc2e01973cba015d3c5641ab3c4001203b
00e06cbdc5a1623d9201493c4001203be001703b00408dbc7f813f3d4001a03a
00408dbc5641ab3c19810c3c4001203b0080b5bc4201a13c56412b3d4001203b
6a01353c4201a13c009085bdb631db3d00e06cbd1a018d3cbba15d3d00f0b2bd
e4e1713d2421923d00a0c9bd2421123da7a1533d0060b0bde4f1f13d00b094bd
e4e1713dba015d3c00409cbdca41e53d000072bde4c1f13c66e1b23d00a0cebd
00602bbd4001203e00401cbd000072bb0000000000c0d3bc1a010d3d6041303d
e001703b00200dbd00000000002003bd4201a13c0601833c5641ab3c2e21173d
that we can retrieve using DOMParser()
fetch("data.xml")
.then(response => response.text())
.then(xml => {
const res = [];
const parser = new DOMParser();
const doc = parser.parseFromString(xml, "application/xml");
const mkv = doc.documentElement;
const str = [...mkv.querySelectorAll("data")].map(({
textContent
}) => textContent.match(/\w+/g)).flat().join("");
console.log(str);
})
though am not certain about the canonical procedure to convert the above values to Float32Array(s).
Thanks to the author of https://github.com/vi/mkvparse we know that the input is playable using ffplay
$ cat mediarecorder_pcm.xml | xml2 | grep '/mkv2xml/Segment/Cluster/SimpleBlock/data=' | cut -f2-2 -d= | xxd -r -p | ffplay -f f32le -ar 22050 -ac 1 -.
Expected output is a Float32Array 1/4 the length of input hexadecimal string input that is suitable for playback using Web Audio API (i.e., within AudioWorkletProcessor).
Is there a simpler algorithm to convert the hexadecimal values than the approach at https://babbage.cs.qc.cuny.edu/IEEE-754.old/32bit.html?