Five channels?

Unsure if this is a mistake, but WWWT in North-Central Virginia is now showing a total of 5 HD channels:

HD4 isn't generating any audio in nrsc5, so who knows what's really going on. Of course, trying to access HD5 in nrsc5 is a non-starter.

Could you make an I/Q recording (with the -w command line option) and share it? A minute or two of data should be enough.

will do this afternoon.

@argilo Too big for Github, so I left it at <link removed>.

I get "Access Denied" when I open that URL.

yeah... hang on. My security system.

try <link removed because of spider bots>. As if gzipping helped shrink it much...

Got it, thanks.

👍🏼

Interesting! The station is using Primary Service Mode 11. I've only ever seen modes 1 and 3 used in the field, and those are the only modes currently supported by nrsc5. We'll have to add support for mode 11 to get HD4 and HD5 decoding.

In case you're interested, the differences between the various service modes are described here:

https://www.nrscstandards.org/standards-and-guidelines/documents/standards/nrsc-5-d/reference-docs/1011s.pdf

Didn't even think to look at the service mode, just thought maybe someone at the station set something wrong and mistakenly put HD4 into some fictional HD5.

Also means I'll have to add a new tab/info column on dui.

Because service mode 11 is backwards compatible with service mode 3, the following change is enough to get HD4 working:

diff --git a/src/sync.c b/src/sync.c
index e00d567..a4a9488 100644
--- a/src/sync.c
+++ b/src/sync.c
@@ -464,7 +464,7 @@ void sync_process_fm(sync_t *st)
                     decode_push_pm(&st->input->decode, DEMOD(cimagf(c)) * mult_ub);
                 }
             }
-            if (st->psmi == 3) {
+            if ((st->psmi == 3) || (st->psmi == 11)) {
                 for (i = LB_START + (PM_PARTITIONS * PARTITION_WIDTH); i < LB_START + (PM_PARTITIONS + 2) * PARTITION_WIDTH; i += PARTITION_WIDTH)
                 {
                     unsigned int j;

Some additional changes will be required to get HD5, which is in a separate logical channel that is not present in service mode 3.

I added support for service mode 11 (argilo@fbb9179), and it even works correctly with a signal generated by https://github.com/argilo/gr-nrsc5, but it does not receive WWWT's HD5 channel. (The bits in the P4 logical channel appear to be garbage.) So it would appear that I misinterpreted something about service mode 11 in both nrsc5 and gr-nrsc5.

Unfortunately, both of my hardware receivers appear to be first-generation receivers that are incapable of service mode 11, so I cannot double check with a real receiver.

@markjfine Do you have a hardware receiver that can receive WWWT's HD5 channel? It would be nice to confirm that the signal is correct before I carry on trying to find the bug in nrsc5 & gr-nrsc5.

Also means I'll have to add a new tab/info column on dui.

The specification allows up to eight channels. Apparently WSTR has seven: https://en.wikipedia.org/wiki/WSTR_(FM)

It would probably make sense to add keyboard shortcuts for the remaining channels; nrsc5 only has shortcuts for the first four at the moment:

nrsc5/src/main.c

Lines 492 to 497 in b969965

    
           case '0': 
        
           case '1': 
        
           case '2': 
        
           case '3': 
        
               change_program(st, ch - '0'); 
        
               break;

@argilo
Pretty sure the signal is correct. However, you should know that I made the change you suggested and the audio for HD4 came through. It didn't previously. So it would seem your original assessment was correct, at least for HD4. It's possible that HD5 is just blank for now. I can always add case '4' - '7' and see if anything comes through live.

It's possible that HD5 is just blank for now.

Yeah, that's why I'd like to get some confirmation that there's actually an HD5 channel there before I spend a bunch of time debugging. Let me know if you find out. I'll see if I can get my hands on a newer receiver that is capable of mode 11.

Ok.. So you gave me an idea. I remembered my wife's car radio has ibiquity on it. I checked WWWT and although there's a SIG Service marker for HD5, only 1-4 show as accessible on the car radio. HD4 plays on it, so chances are it does do service mode 11. So, I'm going to conclude the SIG marker is just a placeholder and there's no real service there.

There's a somewhat similar situation on a completely different station (WKYS) where HD2 went dark, but I think it still has a SIG Service marker for it when using nrsc5. When I look at it on my wife's car radio it only shows HD1 and HD3.

All things being equal, that would seem to confirm that the car radio has the smarts to not even tune to channels with no data, regardless of what the SIG Service (or Audio program marker for that matter) says.

HD4 plays on it, so chances are it does do service mode 11.

Not necessarily. WWWT puts HD4 in the P3 logical channel, and the P3 logical channel is backwards compatible with service mode 3. The NRSC-5 specification has a backwards compatibility table which says that if the receiver doesn't understand mode 11 it should treat it as mode 3 and decode the P3 logical channel.

My first-generation receivers both decode WWWT HD4 without issue. (I replayed your I/Q recording to them using a HackRF.)

nrsc5 doesn't (yet) implement the backwards compatibility table, which is a bug. My patch above goes partway to fixing it, but I'll open a pull request with a more complete fix soon.

#288 should fix the issue with WWWT HD4.

Just expanded dui to work with 8 channels, but haven't pushed it yet.

#288 does fix the backward compatibility for HD4. However, would like to leave this open as a note for fully expanding nrsc5 to 8 channels, if that's feasible and practicable. Was also wondering about the number of data channels and possibly saving off Navteq files, but that's another ticket.

However, would like to leave this open as a note for fully expanding nrsc5 to 8 channels, if that's feasible and practicable.

Agreed on leaving this open for now, since nrsc5 does not yet decode logical channel P4, which is presumably where WWWT HD5 is lurking. I'll have another look at that today, but it may have to wait until I find a receiver that is confirmed to support MP11.

Also, we may as well add keyboard shortcuts for HD5-HD8.

One last issue is that nrsc5 still does not support the other service modes (MP2, MP5, MP6, MS1, MS2, MS3, MS4) but I'll open a separate issue for that.

Cheers.
As an aside, the regular WTOP has Federal News Network (FNN) as one of three channels. WWWT (a relay of WTOP) has it listed as HD2 and even provides the station art for it. In actuality the audio is a feed for La Rey, a Spanish language channel. Could be that HD5 either is FNN or a planned shuffling of the deck. All of this confusion is why I originally thought there might have been a configuration error at the station.

#292 allows all eight programs to be selected at runtime.

BTW: Noticed you're testing on macOS 11 & 12. I've been building and running on 13/aarch so if that's an unofficial test, it passes, even with the new hokey homebrew install in /opt.

Works...

Bah... forgot about the other service modes...

I've got a prototype MP11 receiver on the go on the add-mp11 branch: https://github.com/theori-io/nrsc5/compare/add-mp11

As before, it works with a signal generated by gr-nrsc5, but does not work with the recording of WWWT. @markjfine Perhaps you could build the branch and test again with WWWT, just in case something has changed with the broadcast itself since the time you made the recording. Basically I'd like to know whether there's any sign of HD5 (i.e. program 4).

What I see with the WWWT recording is that the Viterbi decoder reports ~1150 bit errors per 9216-bit packet packet, which is the same number of errors you get if you just put random bits into the decoder. That suggests that one of the earlier steps is broken, but there aren't many:

OFDM demodulation: The signal-to-noise ratio is high, and I get a clean QPSK constellation when looking at the symbols in the P4 logical channel, so I doubt this is the problem.
OFDM subcarrier mapping: The mapping for the P4 logical channel is very similar to the P3 logical channel (except shifted 38 subcarriers closer to the center). I printed out the subcarrier numbers and they match the specification.
Deinterleaving: The P4 logical channel uses identical interleaving to P3, so there's not likely a problem here.

I'm at a loss for what's happening with WWWT. Even if there was no data present in the P4 logical channel, I'd at least expect to see valid convolutional coding and no bit errors reported by the Viterbi decoder.

07:52:55 Audio program 4: public, type: None, sound experience 0
07:53:01 SIG Service: type=audio number=5 name=HD5
07:53:01 Audio component: id=0 port=0004 type=0 mime=4DC66C5A
07:53:01 Data component: id=1 port=1008 service_data_type=265 type=3 mime=BE4B7536
07:53:01 Data component: id=2 port=1009 service_data_type=265 type=3 mime=D9C72536

The markers are still there, but it's still not generating any audio, just MER, BER, and the occasional LOT. I may just call (or email) the station engineer to ask what's going on, but I doubt I'd get a response until Monday the earliest. Would still beat trying to shadow-box this thing. Maybe I'll get lucky.

As you may have seen (you are cc'd), I sent the tech rep for WTOP an email as I can't get a specific poc for WWWT. Is it possible that HD5 could feasibly be encrypted and being sent as intended, just not for public consumption? 👀
Not being paranoid. I just know things and the thought just occurred to me... lol

Maybe it's a DGPS correction signal.

Why not use a data stream for that?

Interestingly, the bits that are identical in every deinterleaved P3 transfer frame are also coming out the same in each deinterleaved P4 transfer frame. The remaining bits seem to be randomly corrupted, and do not form a valid convolutionally-encoded bit sequence. This suggests to me that bits from different P4 transfer frames are getting mixed together. The interleaver mixes together multiple transfer frames and the deinterleaver unmixes them, so a possible explanation is that the P4 interleaver is somehow different from the P3 interleaver. (But as far as I can tell, the specification says they should be identical.)

After some further analysis I determined that each P4 transfer frame gets its bits mixed with the one that arrived 7 frames earlier. I expect some adjustment to the interleaver formulas could correct that, but for now I added a very ugly hack that keeps track of the previous 7 frames and swaps in the correct bits. It's available on the add-mp11 branch: https://github.com/theori-io/nrsc5/compare/add-mp11

With that hack in place, I can now see that WWWT HD5 has the same audio as HD2 (although quieter for some reason).

23:34:20 Title: La Ley 106.3 FM
23:34:20 Artist: WWWT-FM HD5
23:34:20 XHDR: 1 BE4B7536 -1
23:34:20 Audio bit rate: 23.1 kbps

I have to say that how you're able to see these kinds of things and analyse at that level of complexity simply amazes me.

I'll update what I have and see what pours out. I'd also be interested to see what comes out of the data channels, if anything. As I previously mentioned, HD2 is providing the cover art for FNN, not La Ley.

Well, hack or not, it's certainly pulling it in now. As you showed, it says HD5 in the Artist tag. Interestingly, the XHDRs and files being transmitted are cover art and the station logo for HD3, The Gamut. There's a station logo for HD1, WTOP as well, but it's seldom transmitted.

What might interest you is that HD3 is a feed of WWFD 820kHz - one of the few digital-only AM stations. Unfortunately, I can't pick that up on the longwire clean enough to test nrsc5's --am mode. They're also reduced power at night, which makes it even harder.

After some further analysis I determined that each P4 transfer frame gets its bits mixed with the one that arrived 7 frames earlier. I expect some adjustment to the interleaver formulas could correct that [...]

I thought about this some more, and I'm not sure a change to the interleaver parameters would solve this. The fact that bits from 7 frames apart appear at the same time implies a much longer interleaver depth than what's listed in the NRSC-5 specification.

I'd like to confirm with a hardware receiver before deciding what to do with the hack. If a hardware receiver can decode WWWT but not gr-nrsc5, it would suggest that the hack I added in is correct and the NRSC-5 specification is incorrect. If it's the other way around, then WWWT's transmitter is broken. If it receives both, then there must be a bug somewhere else in nrsc5.

I'll be visiting my parents this weekend and they have at least one vehicle with an NRSC-5 receiver, so I'll see what it thinks of MP11 signals.

Unfortunately, I can't pick that up on the longwire clean enough to test nrsc5's --am mode.

nrsc5 does not yet support all-digital AM (service mode MA3). It's on my to-do list to add that. I already have a recording of WWFD to test with, plus gr-nrsc5 is capable of generating MA3 signals.

I remembered my wife's car radio has ibiquity on it.

@markjfine Do you also have a transmit-capable SDR? If so, I could send you a GNU Radio flowgraph that you could use to generate an MP11 signal and see whether the car can receive the P4 logical channel.

Think we have as many bases as can be covered at the moment. Still haven't heard back from the Tech Rep, but hopefully I worded it in such a way that he might at least consider investigating it. So we still have that avenue open.

No transmit-capable here. Just an RTL-SDR and AirSpy HF+.

Actually, I did get a response yesterday - from Dave Kolesar. I just didn't see it because I stupidly forgot to whitelist their domain (you have no idea how much spam I get). 🤦🏻‍♂️ Asked him to resend, just in case he didn't reply all.

Could you forward the message to me? I haven't seen it yet.

Lol. Neither have I. Asked him to resend. I saw the header in my email log, but the guts of it went to /dev/nul. Usually that wouldn't happen in this situation, but apparently the station's provider is on a domain I once flagged and it triggered a qmail action to dump it instead of sending to junk.

Either he's busy or it was just a short note to say 'hi' (I think I know him from somewhere) and that he's investigating. He'll probably re-respond when he knows something. I'll be sure to forward to you when I see something.

I came up with a somewhat less hacky workaround, which is now on the add-mp11 branch: 9fb7186

Instead of modifying the interleaver, I moved the hack into the subcarrier mapping step where it takes a simpler form. I've added a 14-block delay to P4 bits before they are copied into the PX2 deinterleaver. This extra delay applies to even-numbered blocks only.

This implies that the transmitter is delaying P4 bits by 14 blocks, in odd-numbered blocks only. Again, I can find no explanation for this in the NRSC-5 standard. I suspect the transmitter is buggy, but I'll need to test with a hardware receiver to confirm that.

It works well. Seems the intent was always to move La Ley to HD5, since HD2 is now a different Spanish-language station called La Power. Think that's (ironically) a low power FM outlet (former TV 6 audio at 87.75MHz) WDCN-LP out of Bowie, MD.

Still no re-response. At this point I'm going to now guess that email minimally said "I'm going on leave, but I'll look into it when I come back", which more than likely will be after the 1st of the year... Kicking myself for not remembering to whitelist the domain.

I just tested two vehicles (2017 and 2023) and neither showed WWWT HD5. Only HD1 through HD4 were visible.

The vehicles were also unable to receive the P4 channel of the test signal I produced with gr-nrsc5.

So it's starting to look like most receivers can't handle the extra logical channel in MP11 broadcasts.

Unfortunately this leaves me no closer to knowing whether WWWT's signal is correct.

Yeah, my wife's Escape is a 2017 and could only see 1-4.
Would be nice to be ahead of this, but it looks like we'll have to wait for Kolesar.

Lol... Just checked WWWT and now HD2 is 25kbps of dead air.

Heard from Dave and forwarded to your gmail address.

In case you hadn't seen it:
https://www.radioworld.com/tech-and-gear/digital-radio/hubbard-turns-on-an-hd5-audio-channel

Looks like a case of "build it and they will come". It will be an uphill battle getting HD Radio manufacturers to implement HD5 in hardware.

Looks like a case of "build it and they will come". It will be an uphill battle getting HD Radio manufacturers to implement HD5 in hardware.

Even if they should start building it today, how many years would it take for in-car radio's and home radio's to be replaced with something newer since I doubt people would be eager to replace their still working radio's just to receive additional streams? I have a now rather old Insignia NS-HD01 Portable HD Radio that receives four HD channels. Works great and I have never really seen anything comparable to it since it was discontinued long ago so I doubt Insignia will start up the productions lines to make a newer version of it. Of course it also goes down the same old rabbit hole of obsolescence of otherwise perfectly good digital hardware. In ten years or so, we will be scrapping any still functioning ATSC 1.0 digital TV's for ATSC 3.0 compatible models (unless one wants to use a converter box). Funny thing is I have an old Meissner 8C FM tuner from around 1950 that had belonged to my late father. It was one of the early "high band" FM radios from when the FCC moved FM to 88-108 MHz. It does not receive stereo obviously but it still works and receives FM radio stations even though it is now 73 or so years old. Unless analog FM is shut down or the band moved (again), it should still keep working as long as the electronics hold up. I wish I could say the same for modern digital hardware.

@Whobeu

I totally agree but I do have to admit that I love the enhanced audio quality of HD Radio and I love being able to view the album art, traffic and weather maps. As a guy who came of age in the 1980s listening to analog broadcasts, HD Radio almost seems like sorcery! LOL!

I would tend to believe that in-dash manufacturers left the firmware of those receivers open to be updated (much like Ford's Sync system) either by the user using download/USB techniques or at the dealership for a modest labor fee. Shouldn't be any harder than that, really. The real question is if those manufacturers see the return on investment into developing the firmware modifications and upgrade procedures that will determine whether it happens or not. We'll see.

It's sort of a chicken and egg problem. Until consumers start asking for HD5, manufacturers won't invest any time and money into new firmware and with only 1 station in the US broadcasting HD5 and no current HD radios supporting HD5, consumers don't even know it's there to take advantage of it.

I didn't even know that spec called for more than 4 channels, which is how this thread got started. Perhaps by taking the first step, WWWT tells the other stations, "come on in, the water's fine". Given enough stations do the same, it creates enough of a ripple effect that causes user demand, and consequently a manufacturer response.

TBH I wished that was the case with DRM, which has basically languished for 2 decades.

It would help if WWWT and other stations wanting to implement HD5 would market it and differentiate it from their regular HD radio content. Maybe a more content-rich broadcast to entice users such as offering higher bit rates, commercial free content, or other data such as song lyrics.

I recently switched to a new ARM-based Mac. After rebuilding the latest I noticed that HD5 no longer works - even against the IQ file. Did not use the -DUSE_NEON switch, as it causes faad2 to fail. However, other than the architecture change there doesn't appear to be any code changes that would cause this... puzzling.

Just making a note of it here and this is not really critical. It appears there still are hardly any stations using HD5-8.

@markjfine Are you sure you're running the add-mp11 branch?

Lol. I am not. I pulled the main branch. 🤦🏻‍♂️

That would explain it. 😄

Assuming it's still working for WWWT, I should probably open a pull request and get that code merged.

I've opened #297. @markjfine I'd appreciate if you could take that for a spin and make sure that it still decodes WWWT correctly, as well as your other local stations.

@argilo Works like a champ on the IQ file as well as all live stations...
Pulling the right branch (e.g., git pull origin add-mp11 --tags) always helps!

The programming guide for Si468x receiver chips sheds some light on why no receivers want to decode MP11's P4 logical channel:

Property 0x9A00. HD_SERVICE_MODE_CONTROL_MP11_ENABLE

This property Enables MP11 mode support. If MP11 support is disabled using this property the receiver will fall back to MP3 mode of operation when tuned to a station that is transmitting the MP11 subcarriers.

Default: 0x0000

The P4 logical channel is not decoded unless the device opts into it by changing this property. No other service mode gets special treatment like this.

It's still a mystery why there was a need to disable the P4 logical channel. Perhaps some buggy transmitters were causing enough havoc that a workaround became necessary.

Some further progress on MP11:

I extracted the firmware from my Sangean HDT-20 (which uses a Si468x chip to decode HD Radio), disassembled it, found the code that configures the Si468x chip, patched it to set the HD_SERVICE_MODE_CONTROL_MP11_ENABLE property to 0x0001 (enabled), and installed the patched firmware.

After this change, the HDT-20 happily decodes the P4 logical channel from an MP11 test signal generated by gr-nrsc5. This suggests to me that gr-nrsc5's MP11 implementation is correct.

However, the HDT-20 still does not decode the P4 logical channel from WWWT. This suggests to me that its non-standard P4 interleaving is a bug. (Perhaps the same bug that prompted the makers of the Si468x chip to turn off P4 by default?)

I'm not sure what nrsc5 should do about this. Maybe it should attempt to decode P4 both ways, and pick the one that works?

@markjfine Could you send me another recording of WWWT? Unfortunately one of my backups failed and I lost some recordings including WWWT.

One minute or so should be enough. Thanks in advance!

No problem. Should be up on my site: https://fineware-swl.com/downloads/wwwt_240824223000.iq (size=175,636,480 bytes).

Got it, thanks!

Interestingly, a friend was able to receive WWWT's HD5 with a Sony XDR-F1HD. So it seems there is hardware in the wild that has the nonstandard/broken MP11 interleaving. I might at some point take a crack at having nrsc5 auto-detect the interleaving so that both standard & nonstandard MP11 signals can be received.

That is very interesting. According to this review by a Northern Virginia FM DXer, the XDR-F1HD was released in 2008. Assuming there were no firmware updates that predates my wife's car radio, which does not see HD5. #weird

2008 is also the year that MP11 was added to the NRSC-5 standard (and MP4 was removed). It could be receivers & transmitters started out with (matching) buggy MP11 implementations, and then the bugs were eventually fixed, but the prevalence of buggy hardware led to MP11 being disabled in many receivers.

	case '0':
	case '1':
	case '2':
	case '3':
	change_program(st, ch - '0');
	break;