theori-io/nrsc5

Fedroa 36 constant Lost synchronization

andrewfer000 opened this issue · 21 comments

andrewfer000 commented

Hello,

I run Fedora 36 and noticed that stations don't play anymore and I get this output consistently. I have very strong signal and no issues on Ubuntu 20.04. Is this a bug or is there a fix? I followed the build instructions to the letter.

20:07:45 MER: 15.7 dB (lower), 15.7 dB (upper)
20:07:45 BER: 0.501401, avg: 0.500109, min: 0.498725, max: 0.501401
20:07:45 Lost synchronization
20:07:45 Synchronized

Thank You.

pclov3r commented

Hum, the MER is very good but the BER is very poor at 0.50. Does the BER always remain that high?

A longer output log could help as well.

You say you have no issues with Ubuntu but have issues with Fedora? Are you able to provide more details if your running this in a VM or native environment?

With that said given the high BER it appear something is happening with the signal and/or samples.

andrewfer000 commented

Hi there,

Thanks for the quick response! I am running on a Native System. I will get more info to you shortly.

andrewfer000 commented

Here is NRSC-5 running

Detached kernel driver
Found Rafael Micro R820T tuner
Exact sample rate is: 1488375.071248 Hz
[R82XX] PLL not locked!
Gain: 0.0 dB, CNR: 7.6 dB
Gain: 0.9 dB, CNR: 9.6 dB
Gain: 1.4 dB, CNR: 10.7 dB
Gain: 2.7 dB, CNR: 14.2 dB
Gain: 3.7 dB, CNR: 15.7 dB
Gain: 7.7 dB, CNR: 18.7 dB
Gain: 8.7 dB, CNR: 19.3 dB
Gain: 12.5 dB, CNR: 21.2 dB
Gain: 14.4 dB, CNR: 19.6 dB
Gain: 15.7 dB, CNR: 16.0 dB
Gain: 16.6 dB, CNR: 15.2 dB
Gain: 19.7 dB, CNR: 13.6 dB
Gain: 20.7 dB, CNR: 12.2 dB
Gain: 22.9 dB, CNR: 10.8 dB
Gain: 25.4 dB, CNR: 8.8 dB
Gain: 28.0 dB, CNR: 6.0 dB
Gain: 29.7 dB, CNR: 6.7 dB
Gain: 32.8 dB, CNR: 8.9 dB
Gain: 33.8 dB, CNR: 8.3 dB
Gain: 36.4 dB, CNR: 6.1 dB
Gain: 37.2 dB, CNR: 7.2 dB
Gain: 38.6 dB, CNR: 5.7 dB
Gain: 40.2 dB, CNR: 6.8 dB
Gain: 42.1 dB, CNR: 6.8 dB
Gain: 43.4 dB, CNR: 5.5 dB
Gain: 43.9 dB, CNR: 5.5 dB
Gain: 44.5 dB, CNR: 6.0 dB
Gain: 48.0 dB, CNR: 5.7 dB
Gain: 49.6 dB, CNR: 5.0 dB
Best gain: 12.5 dB, CNR: 21.2 dB
Allocating 8 zero-copy buffers
CFO: -363.372803 Hz
Block @ 13
01:51:26 Synchronized
01:51:28 MER: 10.2 dB (lower), 8.8 dB (upper)
01:51:28 BER: 0.499455, avg: 0.499455, min: 0.499455, max: 0.499455
01:51:28 Lost synchronization
Block @ 0
Block @ 0
01:51:29 Synchronized
01:51:31 MER: 8.9 dB (lower), 7.5 dB (upper)
01:51:31 BER: 0.499617, avg: 0.499536, min: 0.499455, max: 0.499617
01:51:31 Lost synchronization
Block @ 0
01:51:32 Synchronized
20:51:34 MER: 9.1 dB (lower), 7.5 dB (upper)
20:51:34 BER: 0.500175, avg: 0.499749, min: 0.499455, max: 0.500175
20:51:34 Lost synchronization
Block @ 0
Block @ 0
20:51:35 Synchronized
20:51:37 MER: 9.1 dB (lower), 7.7 dB (upper)
20:51:37 BER: 0.499770, avg: 0.499754, min: 0.499455, max: 0.500175
20:51:37 Lost synchronization
Block @ 0
Block @ 0
20:51:38 Synchronized
20:51:40 MER: 9.3 dB (lower), 7.8 dB (upper)
20:51:40 BER: 0.499360, avg: 0.499675, min: 0.499360, max: 0.500175
20:51:40 Lost synchronization
Block @ 0
20:51:41 Synchronized
20:51:43 MER: 9.6 dB (lower), 7.9 dB (upper)
20:51:43 BER: 0.498738, avg: 0.499519, min: 0.498738, max: 0.500175
20:51:43 Lost synchronization
Block @ 0
Block @ 0
20:51:44 Synchronized
20:51:46 MER: 9.5 dB (lower), 7.8 dB (upper)
20:51:46 BER: 0.500323, avg: 0.499634, min: 0.498738, max: 0.500323
20:51:46 Lost synchronization
Block @ 0
pclov3r commented

Hum...

The same station works without issues on Ubuntu 20.04 but fails with Fedora 36?

andrewfer000 commented

Ubuntu 20.04

Detached kernel driver
Found Rafael Micro R820T tuner
Exact sample rate is: 1488375.071248 Hz
[R82XX] PLL not locked!
Disabled direct sampling mode
[R82XX] PLL not locked!
21:01:33 Synchronized
21:01:33 Country: US, FCC facility ID: 10059
21:01:33 Station name: WSHE-FM
21:01:33 Country: US, FCC facility ID: 10059
21:01:33 Station name: WSHE-FM
21:01:33 Slogan:  
21:01:33 Country: US, FCC facility ID: 10059
21:01:33 Station name: WSHE-FM
21:01:33 Slogan:  
21:01:33 Data service: public, type: Emergency, MIME type 444
21:01:33 Country: US, FCC facility ID: 10059
21:01:33 Station name: WSHE-FM
21:01:33 Slogan:  
21:01:33 Data service: public, type: Emergency, MIME type 444
21:01:33 Country: US, FCC facility ID: 10059
21:01:33 Station name: WSHE-FM
21:01:33 Slogan:  
21:01:33 Audio program 1: public, type: None, sound experience 0
21:01:33 Data service: public, type: Emergency, MIME type 444
21:01:33 Country: US, FCC facility ID: 10059
21:01:33 Station name: WSHE-FM
21:01:33 Slogan:  
21:01:33 Message:  
21:01:33 Audio program 1: public, type: None, sound experience 0
21:01:33 Data service: public, type: Emergency, MIME type 444
21:01:33 Country: US, FCC facility ID: 10059
21:01:33 Station name: WSHE-FM
21:01:33 Slogan:  
21:01:33 Message:  
21:01:33 Station location: 39.185303, -76.806274, 96m
21:01:33 Audio program 1: public, type: None, sound experience 0
21:01:33 Data service: public, type: Emergency, MIME type 444
21:01:33 Country: US, FCC facility ID: 10059
21:01:33 Station name: WSHE-FM
21:01:33 Slogan:  
21:01:33 Message:  
21:01:33 Station location: 39.185303, -76.806274, 96m
21:01:33 Audio program 0: public, type: None, sound experience 0
21:01:33 Audio program 1: public, type: None, sound experience 0
21:01:33 Data service: public, type: Emergency, MIME type 444
21:01:34 MER: 11.8 dB (lower), 11.4 dB (upper)
21:01:34 BER: 0.000142, avg: 0.000142, min: 0.000142, max: 0.000142
21:01:36 MER: 12.6 dB (lower), 12.3 dB (upper)
21:01:36 BER: 0.000047, avg: 0.000094, min: 0.000047, max: 0.000142
21:01:36 Title: Raise Your Glass
21:01:36 Artist: PINK
21:01:36 Album:  
21:01:36 Genre:  
21:01:36 XHDR: 0 BE4B7536 54054
21:01:36 Audio bit rate: 56.0 kbps
21:01:37 MER: 11.8 dB (lower), 11.2 dB (upper)
21:01:37 BER: 0.000145, avg: 0.000111, min: 0.000047, max: 0.000145
21:01:37 Title: Raise Your Glass
21:01:37 Artist: PINK
21:01:37 Album:  
21:01:37 Genre:  
21:01:37 XHDR: 0 BE4B7536 54054
21:01:37 Audio bit rate: 56.3 kbps
^C
andrewfer000 commented

Ubuntu 20.04 with latest compiled NRSC5

Found Rafael Micro R820T tuner
Exact sample rate is: 1488375.071248 Hz
[R82XX] PLL not locked!
Gain: 0.0 dB, CNR: 8.3 dB
Gain: 0.9 dB, CNR: 10.9 dB
Gain: 1.4 dB, CNR: 12.2 dB
Gain: 2.7 dB, CNR: 14.9 dB
Gain: 3.7 dB, CNR: 16.7 dB
Gain: 7.7 dB, CNR: 19.7 dB
Gain: 8.7 dB, CNR: 20.3 dB
Gain: 12.5 dB, CNR: 21.8 dB
Gain: 14.4 dB, CNR: 17.8 dB
Gain: 15.7 dB, CNR: 15.2 dB
Gain: 16.6 dB, CNR: 12.3 dB
Gain: 19.7 dB, CNR: 9.8 dB
Gain: 20.7 dB, CNR: 9.2 dB
Gain: 22.9 dB, CNR: 8.9 dB
Gain: 25.4 dB, CNR: 8.9 dB
Gain: 28.0 dB, CNR: 8.3 dB
Gain: 29.7 dB, CNR: 9.2 dB
Gain: 32.8 dB, CNR: 6.3 dB
Gain: 33.8 dB, CNR: 5.5 dB
Gain: 36.4 dB, CNR: 4.9 dB
Gain: 37.2 dB, CNR: 6.9 dB
Gain: 38.6 dB, CNR: 4.8 dB
Gain: 40.2 dB, CNR: 6.7 dB
Gain: 42.1 dB, CNR: 6.2 dB
Gain: 43.4 dB, CNR: 6.4 dB
Gain: 43.9 dB, CNR: 6.0 dB
Gain: 44.5 dB, CNR: 4.9 dB
Gain: 48.0 dB, CNR: 4.5 dB
Gain: 49.6 dB, CNR: 4.9 dB
Best gain: 12.5 dB, CNR: 21.8 dB
CFO: -363.372803 Hz
Block @ 6
21:07:22 Synchronized
21:07:22 Country: US, FCC facility ID: 10059
21:07:22 Station name: WSHE-FM
21:07:22 Country: US, FCC facility ID: 10059
21:07:22 Station name: WSHE-FM
21:07:22 Slogan:  
21:07:22 Country: US, FCC facility ID: 10059
21:07:22 Station name: WSHE-FM
21:07:22 Slogan:  
21:07:22 Data service: public, type: Emergency, MIME type 444
21:07:22 Country: US, FCC facility ID: 10059
21:07:22 Station name: WSHE-FM
21:07:22 Slogan:  
21:07:22 Data service: public, type: Emergency, MIME type 444
21:07:22 Country: US, FCC facility ID: 10059
21:07:22 Station name: WSHE-FM
21:07:22 Slogan:  
21:07:22 Audio program 1: public, type: None, sound experience 0
21:07:22 Data service: public, type: Emergency, MIME type 444
21:07:22 Country: US, FCC facility ID: 10059
21:07:22 Station name: WSHE-FM
21:07:22 Slogan:  
21:07:22 Message:  
21:07:22 Audio program 1: public, type: None, sound experience 0
21:07:22 Data service: public, type: Emergency, MIME type 444
21:07:22 Country: US, FCC facility ID: 10059
21:07:22 Station name: WSHE-FM
21:07:22 Slogan:  
21:07:22 Message:  
21:07:22 Station location: 39.185303, -76.806274, 96m
21:07:22 Audio program 1: public, type: None, sound experience 0
21:07:22 Data service: public, type: Emergency, MIME type 444
21:07:22 Country: US, FCC facility ID: 10059
21:07:22 Station name: WSHE-FM
21:07:22 Slogan:  
21:07:22 Message:  
21:07:22 Station location: 39.185303, -76.806274, 96m
21:07:22 Audio program 0: public, type: None, sound experience 0
21:07:22 Audio program 1: public, type: None, sound experience 0
21:07:22 Data service: public, type: Emergency, MIME type 444
21:07:23 MER: 10.1 dB (lower), 8.7 dB (upper)
21:07:23 BER: 0.003399, avg: 0.003399, min: 0.003399, max: 0.003399
ignoring partial pdu
ignoring partial pdu
ALFN of pending leap second adjustment: 0
21:07:25 MER: 8.7 dB (lower), 7.2 dB (upper)
21:07:25 BER: 0.006425, avg: 0.004912, min: 0.003399, max: 0.006425
21:07:25 Title: Uptown Funk
21:07:25 Artist: Mark Ronson W/ Bruno Mars
21:07:25 Album:  
21:07:25 Genre:  
21:07:25 XHDR: 0 BE4B7536 54055
21:07:25 Audio bit rate: 56.5 kbps
Local time zone offset: -300 minutes, DST sched. 1, local DST? yes, regional DST? yes
21:07:26 MER: 8.8 dB (lower), 7.3 dB (upper)
21:07:26 BER: 0.005040, avg: 0.004955, min: 0.003399, max: 0.006425
21:07:26 Title: Uptown Funk
21:07:26 Artist: Mark Ronson W/ Bruno Mars
21:07:26 Album:  
21:07:26 Genre:  
21:07:26 XHDR: 0 BE4B7536 54055
21:07:26 Title: Uptown Funk
21:07:26 Artist: Mark Ronson W/ Bruno Mars
21:07:26 Album:  
21:07:26 Genre:  
21:07:26 XHDR: 0 BE4B7536 54055
21:07:26 Audio bit rate: 56.2 kbps
Pending leap second offset: 0, current leap second offset: 0
21:07:28 MER: 8.6 dB (lower), 7.2 dB (upper)
21:07:28 BER: 0.005755, avg: 0.005155, min: 0.003399, max: 0.006425
21:07:28 Title: Uptown Funk
21:07:28 Artist: Mark Ronson W/ Bruno Mars
21:07:28 Album:  
21:07:28 Genre:  
21:07:28 XHDR: 0 BE4B7536 54055
21:07:28 Audio bit rate: 57.2 kbps
21:07:29 MER: 9.2 dB (lower), 7.4 dB (upper)
21:07:29 BER: 0.004657, avg: 0.005055, min: 0.003399, max: 0.006425
21:07:29 Title: Uptown Funk
21:07:29 Artist: Mark Ronson W/ Bruno Mars
21:07:29 Album:  
21:07:29 Genre:  
21:07:29 XHDR: 0 BE4B7536 54055
21:07:29 Audio bit rate: 56.3 kbps
^C
pclov3r commented

What happens if you play the included sample https://github.com/theori-io/nrsc5/blob/master/support/sample.xz using Fedora 36?

You will need to decompress it before playing and use the -r iq-input flag.

markjfine commented

I think on Fedora 35 in an x86 architecture I had to use the -DUSE_SSE=ON build option. Without it I was getting the same kind of dropped synch issue.

andrewfer000 commented

I fixed it. The version I compiled on Ubuntu right now works perfect on Fedora 36.
I think I botched the first compile. So I will recompile on Fedora 36 fresh again and see if it works.

pclov3r commented

It's possible there is a bug somewhere in one of the libraries, a GCC bug or something with Fedora 36.

Anything you find might be helpful in trying to figure out what went wrong.

andrewfer000 commented

So after recompiling on Fedora 36 I had the same issue. I just replaced the nrsc5 binary with the Ubuntu 20.04 version and it's perfect. My friend (who does DevOps) says it may be the cmake.

pclov3r commented

@markjfine mentioned about using the SSE build option to resolve their issues on Fedora.

You might want to try to rebuild using the -DUSE_SSE=ON build option and see if that resolves the issue.

andrewfer000 commented

I always use the SSE Option. But I found this when I run ldd against the binaries it's different.

Fedora 36:

	linux-vdso.so.1 (0x00007ffcaff6c000)
	libao.so.4 => /lib64/libao.so.4 (0x00007faa3d27d000)
	libfftw3f.so.3 => /lib64/libfftw3f.so.3 (0x00007faa3d07d000)
	librtlsdr.so.0 => /lib64/librtlsdr.so.0 (0x00007faa3d06b000)
	libm.so.6 => /lib64/libm.so.6 (0x00007faa3cf8d000)
	libc.so.6 => /lib64/libc.so.6 (0x00007faa3cd8c000)
	libusb-1.0.so.0 => /lib64/libusb-1.0.so.0 (0x00007faa3cd6b000)
	/lib64/ld-linux-x86-64.so.2 (0x00007faa3d2a4000)
	libudev.so.1 => /lib64/libudev.so.1 (0x00007faa3cd3e000)
	libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007faa3cd1e000)

Ubuntu:

	linux-vdso.so.1 (0x00007ffee3dd0000)
	libao.so.4 => /lib64/libao.so.4 (0x00007feca21fe000)
	libpthread.so.0 => /lib64/libpthread.so.0 (0x00007feca21f9000)
	libfftw3f.so.3 => /lib64/libfftw3f.so.3 (0x00007feca1ff9000)
	librtlsdr.so.0 => /lib64/librtlsdr.so.0 (0x00007feca1fe7000)
	libm.so.6 => /lib64/libm.so.6 (0x00007feca1f09000)
	libc.so.6 => /lib64/libc.so.6 (0x00007feca1d08000)
	libusb-1.0.so.0 => /lib64/libusb-1.0.so.0 (0x00007feca1ce7000)
	/lib64/ld-linux-x86-64.so.2 (0x00007feca2283000)
	libudev.so.1 => /lib64/libudev.so.1 (0x00007feca1cba000)
	libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007feca1c9a000)

All libraries exist on both systems in /lib64. For some reason Fedora 36 is not linking libpthread.

pclov3r commented

libpthread is optional so that isn't the cause.

What happens if you play the sample https://github.com/theori-io/nrsc5/blob/master/support/sample.xz on Fedora using the build you made on it? Does it play correctly or show similar behavior.

andrewfer000 commented

Same issue when compiled on F36.

xz -d < ../sample.xz | src/nrsc5 -r - 0
Block @ 3
Block @ 31
Block @ 0
22:52:55 Synchronized
22:52:55 MER: 13.5 dB (lower), 12.5 dB (upper)
22:52:55 BER: 0.499086, avg: 0.499086, min: 0.499086, max: 0.499086
22:52:55 Lost synchronization
Block @ 0
Block @ 0
22:52:56 Synchronized
22:52:56 MER: 13.6 dB (lower), 12.3 dB (upper)
22:52:56 BER: 0.500424, avg: 0.499755, min: 0.499086, max: 0.500424
22:52:56 Lost synchronization
Block @ 0
Block @ 0
22:52:56 Synchronized
22:52:56 MER: 7.3 dB (lower), 6.5 dB (upper)
22:52:56 BER: 0.501790, avg: 0.500433, min: 0.499086, max: 0.501790
22:52:56 Lost synchronization
Block @ 0
22:52:57 Synchronized
22:52:57 MER: 13.2 dB (lower), 12.1 dB (upper)
22:52:57 BER: 0.498591, avg: 0.499973, min: 0.498591, max: 0.501790
22:52:57 Lost synchronization
Block @ 0
Block @ 0
22:52:57 Synchronized
22:52:57 MER: 13.5 dB (lower), 12.3 dB (upper)
22:52:57 BER: 0.501026, avg: 0.500183, min: 0.498591, max: 0.501790
22:52:57 Lost synchronization
Block @ 0

I also built in a Ubuntu 22.04 Docker container and it works fine on the Fedora 36 System. I think it might have something to do with GCC 12 since Ubuntu 22.04 uses GCC 11.2.

pclov3r commented

There might be some compile optimizations that are not playing well if the new version of GCC. It's happened before.

You could attempt to invoke a older version of GCC and see if that works.

@argilo any ideas what might be going on here? I don't have a Fedora 36 system right now for testing.

andrewfer000 commented

Umm it's almost definitely GCC 12. I compiled it on Ubuntu 22.04 and Fedora 35 in docker and ran the binary on my F36 system. Ubuntu and F35 come with GCC 11.2

argilo commented

I was able to reproduce the problem on Ubuntu 22.04 by installing GCC 12 (sudo apt install gcc-12) and asking cmake to use it with -DCMAKE_C_COMPILER=gcc-12. The most likely explanation is that nrsc5 has a bug which happens to be exposed by this particular compiler version. The problem only occurs with -DUSE_SSE=1, which suggests that the bug is in the SSE-optimized convolutional decoder.

argilo commented

GCC 12 produces the following warning, which could well point to the source of the problem:

In file included from /home/argilo/git/nrsc5/src/conv_sse.h:25,
                 from /home/argilo/git/nrsc5/src/conv_dec.c:37:
In function ‘_mm_load1_pd’,
    inlined from ‘_mm_loaddup_pd’ at /usr/lib/gcc/x86_64-linux-gnu/12/include/pmmintrin.h:101:10,
    inlined from ‘_sse_metrics_k7_n4’ at /home/argilo/git/nrsc5/src/conv_sse.h:254:7,
    inlined from ‘gen_metrics_k7_n3’ at /home/argilo/git/nrsc5/src/conv_sse.h:322:2,
    inlined from ‘_conv_decode.constprop’ at /home/argilo/git/nrsc5/src/conv_dec.c:415:4:
/usr/lib/gcc/x86_64-linux-gnu/12/include/emmintrin.h:138:10: warning: ‘_val’ may be used uninitialized [-Wmaybe-uninitialized]
  138 |   return _mm_set1_pd (*__P);
      |          ^~~~~~~~~~~~~~~~~~
/home/argilo/git/nrsc5/src/conv_sse.h: In function ‘_conv_decode.constprop’:
/home/argilo/git/nrsc5/src/conv_sse.h:320:23: note: ‘_val’ declared here
  320 |         const int16_t _val[4] = { val[0], val[1], val[2], 0 };
      |                       ^~~~
argilo commented

The following patch appears to fix the issue:

diff --git a/src/conv_sse.h b/src/conv_sse.h
index 1165c65..98f9619 100644
--- a/src/conv_sse.h
+++ b/src/conv_sse.h
@@ -251,7 +251,7 @@ static inline void _sse_metrics_k7_n4(const int16_t *val, const int16_t *out,
                            m8, m9, m10, m11, m12, m13, m14, m15)
 
        /* (BMU) Load and expand 8-bit input out to 16-bits */
-       m7 = _mm_castpd_si128(_mm_loaddup_pd((double const *) val));
+       m7 = _mm_loadu_si128((__m128i *) val);
 
        /* (BMU) Load and compute branch metrics */
        m0 = _mm_load_si128((__m128i *) &out[0]);
@@ -317,7 +317,7 @@ static inline void _sse_metrics_k7_n4(const int16_t *val, const int16_t *out,
 static void gen_metrics_k7_n3(const int8_t *val, const int16_t *out,
                       int16_t *sums, int16_t *paths, int norm)
 {
-       const int16_t _val[4] = { val[0], val[1], val[2], 0 };
+       const int16_t _val[8] = { val[0], val[1], val[2], 0, val[0], val[1], val[2], 0 };
 
        _sse_metrics_k7_n4(_val, out, sums, paths, norm);
 }
ferrellsl commented