2024.06: Toolchain verification summary
Closed this issue · 11 comments
When comparing the results from a Jenkins baseline to those from the 2024.06 release, several discrepancies were identified.
For instance, the configuration "arc-multilib-elf32_hs_nsim" displays new failures that are common across many other configurations. Thus, resolving these issues could potentially address a broad range of other configuration problems.
+--------------------------------+-------------------------------------+----------------------------------------------------------+----------------------------------------------------------+
| | Delta | baseline | 2024.06 |
+--------------------------------+---------+---------+--------+--------+--------+------+-------+-------+------------+-------------+--------+------+-------+-------+------------+-------------+
| | D(PASS) | D(FAIL) | D(NEW) | D(REM) | PASS | FAIL | XFAIL | XPASS | UNRESOLVED | UNSUPPORTED | PASS | FAIL | XFAIL | XPASS | UNRESOLVED | UNSUPPORTED |
+--------------------------------+---------+---------+--------+--------+--------+------+-------+-------+------------+-------------+--------+------+-------+-------+------------+-------------+
| gcc-arc-multilib-elf32_hs_nsim | 14741 | 40 | 15889 | 7212 | 125824 | 31 | 829 | 2 | 19 | 3581 | 134434 | 62 | 867 | 0 | 19 | 4425 |
+--------------------------------+---------+---------+--------+--------+--------+------+-------+-------+------------+-------------+--------+------+-------+-------+------------+-------------+
=== gcc-arc-multilib-elf32_hs_nsim ===
New fail
(PASS) => (FAIL) : gcc.dg/tree-ssa/update-threading.c scan-tree-dump-times optimized "Invalid sum" 0
(PASS) => (FAIL) : gcc.target/arc/pr9001184797.c (test for excess errors)
(PASS) => (FAIL) : outputs-22 exe savetmp namedb-2: outputs.ld1_args
(PASS) => (FAIL) : outputs-23 exe savetmp named2-2: outputs.ld1_args
(PASS) => (FAIL) : outputs-24 exe savetmp named2-3: outputs.ld1_args
(PASS) => (FAIL) : outputs-25 exe savetmp named2-4: outputs.ld1_args
(PASS) => (FAIL) : outputs-294 lto sing unnamed-3: a.ld1_args
(PASS) => (FAIL) : outputs-294 lto sing unnamed-3: a.ld_args
((null)) => (FAIL) : c-c++-common/analyzer/out-of-bounds-diagram-11.c (test for excess errors)
((null)) => (FAIL) : c-c++-common/analyzer/out-of-bounds-diagram-11.c 2 blank line(s) in output
((null)) => (FAIL) : c-c++-common/analyzer/out-of-bounds-diagram-11.c expected multiline pattern lines 49-64
((null)) => (FAIL) : c-c++-common/analyzer/out-of-bounds-diagram-8.c (test for excess errors)
((null)) => (FAIL) : c-c++-common/analyzer/out-of-bounds-diagram-8.c 2 blank line(s) in output
((null)) => (FAIL) : c-c++-common/analyzer/out-of-bounds-diagram-8.c expected multiline pattern lines 19-34
((null)) => (FAIL) : gcc.dg/analyzer/out-of-bounds-diagram-10.c (test for excess errors)
((null)) => (FAIL) : gcc.dg/analyzer/out-of-bounds-diagram-10.c 2 blank line(s) in output
((null)) => (FAIL) : gcc.dg/analyzer/out-of-bounds-diagram-10.c expected multiline pattern lines 13-28
((null)) => (FAIL) : gcc.dg/pr110279-1.c scan-tree-dump-times widening_mul "Generated FMA" 3
((null)) => (FAIL) : gcc.dg/torture/pr113026-1.c -O3 -fomit-frame-pointer -funroll-loops -fpeel-loops -ftracer -finline-functions (test for bogus messages, line 10)
((null)) => (FAIL) : gcc.dg/tree-ssa/vrp-loop-1.c scan-tree-dump-not optimized "foo "
((null)) => (FAIL) : gcc.target/arc/ashrsi-2.c (test for excess errors)
((null)) => (FAIL) : gcc.target/arc/ashrsi-3.c (test for excess errors)
((null)) => (FAIL) : gcc.target/arc/ashrsi-4.c (test for excess errors)
((null)) => (FAIL) : gcc.target/arc/ashrsi-5.c (test for excess errors)
((null)) => (FAIL) : gcc.target/arc/extvsi-1.c (test for excess errors)
((null)) => (FAIL) : gcc.target/arc/extvsi-2.c (test for excess errors)
((null)) => (FAIL) : gcc.target/arc/lshrsi-2.c (test for excess errors)
((null)) => (FAIL) : gcc.target/arc/lshrsi-3.c (test for excess errors)
((null)) => (FAIL) : gcc.target/arc/lshrsi-4.c (test for excess errors)
((null)) => (FAIL) : gcc.target/arc/lshrsi-5.c (test for excess errors)
((null)) => (FAIL) : gcc.target/arc/lsl16-1.c (test for excess errors)
((null)) => (FAIL) : gcc.target/arc/lsr16-1.c (test for excess errors)
((null)) => (FAIL) : gcc.target/arc/pr101955.c (test for excess errors)
((null)) => (FAIL) : gcc.target/arc/scc-ltu.c (test for excess errors)
((null)) => (FAIL) : gcc.target/arc/shlsi-2.c (test for excess errors)
((null)) => (FAIL) : gcc.target/arc/shlsi-3.c (test for excess errors)
((null)) => (FAIL) : gcc.target/arc/shlsi-4.c (test for excess errors)
((null)) => (FAIL) : gcc.target/arc/shlsi-5.c (test for excess errors)
((null)) => (FAIL) : gcc.target/arc/swap-1.c (test for excess errors)
((null)) => (FAIL) : gcc.target/arc/swap-2.c (test for excess errors)- (PASS) => (FAIL) : Indicates a test case that previously passed but now fails.
- ((null)) => (FAIL): Indicates a new test case that fails.
This is just the tip of the iceberg; additional test failures are observed in other configurations:
- arc{eb}-multilib-elf32_arc600_nsim
- arc{eb}-multilib-elf32_arc700_nsim
- arc{eb}-multilib-elf32_arcem_nsim
- arc{eb}-multilib-elf32_archs_nsim
- arc{eb}-multilib-elf32_em4_fpuda_nsim
- arc{eb}-multilib-elf32_em_nsim
- arc{eb}-multilib-elf32_hs34_nsim
- arc{eb}-multilib-elf32_hs38_linux_nsim
- arc{eb}-multilib-elf32_hs4xd_nsim
- arc{eb}-multilib-elf32_hs_nsim
Several tests are currently failing due to a specific warning:
cc1: warning: multiple '-mcpu=' options specifiedThis issue arises because Jenkins executions default to including the "-mcpu=" option. Some tests require a particular "-mcpu" setting, which dejagnu passes to GCC. Consequently, both configurations overlap, causing errors.
For instance, the logs illustrate this conflict:
spawn -ignore SIGHUP /SCRATCH/arcjenkins2/slaves/us01-odc-custom-arcoss1/workspace/arcoss_verification/arc_gnu_toolchain_verification/baremetal_tests-22/toolchains/arc-multilib-elf32/bin/arc-snps-elf-gcc /SCRATCH/arcjenkins2/slaves/us01-odc-custom-arcoss1/workspace/arcoss_verification/arc_gnu_toolchain_verification/baremetal_tests-22/sources/src/gcc/gcc/testsuite/gcc.target/arc/ashrsi-2.c -mcpu=hs -fdiagnostics-plain-output -O2 -mcpu=em -ffat-lto-objects -fno-ident -S --specs=nsim.specs -o ashrsi-2.s
cc1: warning: multiple '-mcpu=' options specified
FAIL: gcc.target/arc/ashrsi-2.c (test for excess errors)
Excess errors:
cc1: warning: multiple '-mcpu=' options specified
The "-mcpu=" option is initially provided by the testing configuration, while dejagnu adds another based on the test case's requirements.
This conflict must be addressed. @yaroslavsadin
@luismgsilva Verifications scripts correctly set -mcpu=archs, the other mcpu option comes from dg directives in the source files:
/* { dg-options "-O2 -mcpu=em" } */List of failing cases:
gcc/gcc/testsuite/gcc.target/arc/ashrsi-2.c
gcc/gcc/testsuite/gcc.target/arc/ashrsi-3.c
gcc/gcc/testsuite/gcc.target/arc/ashrsi-4.c
gcc/gcc/testsuite/gcc.target/arc/ashrsi-5.c
gcc/gcc/testsuite/gcc.target/arc/extvsi-1.c
gcc/gcc/testsuite/gcc.target/arc/extvsi-2.c
gcc/gcc/testsuite/gcc.target/arc/lshrsi-2.c
gcc/gcc/testsuite/gcc.target/arc/lshrsi-3.c
gcc/gcc/testsuite/gcc.target/arc/lshrsi-4.c
gcc/gcc/testsuite/gcc.target/arc/lshrsi-5.c
gcc/gcc/testsuite/gcc.target/arc/lsl16-1.c
gcc/gcc/testsuite/gcc.target/arc/lsr16-1.c
gcc/gcc/testsuite/gcc.target/arc/pr101955.c
gcc/gcc/testsuite/gcc.target/arc/scc-ltu.c
gcc/gcc/testsuite/gcc.target/arc/shlsi-2.c
gcc/gcc/testsuite/gcc.target/arc/shlsi-3.c
gcc/gcc/testsuite/gcc.target/arc/shlsi-4.c
gcc/gcc/testsuite/gcc.target/arc/shlsi-5.c
gcc/gcc/testsuite/gcc.target/arc/swap-1.c
gcc/gcc/testsuite/gcc.target/arc/swap-2.c
Related commits:
foss-for-synopsys-dwc-arc-processors/gcc@3ceb109
foss-for-synopsys-dwc-arc-processors/gcc@ff8d0ce
foss-for-synopsys-dwc-arc-processors/gcc@a3da9ad
foss-for-synopsys-dwc-arc-processors/gcc@d24c3c5
foss-for-synopsys-dwc-arc-processors/gcc@0988121
These are recent changes so they didn't affect the reference build. Please check with commits' authors. I guess they forgot to specify unsupported targets with dg-skip-if.
We have another issue due to using nSIM GNU IO Hostlink (nsim.specs) that can't handle test exit code:
$ ./arc-multilib-elf32/bin/arc-elf32-gcc -mcpu=archs -w -c --specs=nsim.sp
ecs -o gcc-testglue.o /usr/local/share/dejagnu/testglue.c
$ ./arc-multilib-elf32/bin/arc-snps-elf-gcc sources/src/gcc/gcc/testsuite/gcc.dg/tree-prof/pr79587.c -mcpu=archs -dumpbase -fdiagnostics-plain-output -O2 -flto -fprofile-generate -D_PROFILE_GENERATE -dumpbase-ext .x01 --specs=nsim.specs -Wl,--defsym=__DEFAULT_HEAP_SIZE=256m -Wl,--defsym=__DEFAULT_STACK_SIZE=1024m -Wl,-wrap,exit -Wl,-wrap,_exit -Wl,-wrap,main -Wl,-wrap,abort -Wl,gcc-testglue.o -lm -o pr79587.x01
$ ${NSIM_HOME}/bin/nsimdrv -on nsim_isa_enable_timer_0 -on nsim_isa_enable_timer_1 -off invalid_instruction_interrupt -off memory_exception_interrupt -on nsim_download_elf_sections -prop=nsim_emt=1 -p nsim_isa_family=av2hs -p nsim_isa_core=3 -on nsim_isa_sat -p nsim_isa_atomic_option=1 -p nsim_isa_code_density_option=2 -p nsim_isa_div_rem_option=2 -p nsim_isa_ll64_option=1 -p nsim_isa_mpy_option=2 pr79587.x01
*** EXIT code 4242This will be fixed internally by using MetaWare Hostlink (hl.specs). No need to change exp files, can be done by exporting ARC_NSIM_HOSTLINK=metaware.
Possible misconfiguration for hs38_linux on the "c23-stdarg-9.c" test case. [0]
08:08:08.741902 ERROR:[CPU0] Encountered 'InstructionError' exception while this exception is disabled in nSIM - halting. PC:0x00000574 EFA:0x00000574 ECR:0x00020000
[0x00000574] 0x2c72104d AD K DZ dmpywh r13,r12,0x1
EXCEPTION RAISED:
ECR <= 0x00020000 - VECTOR:InstructionError
ERET <= 0x00000574
ERSTATUS <= 0x00080841
BTA <= 0x000014b8
EFA <= 0x00000574
PC <= 0x00000000
STATUS32 <= 0x00081021We have another issue due to using nSIM GNU IO Hostlink (nsim.specs) that can't handle test exit code:
$ ./arc-multilib-elf32/bin/arc-elf32-gcc -mcpu=archs -w -c --specs=nsim.sp ecs -o gcc-testglue.o /usr/local/share/dejagnu/testglue.c $ ./arc-multilib-elf32/bin/arc-snps-elf-gcc sources/src/gcc/gcc/testsuite/gcc.dg/tree-prof/pr79587.c -mcpu=archs -dumpbase -fdiagnostics-plain-output -O2 -flto -fprofile-generate -D_PROFILE_GENERATE -dumpbase-ext .x01 --specs=nsim.specs -Wl,--defsym=__DEFAULT_HEAP_SIZE=256m -Wl,--defsym=__DEFAULT_STACK_SIZE=1024m -Wl,-wrap,exit -Wl,-wrap,_exit -Wl,-wrap,main -Wl,-wrap,abort -Wl,gcc-testglue.o -lm -o pr79587.x01 $ ${NSIM_HOME}/bin/nsimdrv -on nsim_isa_enable_timer_0 -on nsim_isa_enable_timer_1 -off invalid_instruction_interrupt -off memory_exception_interrupt -on nsim_download_elf_sections -prop=nsim_emt=1 -p nsim_isa_family=av2hs -p nsim_isa_core=3 -on nsim_isa_sat -p nsim_isa_atomic_option=1 -p nsim_isa_code_density_option=2 -p nsim_isa_div_rem_option=2 -p nsim_isa_ll64_option=1 -p nsim_isa_mpy_option=2 pr79587.x01 *** EXIT code 4242This will be fixed internally by using MetaWare Hostlink (hl.specs). No need to change exp files, can be done by exporting
ARC_NSIM_HOSTLINK=metaware.
That fixes all 59 errors.
Possible misconfiguration for hs38_linux on the "c23-stdarg-9.c" test case. [0]
08:08:08.741902 ERROR:[CPU0] Encountered 'InstructionError' exception while this exception is disabled in nSIM - halting. PC:0x00000574 EFA:0x00000574 ECR:0x00020000 [0x00000574] 0x2c72104d AD K DZ dmpywh r13,r12,0x1 EXCEPTION RAISED: ECR <= 0x00020000 - VECTOR:InstructionError ERET <= 0x00000574 ERSTATUS <= 0x00080841 BTA <= 0x000014b8 EFA <= 0x00000574 PC <= 0x00000000 STATUS32 <= 0x00081021
Not a misconfiguration, dmpywh is enabled by nSIM property -p nsim_isa_mpy_option=9. Might be a compiler bug. Look at the last instructions in nSIM trace:
[0x000004d8] 0x0a0600a0 AD K Z bl.d 0x1204 : (w0) r31 <= 0x000004e0 *
[0x000004dc] 0x2c72104d AD K DZ dmpywh r13,r12,0x1
I have a strong feeling this has something to do with the dmpywh used in the delay slot.
Possible misconfiguration for hs38_linux on the "c23-stdarg-9.c" test case. [0]
08:08:08.741902 ERROR:[CPU0] Encountered 'InstructionError' exception while this exception is disabled in nSIM - halting. PC:0x00000574 EFA:0x00000574 ECR:0x00020000 [0x00000574] 0x2c72104d AD K DZ dmpywh r13,r12,0x1 EXCEPTION RAISED: ECR <= 0x00020000 - VECTOR:InstructionError ERET <= 0x00000574 ERSTATUS <= 0x00080841 BTA <= 0x000014b8 EFA <= 0x00000574 PC <= 0x00000000 STATUS32 <= 0x00081021Not a misconfiguration,
dmpywhis enabled by nSIM property-p nsim_isa_mpy_option=9. Might be a compiler bug. Look at the last instructions in nSIM trace:[0x000004d8] 0x0a0600a0 AD K Z bl.d 0x1204 : (w0) r31 <= 0x000004e0 * [0x000004dc] 0x2c72104d AD K DZ dmpywh r13,r12,0x1I have a strong feeling this has something to do with the
dmpywhused in the delay slot.
ARCv2 PRM states that for all operations that produce a 64-bit result, the A operand must be an even-numbered register.
The compiler uses r13 as A, so that must be the problem here. Reproduce as follows:
$ ./arc-multilib-elf32/bin/arc-snps-elf-gcc ./sources/src/gcc/gcc/testsuite/gcc.dg/c23-stdarg-9.c -mcpu=hs38_linux -dumpbase -fdiagnostics-plain-output -O2 -std=c23 -pedantic-errors --specs=nsim.specs -Wl,--defsym=__DEFAULT_HEAP_SIZE=256m -Wl,--defsym=__DEFAULT_STACK_SIZE=1024m -Wl,-wrap,exit -Wl,-wrap,_exit -Wl,-wrap,main -Wl,-wrap,abort -Wl,gcc-testglue.o -fno-delayed-branch -lm -o ./c23-stdarg-9.exe
$ ${NSIM_HOME}/bin/nsimdrv -on trace -on nsim_isa_enable_timer_0 -on nsim_isa_enable_timer_1 -off invalid_instruction_interrupt -off memory_exception_interrupt -on nsim_download_elf_sections -prop=nsim_emt=1 -p nsim_isa_family=av2hs -p nsim_isa_core=3 -on nsim_isa_sat -p nsim_isa_atomic_option=1 -p nsim_isa_code_density_option=2 -p nsim_isa_div_rem_option=2 -p nsim_isa_fpud_option=1 -p nsim_isa_fpud_div_option=1 -p nsim_isa_fpu_mac_option=1 -p nsim_isa_fpus_option=1 -p nsim_isa_fpus_div_option=1 -p nsim_isa_fpu_mac_option=1 -p nsim_isa_ll64_option=1 -p nsim_isa_mpy_option=9 ./c23-stdarg-9.exe
@luismgsilva Verifications scripts correctly set
-mcpu=archs, the other mcpu option comes from dg directives in the source files:/* { dg-options "-O2 -mcpu=em" } */List of failing cases:
gcc/gcc/testsuite/gcc.target/arc/ashrsi-2.c gcc/gcc/testsuite/gcc.target/arc/ashrsi-3.c gcc/gcc/testsuite/gcc.target/arc/ashrsi-4.c gcc/gcc/testsuite/gcc.target/arc/ashrsi-5.c gcc/gcc/testsuite/gcc.target/arc/extvsi-1.c gcc/gcc/testsuite/gcc.target/arc/extvsi-2.c gcc/gcc/testsuite/gcc.target/arc/lshrsi-2.c gcc/gcc/testsuite/gcc.target/arc/lshrsi-3.c gcc/gcc/testsuite/gcc.target/arc/lshrsi-4.c gcc/gcc/testsuite/gcc.target/arc/lshrsi-5.c gcc/gcc/testsuite/gcc.target/arc/lsl16-1.c gcc/gcc/testsuite/gcc.target/arc/lsr16-1.c gcc/gcc/testsuite/gcc.target/arc/pr101955.c gcc/gcc/testsuite/gcc.target/arc/scc-ltu.c gcc/gcc/testsuite/gcc.target/arc/shlsi-2.c gcc/gcc/testsuite/gcc.target/arc/shlsi-3.c gcc/gcc/testsuite/gcc.target/arc/shlsi-4.c gcc/gcc/testsuite/gcc.target/arc/shlsi-5.c gcc/gcc/testsuite/gcc.target/arc/swap-1.c gcc/gcc/testsuite/gcc.target/arc/swap-2.cRelated commits: foss-for-synopsys-dwc-arc-processors/gcc@3ceb109 foss-for-synopsys-dwc-arc-processors/gcc@ff8d0ce foss-for-synopsys-dwc-arc-processors/gcc@a3da9ad foss-for-synopsys-dwc-arc-processors/gcc@d24c3c5 foss-for-synopsys-dwc-arc-processors/gcc@0988121
These are recent changes so they didn't affect the reference build. Please check with commits' authors. I guess they forgot to specify unsupported targets with
dg-skip-if.
@luismgsilva So here's the right way to write those test cases: thumb2-slow-flash-data-5.c. We also have some ARC-specific procedures checking for the mcpu: arc.exp (seems incomplete though). It's responsibility of each nongeneric test case to check if it's supported in the given run, and mark itself skipped otherwise, external system can't do that.
Possible misconfiguration for hs38_linux on the "c23-stdarg-9.c" test case. [0]
08:08:08.741902 ERROR:[CPU0] Encountered 'InstructionError' exception while this exception is disabled in nSIM - halting. PC:0x00000574 EFA:0x00000574 ECR:0x00020000 [0x00000574] 0x2c72104d AD K DZ dmpywh r13,r12,0x1 EXCEPTION RAISED: ECR <= 0x00020000 - VECTOR:InstructionError ERET <= 0x00000574 ERSTATUS <= 0x00080841 BTA <= 0x000014b8 EFA <= 0x00000574 PC <= 0x00000000 STATUS32 <= 0x00081021Not a misconfiguration,
dmpywhis enabled by nSIM property-p nsim_isa_mpy_option=9. Might be a compiler bug. Look at the last instructions in nSIM trace:[0x000004d8] 0x0a0600a0 AD K Z bl.d 0x1204 : (w0) r31 <= 0x000004e0 * [0x000004dc] 0x2c72104d AD K DZ dmpywh r13,r12,0x1I have a strong feeling this has something to do with the
dmpywhused in the delay slot.ARCv2 PRM states that for all operations that produce a 64-bit result, the A operand must be an even-numbered register.
The compiler uses r13 as A, so that must be the problem here. Reproduce as follows:
$ ./arc-multilib-elf32/bin/arc-snps-elf-gcc ./sources/src/gcc/gcc/testsuite/gcc.dg/c23-stdarg-9.c -mcpu=hs38_linux -dumpbase -fdiagnostics-plain-output -O2 -std=c23 -pedantic-errors --specs=nsim.specs -Wl,--defsym=__DEFAULT_HEAP_SIZE=256m -Wl,--defsym=__DEFAULT_STACK_SIZE=1024m -Wl,-wrap,exit -Wl,-wrap,_exit -Wl,-wrap,main -Wl,-wrap,abort -Wl,gcc-testglue.o -fno-delayed-branch -lm -o ./c23-stdarg-9.exe $ ${NSIM_HOME}/bin/nsimdrv -on trace -on nsim_isa_enable_timer_0 -on nsim_isa_enable_timer_1 -off invalid_instruction_interrupt -off memory_exception_interrupt -on nsim_download_elf_sections -prop=nsim_emt=1 -p nsim_isa_family=av2hs -p nsim_isa_core=3 -on nsim_isa_sat -p nsim_isa_atomic_option=1 -p nsim_isa_code_density_option=2 -p nsim_isa_div_rem_option=2 -p nsim_isa_fpud_option=1 -p nsim_isa_fpud_div_option=1 -p nsim_isa_fpu_mac_option=1 -p nsim_isa_fpus_option=1 -p nsim_isa_fpus_div_option=1 -p nsim_isa_fpu_mac_option=1 -p nsim_isa_ll64_option=1 -p nsim_isa_mpy_option=9 ./c23-stdarg-9.exe
Reduced test case:
$ cat small.c
#include <stdarg.h>
struct b {
int a[1024];
} c() {
int d;
va_list ap;
d = va_arg(ap, int);
d += va_arg(ap, int);
d += va_arg(ap, int);
d += va_arg(ap, int);
struct b e = {};
e.a[0] = d;
return e;
}
$ CC=${BLD}/gcc/cc1
$ $CC -quiet \
-O2 \
-std=c23 \
-pedantic-errors \
small.c \
-o small.s
$ cat smalls.s
...
dmpywh r13,r12,1
...
@luismgsilva please make necessary changes here: we need to close it for 2024.06 with remaining work filed as a separate issue to be addressed in the next release.