BrianSidebotham/arm-tutorial-rpi

part-1 / armc-03 : _start defaulting to 0x8024 (small stack @ 0x8000)

rwdavis3rd opened this issue · 8 comments

rwdavis3rd commented

Hi Brian,

Thanks for your great tutorials! I wanted to alert you to a problem I just encountered, in case others experience similar...

I was compiling (build.sh) and trying to run part-1 / armc-03 and watching for a blinking LED, with no joy. A clue as to the problem was coming out of the linker:

/arm-none-eabi/bin/ld: warning: cannot find entry symbol _start; defaulting to 0000000000008024.

Using objdump to produce a disassembly listing, I found the linker was putting a small stack (space reservation) at 0x8000 and following that with the main() entry point. If I understand Pi booting correctly, that would have the CPU jumping into the stack instead of to main().

I've gotten relief by adding some linker directives: (file linker.ld)

MEMORY
{
    ram : ORIGIN = 0x8000, LENGTH = 20K
}

SECTIONS
{
    .text : { *(.text*) } > ram
    .note.gnu.build-id : { *(.text*) } > ram    
    .bss : { *(.bss*) } > ram
}

And adding this, via the -T option, to the executable build:

${toolchain}gcc ${cflags} ${scriptdir}/*.c -T linker.ld -o ${kernel_elf}

In short, I had to force the .text section (i.e. the main() program) to be at the top of the img. This (finally) yielded a satisfying blink. :-)

For what it matters: I'm cross-compiling for pi0 from Ubuntu 18.04 using ARM Corp's gnu

    armc-03$ arm-none-eabi-gcc --version
    arm-none-eabi-gcc (GCC) 8.2.0
    Copyright (C) 2018 Free Software Foundation, Inc.
    This is free software; see the source for copying conditions.  There is NO
    warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Cheers!

rwdavis3rd commented

Correction (sorry): the section that claimed the 0x8000 address was .note.gnu.build-id

Disassembly of section .note.gnu.build-id:

00008000 <.note.gnu.build-id>:
    8000:	00000004 	andeq	r0, r0, r4
    8004:	00000014 	andeq	r0, r0, r4, lsl r0
    8008:	00000003 	andeq	r0, r0, r3
    800c:	00554e47 	subseq	r4, r5, r7, asr #28
    8010:	84449b50 	strbhi	r9, [r4], #-2896	; 0xfffff4b0
    8014:	f8943dd3 			; <UNDEFINED> instruction: 0xf8943dd3
    8018:	01710b24 	cmneq	r1, r4, lsr #22
    801c:	757cf378 	ldrbvc	pc, [ip, #-888]!	; 0xfffffc88	; <UNPREDICTABLE>
    8020:	14b38c85 	ldrtne	r8, [r3], #3205	; 0xc85

Disassembly of section .text:

00008024 <main>:
    8024:	e59f30c4 	ldr	r3, [pc, #196]	; 80f0 <main+0xcc>
    8028:	e59f20c4 	ldr	r2, [pc, #196]	; 80f4 <main+0xd0>
rwdavis3rd commented

By the time I got to part-2 / arm-06, where the use of a user-supplied linker script is illustrated, I got relief from the problem above by moving the inclusion of the .note.gnu.build-id waay down to the bottom of the script. So the last few lines of the modified rpi.x file looks like:

  /* DWARF Extension.  */
  .debug_macro    0 : { *(.debug_macro) }
  .debug_addr     0 : { *(.debug_addr) }
  .ARM.attributes 0 : { KEEP (*(.ARM.attributes)) KEEP (*(.gnu.attributes)) }
  .note.gnu.arm.ident 0 : { KEEP (*(.note.gnu.arm.ident)) }
  /DISCARD/ : { *(.note.GNU-stack) *(.gnu_debuglink) *(.gnu.lto_*) }
  .note.gnu.build-id : { *(.note.gnu.build-id) } 
}
BrianSidebotham commented

Hi @rwdavis3rd

Could you provide me some further information? A linker that defaults _start to 0x8024 is very unusual behaviour.

I just tested with the compiler that the tutorial is built on (using the compiler/get_compiler.sh script) and everything looks right for the tutorial:

[bjs@localhost armc-03]$ ./build.sh rpi0
.../compiler/gcc-arm-none-eabi-7-2018-q2-update/bin/arm-none-eabi-gcc -g -nostartfiles -mfloat-abi=hard -O0 -DRPI0 -mfpu=vfp -march=armv6zk -mtune=arm1176jzf-s .../part-1/armc-03/*.c -o .../part-1/armc-03/kernel.armc-03.rpi0.elf
.../compiler/gcc-arm-none-eabi-7-2018-q2-update/bin/../lib/gcc/arm-none-eabi/7.3.1/../../../../arm-none-eabi/bin/ld: warning: cannot find entry symbol _start; defaulting to 0000000000008000
.../compiler/gcc-arm-none-eabi-7-2018-q2-update/bin/arm-none-eabi-objcopy .../part-1/armc-03/kernel.armc-03.rpi0.elf -O binary .../part-1/armc-03/kernel.armc-03.rpi0.img

Can you give the output of ./arm-none-eabi-gcc --version for the compiler you're using?

We should probably move the compiler forward for the tutorial at some point, 7 is a bit old these days.

rwdavis3rd commented

I'm torn between responding now, with what I know, or waiting longer (too long?) to respond after more complete research. I'll share what I (think I) know at this point, and expand further, as I learn more.

First, I was not using the same compiler as you. I went and found a version titled arm-2008q3 (version 8.2.0) and installed that on my Linux box for use. After I ran into differences from your experience, however, I re-checked and found, due to unexpected $PATH ordering, that I was running a still-different version of ARM cross-compiler which Xilinx supplies as part of their MPSoC Ultrascale Plus SDK at SDK/2019.1/gnu/aarch32/lin/gcc-arm-none-eabi/bin/gcc-arm-none-gcc. This version also reports that it's version 8.2.0. But it acts different (?!)

I corrected paths, so I was back at the version titled arm-2008q3, and hit a new problem. When building (say) part-2/armc-09 the compiler 2008-q3 says "sorry, unimplemented: -mfloat-abi=hard and VFP" ... this later element likely requested via the "-mfpu=vfp" parameter on compiles.

Switching back to the Xilinx-supplied compiler, this error stops, but I get the section .note.gnu.build-id as the first section in my image, which pushes _start to 0x8024 (which therefore won't boot.) That's the problem that started this whole dialog. :-)

My on-going compromise has been to modify the rpi.x script found in one of your tutorials, to push section .note.gnu.build-id to the bottom of the image, and to specify its use by the linker thru the -Wl,T,${scriptdir}/rpi.x option, both in build.sh and in later rpi0 cmake files, and then to continue use of the Xilinx-distributed compiler, which supports hardware floating point and VFP simultaneously.

Here's the --version output from that compiler

armc-09$ `which arm-none-eabi-gcc` --version
arm-none-eabi-gcc (GCC) 8.2.0
Copyright (C) 2018 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Cheers!

BrianSidebotham commented

OK - this tutorial uses a hardware floating point Application Binary Interface. The compiler you've picked up (arm-2008q3) doesn't support the hard floating point ABI. You can omit the ABI reference if you want, but you'll have to do that throughout the entire tutorial.

The best solution is the recommended approach - use the compiler/get_compiler.sh script in the tutorial to get the tutorial's recommended (and tested) toolchain. Things will get more and more painful for you otherwise. This will also ensure you don't have path issues anymore because the tutorial doesn't need to add this compiler to your path.

rwdavis3rd commented

Thanks for turning your experienced eye towards this. I'll switch to the recommended compiler.

If / when you move forward from --version 7.x, beware the dreaded .note.gnu.build-id section! :-)

Cheers!

[ ps: My cheap-and-cheerful, bit-bang JTAG controller arrives this week. Want to share notes? ]

BrianSidebotham commented

The 7.x series had got a bit long in the tooth.

I've setup Travis to now correctly at least build all tutorials for all raspberry pi versions (There were a few fixes committed because of that).

That enables us to more confidently move forward with testing new compilers.

I've created an issue ( #24 ) in order to look at moving the compiler forward.

BrianSidebotham commented

@rwdavis3rd I've not been able to re-create this and the compiler has moved forward to 9.2+ with no issues so I'm going to close this issue.

If you experience it again or have any further info, please open the issue again. Thanks.