ARMmbed/uvisor

mbedTLS unable to write to RTC causing `Access to restricted resource denied` bus fault and halt

Closed this issue · 5 comments

sherrellbc commented

During the TLS handshaking processes, mbedTLS attempts to make use of a timer. If the RTC is not yet enabled, the default behavior of mbed-os is to initialize the hardware. This, in turn, causes a read/write to the RTC peripheral located at 0x4003D000 (on the K64F FRDM board). However, this cause an unhandled bus fault that results in a halt:

***********************************************************
                    BUS FAULT
***********************************************************

* Active Box ID: 0
* FAULT SYNDROME REGISTERS

  CFSR: 0x00008200
  BFAR: 0x4003D010
  --> PRECISERR: precise data access.

* No MPU violation found

* MEMORY MAP
  Address:           0x4003D010
  Region/Peripheral: [not available]
    Base address:    0x4003D010
    End address:     0x4003D010
    AIPS slot:       61

* EXCEPTION STACK FRAME

  lr: 0xFFFFFFFD
  --> FP stack frame not saved.
  --> Exception from NP mode.
  --> Return to PSP.

  sp: 0x2001B4D8
      sp[07]: 0x210F0000 | xPSR
      sp[06]: 0x000559F6 | pc
      sp[05]: 0x00055A47 | lr
      sp[04]: 0x0004FAA5 | r12
      sp[03]: 0x4003D000 | r3
      sp[02]: 0x00000001 | r2
      sp[01]: 0x60800001 | r1
      sp[00]: 0x4003D000 | r0


* MPU CONFIGURATION
  CESR: 0x00815101
       Slave 0    Slave 1    Slave 2    Slave 3    Slave 4
  EAR: 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
  EDR: 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
       Start      End        Perm.      Valid
  R00: 0x00000000 0xFFFFFFFF 0x000007D0 0x00000001
  R01: 0x00000000 0x0007539F 0x0010400D 0x00000001
  R02: 0x20000000 0x2003001F 0x00186017 0x00000001
  R03: 0x1FFF2080 0x1FFF249F 0x0000001E 0x00000000
  R04: 0x1FFF2500 0x1FFF525F 0x0000001E 0x00000000
  R05: 0x00000000 0x0000001F 0x00000000 0x00000000
  R06: 0x00000000 0x0000001F 0x00000000 0x00000000
  R07: 0x00000000 0x0000001F 0x00000000 0x00000000
  R08: 0x00000000 0x0000001F 0x00000000 0x00000000
  R09: 0x00000000 0x0000001F 0x00000000 0x00000000
  R10: 0x00000000 0x0000001F 0x00000000 0x00000000
  R11: 0x00000000 0x0000001F 0x00000000 0x00000000

***********************************************************

HALT_ERROR(./core/vmpu/src/mpu_kinetis/vmpu_kinetis.c#161): Access to restricted resource denied.

Below is the run-time semihosting output from uVisor. As you can see, the RTC peripheral is being added to our MPU configuration for box 0.

uvisor_ram : @0x1FFF0400 (7168 bytes) [config]
             @0x1FFF0400 (7168 bytes) [linker]
bss_boxes  : @0x1FFF2000 (32896 bytes) [config]
Box 0 ACLs:
  - Peripheral: 0x40047000 - 0x40048064 (permissions: 0x0AB6)
  - Peripheral: 0x40037000 - 0x40037140 (permissions: 0x0AB6)
  - Peripheral: 0x40065000 - 0x40065001 (permissions: 0x0AB6)
  - Peripheral: 0x40064000 - 0x4006400E (permissions: 0x0AB6)
  - Peripheral: 0x4007E000 - 0x4007E004 (permissions: 0x0AB6)
  - Peripheral: 0x4003D000 - 0x4003D808 (permissions: 0x0AB6)
  - Peripheral: 0x40029000 - 0x40029010 (permissions: 0x0AB6)
  - Peripheral: 0x4006A000 - 0x4006A020 (permissions: 0x0AB6)
  - Peripheral: 0x400C0000 - 0x400C0628 (permissions: 0x0AB6)
  - Peripheral: 0x40049000 - 0x400490CC (permissions: 0x0AB6)
  - Peripheral: 0x4004A000 - 0x4004A0CC (permissions: 0x0AB6)
  - Peripheral: 0x4004B000 - 0x4004B0CC (permissions: 0x0AB6)
Box 1 ACLs:
  - Stack:      0x1FFF2080 - 0x1FFF2480 (permissions: 0x0076)
  - BSS:        0x1FFF2500 - 0x1FFF5240 (permissions: 0x00B6)
  - Peripheral: 0x4004D000 - 0x4004D0CC (permissions: 0x0AB6)
  - Peripheral: 0x40066000 - 0x4006600C (permissions: 0x0AB6)
Box 2 ACLs:
  - Stack:      0x1FFF52C0 - 0x1FFF56C0 (permissions: 0x0076)
  - BSS:        0x1FFF5740 - 0x1FFF8480 (permissions: 0x00B6)
vmpu_enumerate_boxes [DONE]

Eventually I came across this comment about bus faults on the K64F originating from special register read/write attempts outside of supervisor mode. RTC does indeed appear to be one of these 'corner cases'; section 44.2 of the reference manual informs us:

Write accesses to any register by non-supervisor mode software, when the supervisor
access bit in the control register is clear, will terminate with a bus error.

Read accesses by non-supervisor mode software complete as normal.

Writing to a register protected by the write access register or lock register does not
generate a bus error, but the write will not complete.

Reading a register protected by the read access register does not generate a bus error, but
the register will read zero.

I stepped through the offending code and found that the RTC registers can be read from, but a write will cause a fault. This code must be executing at a higher privilege level than most boxes, because even a read will fault from box context.

Was the RTC intended to remain a protected resource? There appears to be a bit of instruction processing while handling the bus fault, but the RTC case is left to continue faulting.

A backtrace for more insight:

#0  RTC_Init (base=0x4003d000, config=0x2001b51c)
    at ../mbed-os/targets/TARGET_Freescale/TARGET_MCUXpresso_MCUS/TARGET_MCU_K64F/drivers/fsl_rtc.c:210
#1  0x0005748a in rtc_init () at ../mbed-os/targets/TARGET_Freescale/TARGET_MCUXpresso_MCUS/api/rtc_api.c:31
#2  0x0004cd4a in set_time (t=0) at ../mbed-os/platform/mbed_rtc_time.cpp:71
#3  0x0004ccdc in time (timer=0x0 <__isr_vector>) at ../mbed-os/platform/mbed_rtc_time.cpp:52
#4  0x0003d140 in ssl_write_server_hello (ssl=0x20006a38 <emsg+1296>) at ../mbed-os/features/mbedtls/src/ssl_srv.c:2262
#5  0x0003e8a8 in mbedtls_ssl_handshake_server_step (ssl=0x20006a38 <emsg+1296>)
    at ../mbed-os/features/mbedtls/src/ssl_srv.c:3851
#6  0x00044ac2 in mbedtls_ssl_handshake_step (ssl=0x20006a38 <iemsg+1296>) at ../mbed-os/features/mbedtls/src/ssl_tls.c:6328
#7  0x00044b0e in mbedtls_ssl_handshake (ssl=0x20006a38 <emsg+1296>) at ../mbed-os/features/mbedtls/src/ssl_tls.c:6348
#8  0x000581d4 in _enet_mbedtls_listen (emsg=0x20006528 <emsg>) at ../src/enet_msg.cpp:459
#9  0x00058464 in enet_handler_entry (arg=0x20006518 <msg>) at ../src/enet_msg.cpp:605
#10 0x00059176 in mbed::Callback<void ()>::function_context<void (*)(void const*), void const>::operator()() const (this=0x200155b4)
    at ../mbed-os/platform/Callback.h:636
#11 0x00059124 in mbed::Callback<void ()>::function_call<mbed::Callback<void ()>::function_context<void (*)(void const*), void const>
>(void const*) (p=0x200155b4) at ../mbed-os/platform/Callback.h:600
#12 0x0000d3a0 in mbed::Callback<void ()>::call() const (this=0x200155b4) at ../mbed-os/platform/Callback.h:527
#13 0x0000d2d4 in mbed::Callback<void ()>::operator()() const (this=0x200155b4) at ../mbed-os/platform/Callback.h:533
#14 0x0004d324 in rtos::Thread::_thunk (thread_ptr=0x200155b0) at ../mbed-os/rtos/Thread.cpp:351
#15 0x00051be8 in osThreadYield () at ../mbed-os/rtos/rtx5/TARGET_CORTEX_M/rtx_thread.c:1607
#16 0x2001b750 in __uvisor_heap_start ()

Also noteworthy: Attempting to read the RTC Control Register (0x4003D010) from within uVisor is also not handled. I attempted to read this register from uvisor_init_post just after "uvisor initialized" is printed to the debug window. The result was a hard fault:

***********************************************************
                    HARD FAULT
***********************************************************

* Active Box ID: 0
* FAULT SYNDROME REGISTERS

  HFSR: 0x40000000
  --> FORCED: another priority escalated to hard fault.

* EXCEPTION STACK FRAME

  lr: 0xFFFFFFF1
  --> FP stack frame not saved.
  --> Exception from P mode.
  --> Return to MSP.

  sp: 0x1FFF1F28
      sp[07]: 0x01000005 | xPSR
      sp[06]: 0x00006616 | pc
      sp[05]: 0x00002017 | lr
      sp[04]: 0x1FFF0400 | r12
      sp[03]: 0x1FFF1FB8 | r3
      sp[02]: 0x0000656C | r2
      sp[01]: 0x1FFF1FA0 | r1
      sp[00]: 0xFFFFFFF9 | r0


* MPU CONFIGURATION
  CESR: 0x20815101
       Slave 0    Slave 1    Slave 2    Slave 3    Slave 4
  EAR: 0x00000000 0x00000000 0x1FFF1FB8 0x00000000 0x00000000
  EDR: 0x00000000 0x00000000 0x80000002 0x00000000 0x00000000
       Start      End        Perm.      Valid
  R00: 0x00000000 0xFFFFFFFF 0x000007D0 0x00000001
  R01: 0x00000000 0x000753DF 0x0010400D 0x00000001
  R02: 0x20000000 0x2003001F 0x00186017 0x00000001
  R03: 0x00000000 0x0000001F 0x00000000 0x00000000
  R04: 0x00000000 0x0000001F 0x00000000 0x00000000
  R05: 0x00000000 0x0000001F 0x00000000 0x00000000
  R06: 0x00000000 0x0000001F 0x00000000 0x00000000
  R07: 0x00000000 0x0000001F 0x00000000 0x00000000
  R08: 0x00000000 0x0000001F 0x00000000 0x00000000
  R09: 0x00000000 0x0000001F 0x00000000 0x00000000
  R10: 0x00000000 0x0000001F 0x00000000 0x00000000
  R11: 0x00000000 0x0000001F 0x00000000 0x00000000

***********************************************************

HALT_ERROR(./core/vmpu/src/mpu_kinetis/vmpu_kinetis.c#77): Cannot recover from a hard fault.
ciarmcom commented

ARM Internal Ref: IOTSEC-401

Patater commented

The RTC peripheral may be one of those K64F peripherals that cannot be accessed from unprivileged code, even when there is an ACL for the peripheral.

One thing to check

  • Is the RTC clocked before its registers are accessed?

Consider using register-level gateways to modify registers belonging to peripherals that cannot be accessed by unprivileged code.

sherrellbc commented

@Patater

The RTC is a protected peripheral per section 44.2 of the reference manual. There is a Supervisor Access bit in the RTC Control Register (RTC->CR.SUP) that controls whether or not bus fault exceptions are generated on write access from unprivileged (non supervisor) mode.

Using the hints provided, I found that the initial faults (my early read attempts) were triggering execptions because the RTC clock was gated in the System Integration Module (SIM). Setting bit 29 in the SIM->SCGC6 ungated the RTC clock and reads were thenceforth permitted. However, the RTC driver was written assuming it was executing with supervisor privileges, so run-time exceptions were still being generated. The default behavior of the driver is to ungate the clock and reset the RTC. However, this code is not executing as a supervisor and will cause a bus fault exception to be generated at run-time.

To test this, I patched uVisor to ungate the RTC clock and allow non-supervisor write access to the RTC registers (by setting RTC->CR.SUP). This nearly worked, but the RTC_Reset function effectively undoes this action by setting RTC->CR.SWR, which resets the register state. So, the next write attempt will fail with the same bus fault as before.

I was able to overcome this by using SECURE_WRITE and patching the RTC driver to skip the RTC->CR.SWR step so we do not lose write access. So, in summary, this seems to be more of an issue with mbed-os than uVisor. The driver should be updated to work with the register-level gateways, assuming this is the best path forward for uVisor support. However, this will have to wait until a dynamic alternative of uvisor_{read|write} is available.

Patater commented

Yes, the driver does need updating to work better with uVisor.

There will be no dynamic version of uvisor_{read|write}. Doing a SECURE_WRITE (which does uvisor_write(public_box, ...) will allow every box to do the secure write if they jump to the location of the gateway.

sherrellbc commented

Closing as this is an issue with mbed-os' RTC driver, not uVisor.